I am working on speedystride.com, a programming tool that helps athletes quickly input workouts on their Apple and Garmin watches.
These watches come with a built-in workout programming feature that is especially useful for structured programs. For example, runners will often do interval training, which could be something like 5x1000m with 2 minute rest.
And sometimes they’ll want to do a fartlek (Swedish for ‘Speed Play’) where they will vary their speed: run 400 meters fast - run 800 meters slower - sprint 200 meters. These smartwatches will vibrate and beep to help the user perform at the desired target, and also count down rest periods so the user is rested enough for that next hard interval.
Unfortunately, I did not like any of the first party workout builders. These are form-based, with a drag-and-drop interface to structure your workouts. I think these builders have a high user friction; more user inputs are required in proportion to the output. Additionally, these builders run on a small watch screen, or require a separate app. This is less than ideal when you are trying to program your watch right before a track workout. There are third party tools in this space, but as far as I can tell, they do not fundamentally break this pattern.
I also wanted to share these workouts with everyone at my city’s track club. My service provides a scheduler that pushes workouts automatically at a specified time for our club training; about 90% of our members have either an Apple Watch or a Garmin so cross-platform compatibility is a very important factor.
To solve these problems, I came up with a very simple domain specific language (DSL) for both people and machines. It can describe exercises, define rest times, and combine everything together into repeat intervals. I implemented a simple recursive descent parser, and it outputs data formats for both Apple and Garmin devices. By defining a small language, I was able to avoid implementing complex forms unlike the current offerings. User input is reduced to plain text.
User:
10x200m max effort with 2 minute rest
DSL:
Repeat 10 times:
- Run 200m @RPE 10
- Rest 2 minutes
I had initially wanted coaches to learn this DSL and enter programs into my website assisted by a Codemirror editor. I incorrectly thought that it was close enough to English for people to quickly learn it when assisted by autocomplete features. I was not meeting my users where they were at; graybeard track coaches had zero interest in learning how to program. What I needed was a translator that could convert natural English into my DSL.
As I started sharing my project with other people, large language models were becoming popular, and it was an obvious tool to translate natural English workouts into my training DSL. By integrating LLMs, I can massively reduce user friction. There are no forms with complex UIs to implement. There is no DSL to learn as the AI can translate natural language for you. Users can now express their workouts in their own way.
An AI can transform the above user input to this Domain Specific Language with a relatively small language specification. There was also a nice side effect of being very token efficient. JSON payloads defining a repeated workout set can get quite large while my DSL can stay compact. Any errors can be corrected as my parser can provide rich feedback on what went wrong. I have found that 95% of interval workouts I see can be expressed through my language.
LLMs also enable new capabilities, such as programming your watch from a photo of a whiteboard. Even more importantly, Model Context Protocol (MCP) was starting to gain traction. MCPs are a way for LLM systems to interact with the real world, which means that besides just outputting workout programs, the LLM can call a remote function to actually send that workout to your device.
Anthropic and OpenAI both support MCP. So it would be awesome for my business to support LLM integrations since so many users already have Claude and ChatGPT installed on their phone.
Still, there were opportunities to further improve user experience.
I had mentioned earlier that my track club uses speedystride.com to program members’ watches. In order to do so, we have to define a few parameters:
What is the workout?
Should the workout be added to my Monday night track intervals, or for Tuesday fartleks?
LLMs can help massively with the first question. But how about the second? Much of the current human-to-LLM interaction is text-based, and MCPs are no exception. Besides improving workout building UX, AI tools introduced new frictions. To associate a workout with an event, I had an events tool that would fetch upcoming events for the user and add them to the LLM context. Then it was up to the LLM to guide the user. Some systems like Claude do provide simple select controls if your tools output JSON objects that look like a set of choices. However, this interaction forces the developer to surrender control of the happy path, which often leaves users confused. Also, the AI would sometimes try to be too helpful and just guess the tool inputs. In summary, back and forth conversation with the LLM is not an ideal UX as the users have to figure out how to guide the AI to the right inputs.
A form with a selector interface is an obvious way to solve this problem.
Luckily for me, the new MCP Apps specification was released in January 2026. This is an extension to the MCP specification that allows rendering custom UI inside an `` of the MCP Host.
References:
You need to host an MCP server that can communicate with the AI systems.
MCP Server LLM Host (Claude or ChatGPT) MCP App UI (rendered inside LLM )
All traffic between the MCP App UI and the LLM host must be routed through the App Bridge. The LLM host will then make proxied requests to my MCP server.
Let’s say that we are developing a simple workout scheduler, where we have the LLM generated workout program. Our goal is to associate this workout with a calendar event occurrence. A user could have multiple events on her calendar, so a dynamic choice of occurrences should be available for each event she is subscribed to. On a traditional website, this could be trivially handled by a full page refresh.
On MCP App systems without interactivity, we would have to ask the LLM to fully render the MCP App in the chat. This adds friction and unnecessarily consumes tokens. So we must find a path to interactivity within the same UI context.
There are existing UI toolkits, such as MCP-UI that works really well with React. However, using React for a simple `` felt too complex, so I wanted a hypermedia solution.
I initially considered using HTMX, since my HTTP site uses it already. The challenge was that HTMX is heavily geared towards processing HTTP requests. Since my MCP App UI renders inside an ``, I don’t need to push URLs or manage history.
Then I remembered Fixi library from the creator of HTMX. It is a minimal version of generalized hypermedia controls. Fixi proved to be very useful precisely because it doesn’t do much. In addition to lacking history or URL push features, I can also copy & paste Fixi to my project to skip build steps. I also don’t have to worry about CSP configuration if my Fixi code is served with my HTML.
The killer feature of Fixi is its hackability.
Because it expects a Response compatible object (or a Promise) rather than a strict network request, we can entirely
bypass HTTP.
I can configure Fixi to call app.callServerTool when it sees any fx-action that starts with tool:. If I use a
``, its inputs will become the function args.
Let’s examine how to architect MCP features to serve hypermedia.
From MCP docs:
Create: Instantiate App with info and capabilities
Connect: Call connect() to establish transport and perform handshake
Interactive: Send requests, receive notifications, call tools
Cleanup: Host sends teardown request before unmounting
We will focus on items 2 and 3 to demonstrate a hypermedia driven MCP App.
I host my server on a Gunicorn + Uvicorn monolith. ASGI Django handles regular HTTP traffic, and MCP traffic is routed to FastMCP. Since both Django and FastMCP work together, I can share resources between the HTTP and MCP domains including ORM and template rendering.
The LLM host will render a UI by calling an MCP tool and its associated resource.
A tool is similar to a view- it is a function that can return JSON or hypertext. Let’s say that our UI tool is
called show_user_ui.
A resource is a bit like a pointer to assets. LLM hosts can preload resources to deliver them more quickly to users.
Tools and resources are registered by @mcp.tool or @mcp.resource decorators. You could think of registration as
defining urls.py in Django.
Since our Django and FastMCP Applications live on the same server, we can render HTML with Django’s render_to_string
function with ORMs and templatetags.
Django 6’s
new built-in template partials also
make
template organization and partial rendering easy. This allows us to co-locate our initial UI render and our dynamic
hypermedia fragments in a single file, keeping the MCP tool logic incredibly clean.
Let’s talk about the resource first. It points to a HTML where our Fixi and MCP App Bridge code will be placed.
from django.template.loader import render_to_string
@mcp.resource(
"ui://user_ui_resource.html",
mime_type="text/html;profile=mcp-app",
description="User UI resource",
)
async def user_ui_resource():
return render_to_string("mcp/user_ui_resource.html")
This resource serves HTML rendered with render_to_string to resolve static files and template tags, but not any
user-specific data.
mcp/user_ui_resource.html
html>
head>
{% include "fixi, app bridge source code, styles, etc" %}
style>
/* CSS indicator classes to toggle visibility during requests */
#indicator {
display: none;
}
#indicator.fixi-request-in-flight {
display: inline-block;
}
style>
script type="module">
...
MCP
App
bridge
setup
...
// 1. Configure Fixi to route `tool:` fx-actions through MCP App bridge
document.addEventListener("fx:config", (evt) => {
const action = evt.detail.cfg.action;
if (action.startsWith("tool:")) {
const toolName = action.replace("tool:", "");
console.log(`callServerTool: ${toolName}`);
const args = Object.fromEntries(evt.detail.cfg.body ?? []);
evt.detail.cfg.fetch = async () => {
const result = await app.callServerTool({name: toolName, arguments: args});
return {text: async () => result.structuredContent?.html};
};
}
});
// 2. Set up request indicator extension: ext-fx-indicator
document.addEventListener("fx:init", (evt) => {
if (evt.target.matches("[ext-fx-indicator]")) {
let disableSelector = evt.target.getAttribute("ext-fx-indicator")
evt.target.addEventListener("fx:before", () => {
let disableTarget = disableSelector === "" ? evt.target : document.querySelector(disableSelector)
disableTarget.classList.add("fixi-request-in-flight")
evt.target.addEventListener("fx:after", (afterEvt) => {
if (afterEvt.target === evt.target) {
disableTarget.classList.remove("fixi-request-in-flight")
}
})
})
}
});
// 3. Populate UI on initial render with dynamic content fetched with `show_user_ui` tool
app.ontoolresult = (params) => {
document.body.innerHTML = params.structuredContent?.html ?? document.body.innerHTML;
};
script>
head>
{# Empty body, since `app.ontoolresult` will populate it on load #}
body>body>
{% partialdef save-form-fragment %}
div id="save-form-fragment">
Workout {{ workout.id }} has been saved for {{ workout.occurrence.event.id }} on {{ workout.occurrence.start_date
}}.
div>
{% endpartialdef %}
{% partialdef main-contents %}
div id="main-contents">
form>
input name="workout_dsl" type="hidden" value="{{workout_dsl}}">
select
id="event-selector"
name="event_id"
fx-action="tool:render_occurrences_fragment"
fx-target="#occurrence-fragment"
fx-swap="outerHTML"
>
option>-----option>
{% for evt in events %}
option value="{{ evt.id }}">{{ evt.name }}option>
{% endfor %}
select>
{% partialdef occurrence-fragment inline %}
select id="occurrence-fragment" name="occurrence_id">
{% for occ in occurrences %}
option value="{{ occ.id }}">{{ occ.start_date }}option>
{% endfor %}
select>
{% endpartialdef %}
button
type="submit"
fx-action="tool:save_form_fragment"
fx-swap="outerHTML"
fx-target="#main-contents"
ext-fx-indicator
>
span>Save Workoutspan>
svg id="indicator">...svg>
button>
form>
div>
{% endpartialdef %}
html>
This HTML resource defines three core pieces of logic within its `` tag:
Event listener for fx:config: We can use fx-action attribute to make MCP tool calls.
Event listener for fx:init: Set up an indicator to show that a tool call is being processed.
Handle app.ontoolresult: Display the main UI with the output by processing CallToolResult.
The LLM will load the resource in the chat, and then call show_user_ui tool, which renders the main-contents
template partial.
from django.template.loader import render_to_string
from mcp.server.fastmcp import FastMCP
from mcp.server.fastmcp import Context
from mcp.types import CallToolResult
from events.models import Events, Occurrences
from workout_validator import validate_program, DSLValidationError
mcp = FastMCP(...)
@mcp.tool(
name="show_user_ui",
description="Display UI",
meta={"ui/resourceUri": "ui://user_ui_resource.html"}
)
async def show_user_ui(ctx: Context, workout_dsl: str) -> CallToolResult:
user = await get_user_from_context(ctx)
try:
validate_workout = validate_program(workout_dsl)
except DSLValidationError as e:
...
handle
invalid
workout
data...
initial_events_qs = Events.objects.filter(user=user)
initial_events = [evt async for evt in initial_events_qs.aiterator()]
if len(initial_events) > 0:
initial_occurrences_qs = Occurrences.objects.select_related("event").filter(
event_id=initial_events[0]).order_by("start_date")
initial_occurrences = [occ async for occ in initial_occurrences_qs.aiterator()]
else:
initial_occurrences = []
template_context = {"user": user, "workout_dsl": workout_dsl, "events": initial_events,
"occurrences": initial_occurrences}
rendered_html = render_to_string("mcp/user_ui_resource.html#main-contents", template_context)
return CallToolResult(
content=[TextContent(type="text", text="UI ready")],
structuredContent={"html": rendered_html}
)
Args for show_user_ui:
ctx: object will allow us to authenticate our users, and is passed to our tool by the LLM host.
workout_dsl: AI will generate and pass this parameter. We will render our HTML with this data stored in
`` and save it later.
Return values from show_user_ui:
content: An array of text or other data to show the User/LLM
structuredContent: Rendered hypertext
MCP’s ontoolresult handler will insert structuredContent.html into `` tag to render our initial UI.
Note that this innerHTML assignment is reasonably safe in our context. The only untrusted input here is AI generated
workout_dsl, and we run validation on it before rendering our template.
Now we have a full with two controls. How do we add interactivity for our event occurrence
selectors?
Every time the user changes #event-selector option, we should update our options shown in #occurrence-fragment with
a new tool.
We need to use template fragments to fetch occurrence options for different events without triggering another UI render.
Let’s first examine how Fixi will trigger a fragment request. Scroll back above and read #event-selector defined in
#main-contents.
We see these Fixi attributes:
fx-action: Calls render_occurrences_fragment MCP tool. Fixi will bind fx-action to appropriate default events,
such as change for ``.
fx-target: Insert render_occurrences_fragment tool’s HTML output in the #occurrence-fragment div
fx-swap: Use outerHTML swap on #occurrence-fragment, which replaces this div instead of inserting contents.
Below shows occurrence HTML fragment rendering occurrence-fragment Django template partial. This tool’s output will
replace #occurrence-fragment.
from events.models import Occurrences
@mcp.tool(
name="render_occurrences_fragment",
description="Fetches event occurrences available to the logged in user",
meta={"ui": {"visibility": ["app"]}}
)
async def render_occurrences_fragment(ctx: Context, event_id) -> CallToolResult:
...
validate
event_id and authenticate
user...
occurrences_qs = Occurrences.objects.select_related("event").filter(
event_id=event_id
).order_by("start_date")
occurrences = [occ async for occ in occurrences_qs.aiterator()]
rendered_html = render_to_string("mcp/user_ui_resource.html#occurrence-fragment", {"occurrences": occurrences})
return CallToolResult(
text="Here are this user's event occurrences",
structuredContent={"html": rendered_html}
)
It does not make sense for LLM to fetch this HTML fragment by itself, so we can set
meta={"ui": {"visibility": ["app"]}} registration parameter to prevent unnecessary tool calling.
Finally, let’s submit this form. This action is triggered by , and all of the fields will be sent to
save_form_fragment as function args.
from workouts.models import Workout
from events.models import Events, Occurrences
@mcp.tool(
name="save_form_fragment",
description="Saves workout",
meta={"ui": {"visibility": ["app"]}}
)
async def save_form_fragment(ctx: Context, workout_dsl, event_id, occurrence_id) -> CallToolResult:
...
validate
args and authenticate
user...
occurrence = await Occurrences.objects.select_related("event").aget(id=occurrence_id, event_id=event_id)
workout = await Workout.objects.acreate(occurrence=occurrence, workout_dsl=workout_dsl)
# Render the events fragment
rendered_html = render_to_string("mcp/user_ui_resource.html#save-form-fragment", {"workout": workout})
return CallToolResult(
text="Workout saved",
structuredContent={"html": rendered_html}
)
The submit indicator is defined by configuring Fixi event fx:init to add .fixi-request-in-flight class to
#indicator SVG inside our submit button if it has ext-fx-indicator attribute.
After this, the interaction cycle is finished. If the user wants to make modifications, she can ask the AI to render a
new UI or visit the website to make quick changes.
Here is a demo of this application in action:
I hope to have demonstrated a simple way to develop user interfaces that work well with AI. MCP Apps introduce a new rendering environment, but hypermedia systems continue to work well in this context with some modifications.
This is worth emphasizing because the current zeitgeist when building with AI is to reach for a client-side framework. But the constraints of MCP Apps actually push in the opposite direction. Your `` cannot talk directly to your server and every interaction must cross the bridge. The less state and logic you pack into the client, the less surface area you have for things to go wrong across that boundary.
Throughout my design process, I tried to channel Nintendo’s Gunpei Yokoi: What can we do with old-fashioned technology using lateral thinking? I was able to sidestep complex drag-and-drop form builders by combining a small DSL with AI and hypermedia. Integrating MCP Apps solved the friction of coaxing the AI to select the right inputs, and using hypermedia made the entire system almost trivially simple: forms, selectors, fragment updates, loading indicators. Simplicity does not guarantee success, but I think that it will give me a better fighting chance.
Special thanks to Carson Gross for creating Fixi, HTMX, and Hyperscript, and for encouraging me to write this post.
I teach computer science at Montana State University. I am the father of three sons who all know I am a computer programmer and one of whom, at least, has expressed interest in the field. I love computer programming and try to communicate that love to my sons, the students in my classes and anyone else who will listen.
A question I am increasingly getting from relatives, friends and students is:
Given AI, should I still consider becoming a computer programmer?
My response to this is: “Yes, and…”
Computer programming is, fundamentally, about two things:
Problem-solving using computers
Learning to control complexity while solving these problems
I have a hard time imagining a future where knowing how to solve problems with computers and how to control the complexity of those solutions is less valuable than it is today, so I think it will continue to be a viable career even with the advent of AI tools.
That being said, I view AI as very dangerous for junior programmers because it is able to effectively generate code for many problems. If a junior programmer does not learn to write code and simply generates it, they are robbing themselves of the opportunity to develop the visceral understanding of code that comes with being down in the trenches.
Because of this, I warn my students:
“Yes, AI can generate the code for this assignment. Don’t let it. You have to write the code.”
I explain that, if they don’t write the code, they will not be able to effectively read the code. The ability to read code is certainly going to be valuable, maybe more valuable, in an AI-based coding future.
If you can’t read the code you are going to fall into The Sorcerer’s Apprentice Trap, creating systems you don’t understand and can’t control.
Some people say that the move from high level languages to AI-generated code is like the move from assembly to high level programming languages.
I do not agree with this simile.
Compilers are, for the most part, deterministic in a way that current AI tools are not. Given a high-level programming language construct such as a for loop or if statement, you can, with reasonable certainty, say what the generated assembly will look like for a given computer architecture (at least pre-optimization).
The same cannot be said for an LLM-based solution to a particular prompt.
High level programming languages are a very good way to create highly specified solutions to problems using computers with a minimum of text in a way that assembly was not. They eliminated a lot of accidental complexity, leaving (assuming the code was written reasonably well) mostly necessary complexity.
LLM generated code, on the other hand, often does not eliminate accidental complexity and, in fact, can add significant accidental complexity by choosing inappropriate approaches to problems, taking shortcuts, etc.
If you can’t read the code, how can you tell?
And if you want to read the code you must write the code.
Another thing that I tell my students is that AI, used properly, is a tremendously effective TA. If you don’t use it as a code-generator but rather as a partner to help you understand concepts and techniques, it can provide a huge boost to your intellectual development.
One of the most difficult things when learning computer programming is getting “stuck”. You just don’t see the trick or know where to even start well enough to make progress.
Even worse is when you get stuck due to accidental complexity: you don’t know how to work with a particular tool chain or even what a tool chain is.
This isn’t a problem with you, this is a problem with your environment. Getting stuck pointlessly robs you of time to actually be learning and often knocks people out of computer science.
(I got stuck trying to learn Unix on my own at Berkeley, which is one reason I dropped out of the computer science program there.)
AI can help you get past these roadblocks, and can be a great TA if used correctly. I have posted an AGENTS.md file that I provide to my students to configure coding agents to behave like a great TA, rather than a code generator, and I encourage them to use AI in this role.
AI doesn’t have to be a detriment to your ability to grow as a computer programmer, so long as it is used appropriately.
I do think AI is going to change computer programming. Not as dramatically as some people think, but in some fundamental ways.
It may be that the act of coding will lose relative value.
I regard this as too bad: I usually like the act of coding, it is fun to make something do something with your (metaphorical) bare hands. There is an art and satisfaction to writing code well, and lots of aesthetic decisions to be made doing it.
However, it does appear that raw code writing prowess may be less important in the future.
As this becomes relatively less important, it seems to me that other skills will become more important.
For example, the ability to write, think and communicate clearly, both with LLMs and humans seems likely to be much more important in the future. Many computer programmers have a literary bent anyway, and this is a skill that will likely increase in value over time and is worth working on.
Reading books and writing essays/blog posts seem like activities likely to help in this regard.
Another thing you can work on is turning some of your mental energy towards understanding a business (or government role, etc) better.
Computer programming is about solving problems with computers and businesses have plenty of both of these.
Some business folks look at AI and say “Great, we don’t need programmers!”, but it seems just as plausible to me that a programmer might say “Great, we don’t need business people!”
I think both of these views are short-sighted, but I do think that AI can give programmers the ability to continue fundamentally working as a programmer while also investing more time in understanding the real-world problems (business or otherwise) that they are solving.
This dovetails well with improving communication skills.
Like many computer programmers, I am ambivalent towards the term “software architect.” I have seen architect astronauts inflict a lot of pain on the world.
For lack of a better term, however, I think software architecture will become a more important skill over time: the ability to organize large software systems effectively and, crucially, to control the complexity of those systems.
A tough part of this for juniors is that traditionally the ability to architect larger solutions well has come from experience building smaller parts of systems, first poorly then, over time, more effectively.
Most bad architects I have met were either bad coders or simply didn’t have much coding experience at all.
If you let AI take over as a code generator for the “simple” stuff, how are you going to develop the intuitions necessary to be an effective architect?
This is why, again, you must write the code.
Another skill that seems likely to increase in value (obviously) is knowing how to use LLMs effectively. I think that currently we are still in the process of figuring out what that means.
I also think that what this means varies by experience level.
Senior programmers who already have a lot of experience from the pre-AI era are in a good spot to use LLMs effectively: they know what “good” code looks like, they have experience with building larger systems and know what matters and what doesn’t. The danger with senior programmers is that they stop programming entirely and start suffering from brain rot.
Particularly dangerous is firing off prompts and then getting sucked into The Eternal Scroll while waiting.
Ask me how I know.
I typically try to use LLMs in the following way:
To analyze existing code to better understand it and find issues and inconsistencies in it
To help organize my thoughts for larger projects I want to take on
To generate relatively small bits of code for systems I am working on
To generate code that I don’t enjoy writing (e.g. regular expressions & CSS)
To generate demos/exploratory code that I am willing to throw away or don’t intend to maintain deeply
To suggest tests for a particular feature I am working on
I try not to use LLMs to generate full solutions that I am going to need to support. I will sometimes use LLMs alongside my manual coding as I build out a solution to help me understand APIs and my options while coding.
I never let LLMs design the APIs to the systems I am building.
Juniors are in a tougher spot. I will say it again: you must write the code.
The temptation to vibe your way through problems is very, very high, but you will need to fight against that temptation.
Peers will be vibing their way through things and that will be annoying: you will need to work harder than they do, and you may be criticized for being slow. The work dynamics here are important to understand: if your company prioritizes speed over understanding (as many are currently) you need to accept that and not get fired.
However, I think that this is a temporary situation and that soon companies are going to realize that vibe coding at speed suffers from worse complexity explosion issues than well understood, deliberate coding does.
At that point I expect slower, more deliberate coding with AI assistance will be understood as the best way to utilize this new technology.
Where AI can help juniors is in accelerating the road to senior developer by eliminating accidental complexity that often trips juniors up. As I said above, viewing AI as a useful although sometimes overly-eager helper rather than a servant can be very effective in understanding the shape of code bases, what the APIs and techniques available for a particular problem are, how a given build system or programming language works, etc.
But you must write the code.
And companies: you must let juniors write the code.
The questions I get around AI and programming fundamentally revolve around getting a decent job.
It is no secret that the programmer job market is bad right now, and I am seeing good CS students struggle to find positions programming.
While I do not have a crystal ball, I believe this is a temporary rather than permanent situation. The computer programmer job market tends to be cyclical with booms and busts, and I believe we will recover from the current bust at some point.
That’s cold comfort to someone looking for a job now, however, so I want to offer the specific job-seeking advice that I give to my students.
I view the online job sites as mostly pointless, especially for juniors. They are a lottery and the chances of finding a good job through them are low. Since they are free they are probably still worth using, but they are not worth investing a lot of time in.
A better approach is the four F’s: Family, Friends & Family of Friends. Use your personal connections to find positions at companies in which you have a competitive advantage of knowing people in the company. Family is the strongest possibility. Friends are often good too. Family of friends is weaker, but also worth asking about. If you know or are only a few degrees separated from someone at a company you have a much stronger chance of getting a job at that company.
I stress to many students that this doesn’t mean your family has to work for Google or some other big tech company.
All companies of any significant size have problems that need to be solved using computers. Almost every company over 100 people has some sort of development group, even if they don’t call it that.
As an example, I had a student who was struggling to find a job. I asked what their parent did, and they said they worked for Costco corporate.
I told them that they were in fact extremely lucky and that this was their ticket into a great company.
Maybe they don’t start as a “computer programmer” there, maybe they start as an analyst or some other role. But the ability to program on top of that role will be very valuable and likely set up a great career.
So I still think pursuing computer programming as a career is a good idea. The current job market is bad, no doubt, but I think this is temporary.
I do think how computer programming is done is changing, and programmers should look at building up skills beyond “pure” code-writing. This has always been a good idea.
I don’t think programming is changing as dramatically as some people claim and I think the fundamentals of programming, particularly writing good code and controlling complexity, will be perennially important.
I hope this essay is useful in answering that question, especially for junior programmers, and helps people feel more confident entering a career that I have found very rewarding and expect to continue to do for a long time.
And companies: let the juniors write at least some of the code. It is in your interest.
During my 6 years at Cisco, I developed numerous web applications to assist network engineers with highly complex operations, both in terms of the volume of tasks to accomplish and the rigor of procedures to follow. Networking is a specialized field in its own right, where the slightest error can have disastrous consequences: a network failure, even partial, can deprive millions of people of essential services like the ability to make a simple phone call.
This criticality imposes strict requirements on code meant for network operations: it must be reliable, readable, and free of unnecessary frills. If there’s a problem, you need to be able to immediately trace the data flow and fix it on the spot. That’s why, for years, I’ve used very few design patterns and banned function calls that call functions that call functions, and so on. Beyond 2 levels of calls, I abstain.
Following this logic, I favor mature tools over the latest trends. Thus, the Django / Celery / SQLite stack had been in my toolbox for a long time. But like everyone else in the 2010s, I built SPAs and had never heard of intercooler.js or hypermedia, and I understood REST the way it’s commonly described pretty much everywhere.
For the JS framework, I made a conservative choice (and a marginal one, I know): I chose Ember.js. My motivations were its strong backward compatibility during updates and native MVC support. This JS framework is excellent, and that’s still my opinion.
After watching David Guillot’s presentation on HTMX at DjangoCon Europe 2022, I dug into the subject and prototyped a component that addressed one of my recurring needs. It’s a kind of datatable on which you can trigger actions. There’s a demo video on the HTMX Discord here.
I was a beginner with HTMX and built it in 2-3 days (no AI :-) ). But what was interesting wasn’t so much having this component quickly at hand : it was the 100% Django codebase. One codebase instead of two, one app to maintain, and no more API contracts to manage between front and back.
And once again, even though I was comfortable with the Ember.js framework, having a single project to maintain changes everything.
A few weeks later, a concrete use case came up for a major French ISP: configuring L2VPNs on brand-new routers, in bulk, without configuration errors (obviously), based on configurations from old routers that were end-of-life and about to be decommissioned. It was highly critical: a single router can handle thousands of clients, and… there are a lot of routers.
From that point on, I used the Django / Celery + HTMX + SQLite (and Hyperscript) stack and delivered the app in 5 weeks. My goal was to guide the network engineer by the hand and spare them 100% of the repetitive, tedious work: they just had to click and confirm, everything was handled. Their role was now limited to their expertise, and if there was a problem, it was up to them to fix the network.
The project, initially estimated at 18 months, ultimately took 9. And we were lucky: there were no complex corner cases to handle. And even if there had been, we had plenty of time to deal with them.
HTMX in all this? If I had to develop the app as a SPA, it would have taken me at least twice as long. Why? As a solo full-stack developer, simply switching back and forth between codebases is already time-consuming. And that’s just the tip of the iceberg: the front/back approach itself adds a layer of complexity that ends up weighing heavily on productivity.
The Paris 2024 Olympic Games network consisted of thousands of Cisco switches pre-configured to accept Wi-Fi access points, which self-configured through an automation system developed by Orange and Cisco. Wi-Fi was the most common connectivity mode at the Games. But in some cases, a physical connection was necessary, most often to plug in a video camera, but not only. Sometimes there was simply no other choice but to rely on a cable to connect, and therefore to configure the relevant switch port. That’s where an application became necessary.
When Pollux contacted me about his need, he already had a data model for his network services in a Django project. Additionally, he could deploy services via CLI: part of the business logic was already in place. The problem was that service deployment parameters needed to be consolidated in an application. In CLI, you have to manage different data sources, which can quickly become complicated for the user. So it was necessary to centralize these business parameters in a webapp, expose all the data needed to deploy a service, and provide a GUI to configure them.
The Games were approaching and Pollux didn’t have time to build the webapp: as the architect of the Olympic network, he was overwhelmed by a colossal number of tasks. I showed him the L2VPN app mentioned above and specified the 5-week delivery timeline. I told him that if it suited him, I could build him an HTMX webapp based on his existing Django project and a Bootstrap CSS customized internally by Orange.
We agreed on an 8-week timeline to cover the need, which involved 3 connectivity services: Direct Internet Access, Private VLAN, and Shared Internet Access.
HTMX is somewhat a return to the roots of web development, and regardless of the web framework: Django, ROR, Symfony… You rediscover everything that makes a web framework useful rather than turning it into a mere JSON provider. Sending HTML directly to the browser, storing the app state directly in the HTML. That’s what true REST is, and it’s so much simpler to manage.
If you ask me what’s most striking, it’s certainly returning to very simple things like this:
Progress bar from RCP Portal
How does this progress bar work?
Exactly like the example in the docs!
Why this choice? Because it’s coded in 10 seconds, because the app won’t have thousands of users on this internal tool, no scaling concerns: you can do good old data polling without any problem.
And the end user? If I use old-school polling, they don’t care: what they want is the information. No SSE or WebSocket for this use case, I don’t need it. And if the need ever arises, the WebSocket (or SSE) plugin is easy to set up.
One of the big advantages of the philosophy surrounding HTMX is the notion of Locality of Behaviour. Let’s take this progress bar: if you want to know how it works, just look at the page source. No need to go into documentation or the codebase, just a right-click and “View Page Source”:
div
hx-get="/job/progress"
hx-trigger="every 600ms"
hx-target="this"
hx-swap="innerHTML">
div class="progress" role="progressbar" aria-valuemin="0" aria-valuemax="100" aria-valuenow="0"
aria-labelledby="pblabel">
div id="pb" class="progress-bar" style="width:0%">
div>
div>
You immediately know that every 600ms, this part of the page is updated with the content returned by the view handling
the /job/progress endpoint. No mystery for the team taking over development who wants to modify something: everything
you need to know is right in front of you.
And that’s exactly what HTMX is about: every component, every interaction remains visible, understandable, and self-documented directly in the HTML. This is important for what comes next.
In the early stages of app development, I focused on the most complex network service: DIA (Direct Internet Access). DIA for the Olympic Games meant many business parameters with many rules to apply.
The DIA creation form calls an endpoint that triggers a very long function, close to 600 lines of code.
Why such a long function?
Because it’s more readable and efficient to concentrate the data flow in one place, rather than dispersing it across a multitude of layers and patterns.
An application is a wrapper around data: it orchestrates the data flow (data must flow!) and CRUD operations. But what control do you retain when this flow is obfuscated in complex patterns or dispersed across 2 codebases?
The data flow must remain readable for the developer.
HTMX, by allowing you to manage the GUI directly server-side, makes this flow even clearer. The same endpoint can return HTML fragments to signal that certain form data is invalid, or conversely indicate that a service deployment has started. You can thus act on any part of the GUI within the same function, while transforming the data to pass it to the system that configures the network switch.
In a traditional frontend/backend approach, this would be more complex: two applications to manage, and a much less readable data flow.
This drastic code simplification enabled by HTMX, combined with a procedural approach, produces compact and transparent logic, easy to navigate for a developer… or for an LLM, as I discovered.
For the Private VLAN (PVLAN) network service, the “shape” of the main function is roughly the same as for DIA: input parameters, validation, then interactions with the GUI via HTML fragments, and, if everything is correct, switch configuration.
The difference? PVLAN is simpler to handle: fewer form parameters and a bit less business logic.
So I took the long DIA function and gave it to an LLM (Claude 3 had just been released), with a prompt specifying the parameters specific to PVLAN. In seconds, Claude returned a new adapted function, and the same for the HTML templates. Result: about 80% of the code was ready, with only a few points to correct and relatively few errors made by the LLM, which freed up time for me to add specific business logic for a major client.
For the third network service, Shared Internet Access (SIA), even simpler than the previous two, I provided the LLM with both the DIA and PVLAN functions. With the magic word “extrapolation” in the prompt, the generated code was 95% correct.
DIA: 0% AI, 100% handwritten code (business logic + GUI + overhaul of the switch configuration task management system) → 4 weeks
PVLAN: 80% AI, 20% handwritten code (corrections + adding specific business logic) → 1 week
SIA: 95% AI, 5% handwritten code (minor corrections) → 1 day
The time saved was reinvested in testing, bug fixes, project management, and even a few additions outside the initial scope.
Moreover, the same app was used on the “Tour de France 2025” with minor changes that were made easily thanks to the hypermedia approach.
This result is possible because of the combination of HTMX + the procedural approach, which produces naturally readable code, without unnecessary abstraction layers. The data flow is clear, concentrated in a single function, and the GUI/server logic is directly accessible.
For an LLM, this is ideal: it doesn’t need to construct context through a complex architecture. It just needs to follow the flow and extrapolate it to a new use case. In other words, what’s simpler for the developer is also simpler for the AI. This is the sense in which HTMX is truly “AI friendly”™.
Ultimately, HTMX mainly allowed me to save time and keep my code clear. No unnecessary layers, no superfluous complexity: just concrete stuff that works, fast.
And that has made a big difference on these critical projects.
OK, I said there would never be a version three of htmx.
But, technically, I never said anything about a version four…
In The Future of htmx I said the following:
We are going to work to ensure that htmx is extremely stable in both API & implementation. This means accepting and documenting the quirks of the current implementation.
Earlier this year, on a whim, I created fixi.js, a hyperminimalist implementation
of the ideas in htmx. That work gave me a chance to get a lot more familiar with the fetch() and, especially, the
async infrastructure
available in JavaScript.
In doing that work I began to wonder if that, while the htmx API is (at least reasonably) correct, maybe there was room for a more dramatic change of the implementation that took advantage of these features in order to simplify the library.
Further, changing from ye olde XMLHttpRequest
(a legacy of htmx 1.0 IE support) to fetch() would
be a pretty violent change, guaranteed to break at least some stuff.
So I began thinking: if we are going to consider moving to fetch, then maybe we should also use this update as a chance address at least some of the quirks & cruft that htmx has acquired over its lifetime.
So, eventually & reluctantly, I have changed my mind: there will be another major version of htmx.
However, in order to keep my word that there will not be a htmx 3.0, the next release will instead be htmx 4.0.
With htmx 4.0 we are rebuilding the internals of htmx, based on the lessons learned from fixi.js and five+ years of supporting htmx.
There are three major simplifying changes:
The biggest internal change is that fetch() will replace XMLHttpRequest as the core ajax infrastructure. This
won’t actually have a huge effect on most usages of htmx except that the events model will necessarily change due
to the differences between fetch() and XMLHttpRequest.
I feel that the biggest mistake in htmx 1.0 & 2.0 was making attribute inheritance implicit. I was inspired by CSS in doing this, and the results have been roughly the same as CSS: powerful & maddening.
In htmx 4.0, attribute inheritance will be explicit by default rather than implicit. Explicit inheritance will
be done via the :inherited modifier:
div hx-target:inherited="#output">
button hx-post="/up">Likebutton>
button hx-post="/down">Dislikebutton>
div>
output id="output">Pick a button...output>
Here the hx-target attribute is explicitly declared as inherited on the enclosing div and, if it wasn’t, the
button elements would not inherit the target from it.
You will be able to revert to htmx 2.0 implicit inheritance behavior via a configuration variable.
Another source of pain for both us and for htmx users is history support. htmx 2.0 stores history in local cache to make navigation faster. Unfortunately, snapshotting the DOM is often brittle because of third-party modifications, hidden state, etc. There is a terrible simplicity to the web 1.0 model of blowing everything away and starting over. There are also security concerns storing history information in session storage.
In htmx 2.0, we often end up recommending that people facing history-related issues simply disable the cache entirely, and that usually fixes the problems.
In htmx 4.0, history support will no longer snapshot the DOM and keep it locally. It will, rather, issue a network request for the restored content. This is the behavior of 2.0 on a history cache-miss, and it works reliably with little effort on behalf of htmx users.
We will offer an extension that enables history caching like in htmx 2.0, but it will be opt-in, rather than the default.
This tremendously simplifies the htmx codebase and should make the out-of-the-box behavior much more plug-and-play.
Most things.
The core functionality of htmx will remain the same, hx-get, hx-post,
hx-target, hx-boost, hx-swap, hx-trigger, etc.
With a few configuration tweaks, most htmx 2.x based applications should work with htmx 4.x.
These changes will make the long term maintenance & sustainability of the project much stronger. It will also take pressure off the 2.0 releases, which can now focus on stability rather than contemplating new features.
htmx 2.0 users will face an upgrade project when moving to 4.0 in a way that they did not have to in moving from 1.0 to 2.0.
I am sorry about that, and want to offer two things to address it:
htmx 2.0 (like htmx 1.0 & intercooler.js 1.0) will be supported in perpetuity, so there is absolutely no pressure to upgrade your application: if htmx 2.0 is satisfying your hypermedia needs, you can stick with it.
We will roll htmx 4.0 out slowly, over a multi-year period. As with the htmx 1.0 -> 2.0 upgrade, there will be a long
period where htmx 2.x is latest and htmx 4.x is next
Beyond simplifying the implementation of htmx significantly, switching to fetch also gives us the opportunity to add some nice new features to htmx
By switching to fetch(), we can take advantage of its support for
readable streams, which
allow for a stream of content to be swapped into the DOM, rather than a single response.
htmx 1.0 had Server Sent Event support integrated into the library. In htmx 2.0 we pulled this functionality out as an extension. It turns out that SSE is just a specialized version of a streaming response, so in adding streaming support, it’s an almost-free free two-fer to add that back into core as well.
This will make incremental response swapping much cleaner and well-supported in htmx.
Three years ago I had an idea for a DOM morphing algorithm that improved on the initial algorithm pioneered by morphdom.
The idea was to use “id sets” to make smarter decisions regarding which nodes to preserve and which nodes to delete when merging changes into the DOM, and I called this idea “idiomorph”. Idiomorph has gone on to be adopted by many other web project such as Hotwire.
We strongly considered including it in htmx 2.0, but I decided not too because it worked well as an extension and htmx 2.0 had already grown larger than I wanted.
In 4.0, with the complexity savings we achieved by moving to fetch(), we can now comfortably fit a morphInner and
morphOuter swap into core, thanks to the excellent work of Michael West.
htmx has, since very early on, supported a concept of “Out-of-band” swaps: content that is removed from the main HTML response and swapped into the DOM elsewhere. I have always been a bit ambivalent about them, because they move away from Locality of Behavior, but there is no doubt that they are useful and often crucial for achieving certain UI patterns.
Out-of-band swaps started off very simply: if you marked an element as hx-swap-oob='true', htmx would swap the element
as the outer HTML of any existing element already in the DOM with that id. Easy-peasy.
However, over time, people started asking for different functionality around Out-of-band swaps: prepending, appending, etc. and the feature began acquiring some fairly baroque syntax to handle all these needs.
We have come to the conclusion that the problem is that there are really two use cases, both currently trying to be filled by Out-of-band swaps:
A simple, id-based replacement
A more elaborate swap of partial content
Therefore, we are introducing the notion of ``s in htmx 4.0
A partial element is, under the covers, a template element and, thus, can contain any sort of content you like. It
specifies on itself all the standard htmx options regarding swapping, hx-target and hx-swap in particular, allowing
you full access to all the standard swapping behavior of htmx without using a specialized syntax. This tremendously
simplifies the mental model for these sorts of needs, and dovetails well with the streaming support we intend to offer.
Out-of-band swaps will be retained in htmx 4.0, but will go back to their initial, simple focus of simply replacing an existing element by id.
htmx 2.0 has had View Transition support since April of 2023. In the interceding two years, support for the feature has grown across browsers (c’mon, safari, you can do it) and we’ve gained experience with the feature.
One thing that has become apparent to us while using them is that, to use them in a stable manner, it is important to establish a queue of transitions, so each can complete before the other begins. If you don’t do this, you can get visually ugly transition cancellations.
So, in htmx 4.0 we have added this queue which will ensure that all view transitions complete smoothly.
CSS transitions will continue to work as before as well, although the swapping model is again made much simpler by the async runtime.
We may enable View Transitions by default, the jury is still out on that.
A wonderful thing about fetch() and the async support in general is that it is much easier to guarantee a stable
order of events. By linearizing asynchronous code and allowing us to use standard language features like try/catch,
the event model of htmx should be much more predictable and comprehensible.
We are going to adopt a new standard for event naming to make things even clearer:
htmx::[:]
So, for example, htmx:before:request will be triggered before a request is made.
Another opportunity we have is to take advantage of the async behavior of fetch() for much better performance in our
preload extension (where we issue a speculative (GET only!) request in anticipation of an actual trigger). We have
also added an optimistic update extension to the core extensions, again made easy by the new async features.
In general, we have opened up the internals of the htmx request/response/swap cycle much more fully to extension developers,
up to and including allowing them to replace the fetch() implementation used by htmx for a particular request. There
should not be a need for any hacks to get the behavior you want out of htmx now: the events and the open “context” object
should provide the ability to do almost anything.
hx-on SupportIn htmx 2.0, I somewhat reluctantly added the hx-on attributes to support light
scripting inline on elements. I added this because HTML does not allow you to listen for arbitrary events via on
attributes: only standard DOM events like onclick can be responded to.
We hemmed and hawed about the syntax and so, unfortunately, there are a few different ways to do it.
In htmx 4.0 we will adopt a single standard for the hx-on attributes: hx-on:. Additionally, we are
working to improve the htmx JavaScript API (especially around async operation support) and will make those features
available in hx-on:
button hx-post="/like"
hx-on:htmx:after:swap="await timeout('3s'); ctx.newContent[0].remove()">
Get A Response Then Remove It 3 Seconds Later
button>
htmx will never support a fully featured scripting mechanism in core, we recommend something like Alpine.js for that, but our hope is that we can provide a relatively minimalist API that allows for easy, light async scripting of the DOM.
I should note that htmx 4.0 will continue to work with eval() disabled, but you will need to forego a few features like
hx-on if you choose to do so.
All in all, our hope is that htmx 4.0 will feel an awful lot like 2.0, but with better features and, we hope, with fewer bugs.
As always, software takes as long as it takes.
However, our current planned timeline is:
An alpha release is available today: htmx@4.0.0-alpha1
A 4.0.0 release should be available in early-to-mid 2026
4.0 will be marked latest in early-2027ish
You can track our progress (and see quite a bit of dust flying around) in the four branch on
github and at:
Thank you for your patience and pardon our dust!
“Well, when events change, I change my mind. What do you do?” –Paul Samuelson or John Maynard Keynes
Leonard Richardson is a long time programmer and author and was the creator of what came to be termed the Richardson Maturity Model (https://en.wikipedia.org/wiki/Richardson_Maturity_Model), a system for classifying Web APIs in terms of their adherence to REST. Here, Web APIs mean data APIs, that is data intended to be consumed by automated systems or code, rather than directly by a web client.
The RMM consists of four levels:
Level 0 - The Swamp of POX (Plain old XML)
Level 1 - The appropriate use of resource-based URLs
Level 2 - The appropriate use of HTTP Methods
Level 3 - Hypermedia Controls
A Web API became more mature as it adopted these technologies and conventions.
Leonard agreed to talk to me about the RMM and his experiences building Web software.
Question: Can you give us some background about yourself and how you came into web programming? Did/do you consider yourself a hypermedia enthusiast?
When I was in high school in the mid-1990s, I got very basic Internet access through BBSes. There were all these amazing, arcane protocols you had to learn to get around: FTP, Gopher, Archie, NNTP, et cetera. And then just as I went to college, the World Wide Web suddenly wiped out all of those domain-specific protocols. Within a couple of years we were using the Web technologies–URI, HTTP and HTML–for everything.
My formative years as a developer happened against the background of the incredible power of those three core technologies. Everything was being built on top of the Web, and it basically worked and was a lot simpler than the old way.
So yes, I am a hypermedia enthusiast, but it took me a really long time to understand the distinct advantages that come from the “hypermedia” part of the Web. And that came with an understanding of when those advantages are irrelevant or undesirable from a business standpoint.
Question: Can you give us a brief history of early Web APIs? What was the origin story of the RMM?
What we now call “web APIs” started as a reaction against SOAP, a technology from a time when corporate firewalls allowed HTTP connections (necessary to get work done) but blocked most other traffic (games?). SOAP let you serialize a procedure call into XML and invoke it over an HTTP connection, punching through the firewall. It was an extraordinarily heavyweight solution, using the exact same tools–XML and HTTP–you’d need to make a good lightweight solution.
Now that I’m an old fogey, I can look back on SOAP and see the previous generation of old fogeys trying to make the 1990s client-server paradigm work over the Internet. SOAP had a lot of mindshare for a while, but there were very few publicly deployed SOAP services. When you deploy a service on the public Internet, people expect to connect to it from a wide variety of programming languages and programming environments. SOAP wasn’t cut out for that because it was so heavy and demanded so much tooling to compensate for its heaviness.
Instead, developers picked and chose their designs from the core web technologies. Thanks to the dot-com boom, those technologies were understood by practicing developers and well supported by every major programming language.
The RMM, as it’s now called, was originally a heuristic I presented in a talk in 2008. The first part of the talk goes over the early history I mentioned earlier, and the second part talks about my first experience trying to sell hypermedia-based API design to an employer.
I’d analyzed about a hundred web API designs for my book on REST and seen very strong groupings around the core web technologies. You’d see a lot of APIs that “got” URLs but didn’t “get” HTTP. But you’d never see one where it happened the other way, an API that took advantage of the features of HTTP but didn’t know what to do with URLs. If I had one insight here, it’s that the URL is the most fundamental web technology. HTTP is a set of rules for efficiently dealing with URLs, and HTML (a.k.a. hypermedia) is a set of instructions for driving an HTTP client.
Question: In “How Did REST come to mean the opposite of REST?” I assert that the term REST has nearly inverted in its meaning. In particular, I claim that most APIs stopped at “Level 2” of the RMM. Do you agree with these claims?
Everyone understands URIs these days, and understanding HTTP is essential for performance reasons if nothing else. That gets you to level 2, and yes, there we have stayed. That’s what I was getting at in this interview from 2007, a year before I gave my famous talk:
The big question in my mind is whether architectures consciously designed with REST in mind will “win” over architectures that are simple but only intermittently RESTful.
You don’t get a certificate signed by Roy Fielding for achieving level 3. The reward for learning and applying the lesson of hypermedia is interoperability. Your users get the ability to use your system in ways you didn’t anticipate, and they get to combine your system with their other systems.
Interoperability is essential in a situation like the Web, where there are millions of server and client deployments, and a dozen major server implementations. (There are now only two major client implementations, but that’s its own sad story.)
For a long time I thought people just didn’t get this and if I hammered on the technical advantages of hypermedia they’d come around. But we’ve been stuck at level 2 for more than half the lifetime of the Web. It’s become clear to me that most situations aren’t like the Web, and the advantages of hypermedia aren’t relevant to most business models.
Question: Level 3 style hypermedia controls never really took off in Web APIs. Why do you think that is?
I don’t do this for everything, but I am going to blame this one on capitalism.
Almost all actually deployed web APIs are under the complete control of one company. They have one server implementation written by that company and one server deployment managed by that company. If the API has any official client implementations, those are also controlled by the company that owns the API. The fact that we say “the [company] API” is the opposite of interoperability.
Users like interoperability, but vendors prefer lock-in. We see that in their behavior. Netflix was happy to provide a hypermedia API for their program data… until their streaming business became big enough. Once they were the dominant player and could dictate the terms of integration, Netflix shut down their API. Twitter used to cut off your API access if your client got too popular; then they banned third-party clients altogether.
There are lots of APIs that consider interoperability a strong point, and many of them are oriented around hypermedia, but almost all of them live outside the space of commercial competition. In 2008 when I gave the “maturity heuristic” talk I was working on software development tools at Canonical, which is a for-profit company but heavily involved in open source development. We wanted lots of tools and programming environments to be able to talk to our API, but the API was a relatively small part of our process and we didn’t have enough developers to manage a bunch of official clients. A hypermedia-based approach gave us a certain flexibility to change our API without breaking all the clients.
After that I spent eight years working on ebook delivery in the US public library space, which is extremely fragmented in terms of IT management. In a nonprofit environment with lots of independent server deployments, hypermedia (in the form of the OPDS protocol) was a really easy pitch. I gave a talk about that.
To get the benefits of hypermedia you have to collaborate with other people in the same field, consider the entire problem space, and come up with a design that works for everyone. Who’s going to go through all that work when the reward is “no vendor lock-in”? People who are not competing with their peers: scientists, librarians, and open source developers.
It might or might not surprise you to learn that the library world is dominated by an antique protocol called SIP. ( Not the VoIP protocol, a different SIP.) SIP is what the self-checkout machine uses to record the fact that you borrowed the book. SIP first showed up in 1993, its design is distinctively non-RESTful, and in many ways it’s simply bad. Every library vendor has come up with their own level 2 “REST” protocol to do what SIP does. But none have succeeded in displacing SIP, because that awful old 1993 design provides something a vendor can’t offer: interoperability between components from different vendors. I gave a talk about that, too.
Question: Do you think the move from XML to JSON as a format had any influence on how Web APIs evolved?
Absolutely. Moving from XML to JSON replaced a document-centric design (more suitable for communications with a human at one end) with a data-centric design (more suitable for machine-to-machine communication). The cost was forgetting about hypermedia altogether.
One thing about Martin’s diagram that I think obscures more than it reveals is: he calls level 0 the “Swamp of POX”. This makes it seem like the problem is (Plain Old) XML. Martin is actually talking about SOAP there. The big problem with SOAP services isn’t XML (although they do have way too much XML), it’s that they don’t use URLs. A SOAP client puts all of the request information into an XML package and tunnels it through a single service endpoint. This makes SOAP opaque to the tools designed to manage and monitor and inspect HTTP traffic. This is by design, because the point of SOAP is to let you make RPC calls when your IT department has everything but port 80 locked down.
Anyway, XML is great! It’s too verbose to make an efficient data representation format, but XML has namespaces, and through namespaces it has hypermedia controls (via XLink, XForms, XHTML, Atom, etc.). JSON has no hypermedia controls, and because it also has no namespaces, you can’t add them after the fact.
People started adopting JSON because they were tired of XML processing and excited about AJAX (in-browser HTTP clients driven by Javascript, for those who weren’t there). But that initial decision started constraining decisions down the road.
By 2011, all new web APIs were using a representation format with no hypermedia controls. You couldn’t do a hypermedia-based design if you wanted to. Our technical language had lost the words. First you’d have to define a JSON-based media type that had hypermedia (like Siren), or namespaces (like JSON-LD).
Question: What are your thoughts on GraphQL and other non-RESTful API technologies?
With regard to non-RESTful API technologies in general, I would suggest that folks take a break from chapter 5 of Roy Fielding’s dissertation, and look at chapters 2 and 4.
Chapter 5 is where Fielding talks about the design of the Web, but Chapter 2 breaks down all of the possible good things you might want from a networked application architecture, only some of which apply to the Web. Chapter 4 explains the tradeoffs that were made when designing the Web, giving us some good things at the expense of others.
Chapter 4 lists five main advantages of the Web: low entry-barrier, extensibility, distributed hypermedia, anarchic scalability, and independent deployment. REST is a really effective way of getting those advantages, but the advantages themselves are what you really want. If you can get them without the Web technologies, then all you’ve lost is the accumulated expertise that comes with those technologies (although that ain’t nothing at this point). And if you don’t want some of these advantages (probably distributed hypermedia) you can go back to chapter 2, start the process over, and end up with a differently optimized architecture.
I don’t have any direct experience with GraphQL, though I’m about to get some at my current job, so take this with a grain of salt:
On a technical level, GraphQL is solving a problem that’s very common in API design: performing a database query across a network connection without sending a bunch of unneeded data over the wire. Looking at the docs I see it also has “ Mutations“ which seem very SOAP-ish. I guess I’d say GraphQL looks like a modern version of SOAP, optimized for the common case of querying a database.
Since GraphQL is independently deployable, supports multiple server implementations and defines no domain-specific semantics, an interoperable domain-specific API could be built on top of it. Rather than exporting your data model to GraphQL and clashing with a dozen similar data models from the same industry, you could get together with your peers and agree upon a common set of semantics and mutations for your problem space. Then you’d have interoperability. It’s not much different from what we did with OPDS in the library world, defining what concepts like “bookshelf” and “borrow” mean.
Would it be RESTful? Nope! But again I’ll come back to SIP, the integration protocol that public libraries use to keep track of loans. SIP is a level zero protocol! It doesn’t use any of the Web technologies at all! But it provides architectural properties that libraries value and vendor-centric solutions can’t offer–mainly interoperability–so it sticks around despite the presence of “RESTful” solutions.
“Vendoring” software is a technique where you copy the source of another project directly into your own project.
It is an old technique that has been used for time immemorial in software development, but the term “vendoring” to describe it appears to have originated in the ruby community.
Vendoring can be and is still used today. You can vendor htmx, for example, quite easily.
Assuming you have a /js/vendor directory in your project, you can just download the source into your own project like
so:
curl https://raw.githubusercontent.com/bigskysoftware/htmx/refs/tags/v2.0.4/dist/htmx.min.js > /js/vendor/htmx-2.0.4.min.js
You then include the library in your head tag:
script src="/js/vendor/htmx-2.0.4.min.js">script>
And then you check the htmx source into your own source control repository. (I would even recommend considering using the non-minimized version, so you can better understand and debug the code.)
That’s it, that’s vendoring.
OK, great, so what are some strengths of vendoring libraries like this?
It turns out there are quite a few:
Your entire project is checked in to your source repository, so no external systems beyond your source control need to be involved when building it
Vendoring dramatically improves dependency visibility: you can see all the code your project depends on, so you won’t have a situation like we have in htmx, where we feel like we only have a few development dependencies, when in fact we may have a lot
This also means if you have a good debugger, you can step into the library code as easily as any other code. You can also read it, learn from it and even modify it if necessary.
From a security perspective, you aren’t relying on opaque code. Even if your package manager has an integrity hash system, the actual code may be opaque to you. With vendored code it is checked in and can be analysed automatically or by a security team.
Personally, it has always seemed crazy to me that people will often resolve dependencies at deployment time, right when your software is about to go out the door. If that bothers you, like it does me, vendoring puts a stop to it.
On the other hand, vendoring also has one massive drawback: there typically isn’t a good way to deal with what is called the transitive dependency problem.
If htmx had sub-dependencies, that is, other libraries that it depended on, then to vendor it properly you would have to start vendoring all those libraries as well. And if those dependencies had further dependencies, you’d need to install them as well… And on and on.
Worse, two dependencies might depend on the same library, and you’ll need to make sure you get the correct version of that library for everything to work.
This can get pretty difficult to deal with, but I want to make a paradoxical claim that this weakness (and, again, it’s a real one) is actually a strength in some way:
Because dealing with large numbers of dependencies is difficult, vendoring encourages a culture of independence.
You get more of what you make easy, and if you make dependencies easy, you get more of them. Making dependencies, especially transitive dependencies, more difficult would make them less common.
And, as we will see in a bit, maybe fewer dependencies isn’t such a bad thing.
That’s great and all, but there are significant drawbacks to vendoring, particular the transitive dependency problem.
Modern software engineering uses dependency managers to deal with the dependencies of software projects. These tools allow you to specify your projects dependencies, typically via some sort of file. They then they will install those dependencies and resolve and manage all the other dependencies that are necessary for those dependencies to work.
One of the most widely used package managers is NPM: The Node Package Manager. Despite having
no runtime dependencies, htmx uses NPM to specify 16 development dependencies. Development dependencies are dependencies
that are necessary for development of htmx, but not for running it. You can see the dependencies at the bottom of
the NPM package.json file for the project.
Dependency managers are a crucial part of modern software development and many developers today couldn’t imagine writing software without them.
So dependency managers solve the transitive dependency problem that vendoring has. But, as with everything in software
engineering, there are tradeoffs associated with them. To see some of these tradeoffs, let’s take a look at the
package-lock.json file in htmx.
NPM generates a package-lock.json file that contains the resolved transitive closure of dependencies for a project, with
the concrete versions of those dependencies. This helps ensure that the same dependencies are used unless a user
explicitly updates them.
If you take a look at the package-lock.json for htmx, you will find that the original 13 development dependencies have
ballooned into a total of 411 dependencies when all is said and done.
htmx, it turns out, relies on a huge number of packages, despite priding itself on being a relatively lean. In fact,
the node_modules folder in htmx is a whopping 110 megabytes!
But, beyond this bloat there are deeper problems lurking in that mass of dependencies.
While writing this essay I found that htmx apparently depends on the
array.prototype.findlastindex, a
polyfill for a JavaScript feature introduced in
2022.
Now, htmx 1.x is IE compatible, and I don’t want polyfills for anything: I want to write code that will work in IE without any additional library support. And yet a polyfill has snuck in via a chain of dependencies (htmx does not directly rely on it) that introduces a dangerous polyfill that would let me write code that would break in IE, as well as other older browsers.
This polyfill may or may not be available when I run the htmx test suite (it’s hard to tell) but that’s the point: some dangerous code has snuck into my project without me even knowing it, due to the number and complexity of the (development) dependencies it has.
This demonstrates significant cultural problem with dependency managers:
They tend to foster a culture of, well, dependency.
A spectacular example of this was the infamous left-pad incident, in which an engineer took down a widely used package and broke the build at companies like Facebook, PayPal, Netflix, etc.
That was a relatively innocuous, although splashy, issue, but a more serious concern is supply chain attacks, where a hostile entity is able to compromise a company via code injected unwittingly via dependencies.
The larger our dependency graph gets, the worse these problems get.
I’m not the only person thinking about our culture of dependency. Here’s what some other, smarter folks have to say about it:
Armin Ronacher, creator of flask recently said this on the ol’twits:
The more I build software, the more I despise dependencies. I greatly prefer people copy/pasting stuff into their own code bases or re-implement it. Unfortunately the vibe of the time does not embrace that idea much. I need that vibe shift.
He also wrote a great blog post about his experience with package management in the Rust ecosystem:
It’s time to have a new perspective: we should give kudos to engineers who write a small function themselves instead of hooking in a transitive web of crates. We should be suspicious of big crate graphs. Celebrated are the minimal dependencies, the humble function that just quietly does the job, the code that doesn’t need to be touched for years because it was done right once.
Please go read it in full.
Back in 2021, Tom Macwright wrote this in Vendor by default
But one thing that I do think is sort of unusual is: I’m vendoring a lot of stuff.
Vendoring, in the programming sense, means “copying the source code of another project into your project.” It’s in contrast to the practice of using dependencies, which would be adding another project’s name to your package.json file and having npm or yarn download and link it up for you.
I highly recommend reading his take on vendoring as well.
Some good news, if you are an open source developer and like the idea of vendoring, is that there is a simple way to make your software vendor-friendly: remove as many dependencies as you can.
DaisyUI, for example, has been in the process of removing their dependencies, going from 100 dependencies in version 3 to 0 in version 5.
There is also a set htmx-adjacent projects that are taking vendoring seriously:
Surreal - a lightweight jQuery alternative
Facet - an HTML-oriented Web Component library
fixi - a minimal htmx alternative
None of these JavaScript projects are available in NPM, and all of them recommend vendoring the software into your own project as the primary installation mechanism.
The last thing I want to briefly mention is a technology that combines both vendoring and dependency management: vendor-first dependency managers. I have never worked with one before, but I have been pointed to vend, a common lisp vendor oriented package manager (with a great README), as well as go’s vendoring option.
In writing this essay, I also came across vendorpull and git-vendor, both of which are small but interesting projects.
These all look like excellent tools, and it seems to me that there is an opportunity for some of them (and tools like them) to add additional functionality to address the traditional weaknesses of vendoring, for example:
Managing transitive dependencies, if any
Relatively easy updates of those dependencies
Managing local modifications made to dependencies (and maybe help manage contributing them upstream?)
With these additional features I wonder if vendor-first dependency managers could compete with “normal” dependency managers in modern software development, perhaps combining some of the benefits of both approaches.
Regardless, I hope that this essay has helped you think a bit more about dependencies and perhaps planted the idea that maybe your software could be a little less, well, dependent on dependencies.
I’m delighted to be able to interview Makinde Adeagbo, one of the creators of Primer, an hypermedia-oriented javascript library that was being used at Facebook in the 2000s.
Thank you for agreeing to an interview!
Q: To begin with, why don’t you give the readers a bit of your background both professionally & technically?
I’ve always been into tech. In high school, I used to build computers for friends and family. I took the computer science classes my high school offered and went on to study computer science in college. I was always amazed by the fact that I could build cool things—games, tools, etc.—with just a computer and an internet connection.
I was lucky enough to participate in Explore Microsoft, an internship that identifies underrepresented college freshmen and gives them a shot at working at Microsoft. After that experience, I was sold on software as my future. I later interned at Apple and Microsoft again. During college, I also worked at Facebook when the company was about 150 employees. It was an incredible experience where engineers had near-total freedom to build and contribute to the company’s growth. It was exactly what I needed early in my career, and I thrived. From there, I went on to work at Dropbox and Pinterest and also co-founded the nonprofit, /dev/color.
Q: Can you give me the history of how Primer came to be?
In 2010, the Facebook website was sloooow. This wasn’t the fault of any specific person—each engineer was adding features and, along the way, small amounts of JavaScript. However, we didn’t have a coherent system for sharing libraries or tracking how much JavaScript was being shipped with each page. Over time, this led to the 90th-percentile page load time ballooning to about 10 seconds! Midway through the year, reducing that load time by half became one of the company’s three top priorities. I was on a small team of engineers tasked with making it happen.
As we investigated where most of the JavaScript was coming from, we noticed the majority of it was performing simple tasks. These tasks involved either fetching additional data or markup from the server, or submitting a form and then receiving more markup to update the page. With limited time, we decided to build a small solution to abstract those patterns and reduce the amount of code needed on the page.
Tom Occhino and I built the first version of Primer and converted a few use cases ourselves to ensure it worked well. Once we were confident, we brought more engineers into the effort to scale it across the codebase.
Q: Primer & React were both created at Facebook. Was there any internal competition or discussion between the teams? What did that look like?
The two projects came from different eras, needs, and parts of the codebase. As far as I know, there was never any competition between them.
Primer worked well for the type of website we were building in 2010. A key part of its success was understanding that it wasn’t meant to handle every use case. It was an 80/20 solution, and we didn’t use it for particularly complex interactions (like the interface for creating a new post).
React emerged from a completely different challenge: the ads tools. Managing, composing, and tracking hundreds of ads required a highly involved, complex interface. I’m not sure if they ever attempted to use Primer for it, but it would have been a miserable experience. We didn’t have the terminology at the time, but this was a classic example of a single-page application needing purpose-built tools. The users of that site also had a very different profile from someone browsing their home feed or clicking through photos.
Q: Why do you think Primer ultimately failed at Facebook?
I don’t think there’s any single technical solution that has spanned 15 years in Facebook’s platform. The site’s needs evolve, technology changes, and the internet’s landscape shifts over time. Primer served the site well for its time and constraints, but eventually, the product demanded richer interactivity, which wasn’t what Primer was designed for.
Other tradeoffs also come into play: developer ease/speed, security, scalability. These priorities and tradeoffs change over time, especially as a company grows 10x in size.
More broadly, these things tend to work in cycles in the industry. Streamlined, fast solutions give way to richer, heavier tools, which eventually cycle back to streamlined and fast. I wouldn’t be surprised if something like Primer made a comeback at some point.
Q: How much “theory” was there to Primer? Did you think much about hypermedia, REST, etc., when you were building it?
Not much. Honestly, I was young and didn’t know a ton about the internet’s history or past research. I was drawn to the simplicity of the web’s underlying building blocks and thought it was fun to use those tools as they were designed. But, as always, the web is a layer cake of hacks and bandaids, so you have to be flexible.
Q: What were the most important technical lessons you took away from Primer?
Honestly, the biggest lessons were about people. Building a system like Primer is one thing, but for it to succeed, you have to train hundreds of engineers to use it. You have to teach them to think differently about building things, ask questions at the right time, and avoid going too far in the wrong direction. At the end of the day, even if the system is perfect, if engineers hate using it, it won’t succeed.
Mike Amundsen is a computer programmer, author and speaker, and is one of the world leading experts on REST & hypermedia. He has been writing about REST and Hypermedia since 2008 and has published two books on the ideas:
Mike agreed to do an interview with me on his view of the history of hypermedia and where things are today.
Q: The “standard” history of hypermedia is Vannevar Bush’s “As We May Think”, followed by Nelson introducing the term “hypermedia” in 1963, Englebart’s “Mother of all Demos” in 1968 and then Berners-Lee creating The Web in 1990. Are there any other important points you see along the way?
I think starting the history of what I call the “modern web” with Bush makes a lot of sense. Primarily because you can directly link Bush to Engelbart to Nelson to Berners-Lee to Fielding. That’s more than half a century of scholarship, design, and implementation that we can study, learn from, and expand upon.
At the same time, I think there is an unsung hero in the hypermedia story; one stretches back to the early 20th century. I am referring to the Belgian author and entrepreneur Paul Otlet. Otlet had a vision of a multimedia information system he named the “World Wide Network”. He saw how we could combine text, audio, and video into a mix of live and on-demand replay of content from around the world. He even envisioned a kind of multimedia workstation that supported searching, storing, and playing content in what was the earliest instance I can find of an understanding of what we call “streaming services” today.
To back all this up, he created a community of researchers that would read monographs, articles, and books then summarize them to fit on a page or less. He then designed an identification system – much like our URI/URN/URLs today and created a massive card catalog system to enable searching and collating the results into a package that could be shared – even by postal service – with recipients. He created web search by mail in the 1920s!
This was a man well ahead of his time that I’d like to see talked about more in hypermedia and information system circles.
Question: Why do you think that The Web won over other hypermedia systems (such as Xanadu)?
The short reason is, I think, that Xanadu was a much more detailed and specific way of thinking about linking documents, documenting provenance, and compensating authors. That’s a grand vision that was difficult to implement back in the 60s and 70s when Nelson was sharing his ideas.
There are, of course, lots of other factors. Berners-Lee’s vision was much smaller (he was trying to make it easy for CERN staff to share contact information!). Berners-Lee was, I think, much more pragmatic about the implementation details. He himself said he used existing tech (DNS, packet networking, etc.) to implement his ideas. That meant he attracted interest from lots of different communities (telephone, information systems, computing, networking, etc.).
I would also say here that I wish Wendy Hall ’s Microcosm had gotten more traction than it did. Hall and her colleagues built an incredibly rich hypermedia system in the 90s and released it before Berners-Lee’s version of “the Web” was available. And Hall’s Microcosm held more closely to the way Bush, Englebart, and Nelson thought hypermedia systems would be implemented – primarily by storing the hyperlinks in a separate “anchor document” instead of in the source document itself.
Question: What do you think of my essay “How did REST come to mean the opposite of REST”? Are there any points you disagree with in it?
I read that piece back in 2022 when you released it and enjoyed it. While I have nothing to quibble with, really, there are a few observations I can share.
I think I see most hypermedia developers/researchers go through a kind of cycle where you get exposed to “common” REST, then later learn of “Fielding’s REST” and then go back to the “common REST” world with your gained knowledge and try to get others on board; usually with only a small bit of success.
I know you like memes so, I’ll add mine here. This journey away from home, into expanded knowledge and the return to the mundane life you once led is – to me – just another example of Campbell’s Hero’s Journey. I feel this so strongly that I created my own Hero’s Journey presentation to deliver at API conferences over the years.
On a more direct note. I think many readers of Fielding’s Dissertation (for those who actually read it) miss some key points. Fielding’s paper is about designing network architecture, not about REST. REST is offered as a real-world example but it is just that; an example of his approach to information network design. There have been other designs from the same school (UC Irvine) including Justin Erenkrantz’s Computational REST (CREST) and Rohit Kare’s Asynchronous REST (A-REST). These were efforts that got the message of Fielding: “Let’s design networked software systems!”
But that is much more abstract work that most real-world developers need to deal with. They have to get code out the door and up and running quickly and consistently. Fielding’s work, he admitted, was on the “scale of decades” – a scale most developers are not paid to consider.
In the long run, I think it amazing that a PhD dissertation from almost a quarter-century ago has had such a strong influence on day-to-day developers. That’s pretty rare.
Question: Hyperview, the mobile hypermedia that Adam Stepinski created, was very explicitly based on your books. Have you looked at his system?
I have looked over Hyperview and like what I see. I must admit, however, that I don’t write mobile code anymore so I’ve not actually written any hyperview code myself. But I like it.
I talked to Adam in 2022 about Hyperview in general and was impressed with his thoughts. I’d like to see more people talking about and using the Hyperview approach.
Something I am pretty sure I mentioned to Adam at the time is that Hyperview reminds me of Wireless Markup Language (WML). This was another XML-based document model aimed at rendering early web content on feature phones (before smartphone technology). Another XML-based hypermedia domain-specific document format is VoiceXML. I still think there are great applications of hypermedia-based domain-specific markup languages (DSML) and would like to see more of them in use.
Question: It’s perhaps wishful thinking, but I feel there is a resurgence in interest in the ideas of hypermedia and REST (real REST.) Are you seeing this as well? Do you have a sense if businesses are starting to recognize the strengths of this approach?
I, myself, think there is a growth in hypermedia-inspired designs and implementations and I’m glad to see it. I think much of the work of APIs in general has been leading the market to start thinking about how to lower the barrier of entry for using and interoperating with remote, independent services. And the hypermedia control paradigm (the one you and your colleagues talk about in your paper “Hypermedia Controls: Feral to Formal”) offers an excellent way to do that.
I think the biggest hurdle for using more hypermedia in business is was laid out pretty conclusively by Leonard Richardson several years ago. He helped build a powerful hypermedia-based book-sharing server and client system to support public libraries around the world. He noted that, in the library domain, each site is not a competitor but a partner. That means libraries are encouraged to make it easier to loan out books and interoperate with other libraries.
Most businesses operate on the opposite model. They typically succeed by creating barriers of entry and by hoarding assets, not sharing them. Hypermedia makes it easier to share and interact without the need of central control or other types of “gatekeeping.”
Having said that, I think a ripe territory for increased use of hypermedia to lower the bar and increase interaction is at the enterprise level in large organizations. Most big companies spend huge amounts of money building and rebuilding interfaces in order to improve their internal information system. I can’t help but think designing and implementing hypermedia-driven solutions would yield long-term savings, and near-term sustainable interoperability.
Question: Are there any concepts in hypermedia that you think we are sleeping on? Or, maybe said another way, some older ideas that are worth looking at again?
Well, as I just mentioned, I think hypermedia has a big role to play in the field of interoperability. And I think the API-era has, in some ways, distracted us from the power of hypermedia controls as a design element for service-to-service interactions.
While I think Nelson, Berners-Lee and others have done a great job of laying out the possibilities for human-to-machine interaction, I think we’ve lost sight of the possibilities hypermedia gives us for machine-to-machine interactions. I am surprised we don’t have more hypermedia-driven workflow systems available today.
And I think the rise in popularity of LLM-driven automation is another great opportunity to create hypermedia-based, composable services that can be “orchestrated” on the fly. I am worried that we’ll get too tied up in trying to make generative AI systems look and act like human users and miss the chance to design hypermedia workflow designed specifically to take advantage of the strengths of statistical language models.
I’ve seen some interesting things in this area including Zdenek Nemec’s Superface project which has been working on this hypermedia-driven workflow for several years.
I just think there are lots of opportunities to apply what we’ve learned from the last 100 years (when you include Otlet) of hypermedia thinking. And I’m looking forward to seeing what comes next.
I’m very excited to be able to interview @defunkt, the author of pjax, an early hypermedia-oriented javascript library that served as an inspiration for intercooler.js, which later became htmx. He’s done a few other things too, like co-founding GitHub, but in this interview I want to focus on pjax, how it came to be, what influenced it and what it in turn influenced.
Thank you for agreeing to an interview @defunkt!
Q: To begin with, why don’t you give the readers a bit of your background both professionally & technically:
I think I can sum up most of my technical background in two quick anecdotes:
For “show and tell” in 6th grade, I brought in a printout of a web page I had made - including its source code. I like to imagine that everyone was impressed.
Right after 7th grade, a bunch of rowdy high schoolers took me down to the local university during a Linux installfest and put Red Hat on my family’s old PC. That became my main computer for all of high school.
So pretty much from the start I was a web-slinging, UNIX-loving hippie.
In terms of coding, I started on QBasic using the IBM PC running OS/2 in my grandparents’ basement. Then I got deep into MUDs (and MUSHes and MUXes and MOOs…) which were written in C and usually included their own custom scripting language. Writing C was “hardcoding”, writing scripts was “softcoding”. I had no idea what I was doing in C, but I really liked the softcoding aspect.
The same rowdy high schoolers who introduced me to Linux gave me the O’Reilly camel book and told me to learn Perl. I did not enjoy it. But they also showed me php3, and suddenly it all came together: HTML combined with MUD-like softcoding. I was hooked.
I tried other things like ASP 3.0 and Visual Basic, but ultimately PHP was my jam for all of high school. I loved making dynamic webpages, and I loved Linux servers. My friends and I had a comedy website in high school that shall remain nameless, and I wrote the whole mysql/php backend myself before blogging software was popular. It was so much fun.
My first year of college I switched to Gentoo and became fascinated with their package manager, which was written in Python. You could write real Linux tools with it, which was amazing, but at the time the web story felt weak.
I bought the huge Python O’Reilly book and was making my way through it when, randomly, I discovered Ruby on Rails. It hit me like a bolt of lightning and suddenly my PHP and Python days were over.
At the same time, Web 2.0 had just been coined and JavaScript was, like, “Hey, everyone. I’ve been here all along.” So as I was learning Rails, I was also learning JavaScript. Rails had helpers to abstract the JS away, but I actually really liked the language (mostly) and wanted to learn it without relying on a framework or library.
The combination of administering my own Linux servers, writing backend code in Rails, and writing frontend code in JavaScript made me fall deeper in love with the web as a platform and exposed me to concepts like REST and HATEOAS. Which, as someone who had been writing HTML for over a decade, felt natural and made sense.
GitHub launched in 2008 powered by, surprise, Gentoo, Rails, and JavaScript. But due to GitHub’s position as not just a Rails community, but a collection of programming communities, I quickly evolved into a massive polyglot.
I went back and learned Python, competing in a few programming competitions like Django Dash and attending (and speaking) at different PyCons. I learned Objective-C and made Mac (and later iPhone) apps. I learned Scheme and Lisp, eventually switching to Emacs from Vim and writing tons of Emacs Lisp. I went back and learned what all the sigils mean in Perl. Then Lua, Java, C++, C, even C# - I wanted to try everything.
And I’m still that way today. I’ve written projects in Go, Rust, Haskell, OCaml, F#, all sorts of Lisps (Chicken Scheme, Clojure, Racket, Gambit), and more. I’ve written a dozen programming languages, including a few that can actually do something. Right now I’m learning Zig.
But I always go back to the web. It’s why I created the Atom text editor using web technologies, it’s why Electron exists, and it’s why I just cofounded the Ladybird Browser Initiative with Andreas Kling to develop the independent, open source Ladybird web browser.
Q: Can you give me the history of how pjax came to be?
It all starts with XMLHttpRequest, of course. Ajax. When I was growing up, walking to school both ways uphill in the snow, the web was simple: you clicked on a link and a new web page loaded. Nothing fancy. It was a thing of beauty, and it was good.
Then folks started building email clients and all sorts of application-like programs in HTML using `` and friends. It was not very beautiful, and not very good, but there was something there.
Luckily, in the mid-2000s, Gmail and Ajax changed things. Hotmail had been around for a while, but Gmail was fast. By updating content without a full page load using XMLHttpRequest, you could make a webpage that felt like a desktop application without resorting to frames or other chicanery. And while other sites had used Ajax before Gmail, Gmail became so popular that it really put this technique on the map.
Soon Ajax, along with the ability to add rounded corners to web pages, ushered in the era known as Web 2.0. By 2010, more and more web developers were pushing more and more of their code into JavaScript and loading dynamic content with Ajax. There was just one problem: in the original, good model of the web, each page had a unique URL that you could use to load its content in any context. This is one of the innovations of the web. When using Ajax, however, the URL doesn’t change. And even worse, it can’t be changed - not the part that gets read by the server, anyway. The web was broken.
As is tradition, developers created hacks to work around this limitation. The era of the #! began, pioneered by Ajax-heavy sites like Facebook and Twitter. Instead of http://twitter.com/htmx_org, you’d see http://twitter.com/#!/htmx_org in your browser’s URL bar when visiting someone’s profile. The # was traditionally used for anchor tags, to link to a sub-section within a full web page, and could be modified by JavaScript. These ancient web 2.0 developers took advantage of #’s malleability and started using it to represent permanent content that could be updated inline, much like a real URL. The only problem was that your server code never saw the # part of a URL when serving a request, so now you needed to start changing your backend architecture to make everything work.
Oh, and it was all very buggy. That was a problem too.
As an HTTP purist, I detested the #!. But I didn’t have a better way.
Time passed and lo, a solution appeared. One magical day, the #!s quietly disappeared from Facebook, replaced by good old fashioned URLs. Had they abandoned Web 2.0? No… they had found a better way.
The history.pushState() function, along with its sibling history.replaceState(), had been recently added to all
major web browsers. Facebook quickly took advantage of this new API to update the full URL in your browser whenever
changing content via Ajax, returning the web to its previous glory.
And so there it was: the Missing Link.
We had our solution, but now a new problem: GitHub was not an SPA, and I didn’t want it to be one. By 2011 I had been writing JavaScript for six years - more than enough time to know that too much JS is a terrible thing. The original GitHub Issue Tracker was a Gmail-style web application built entirely in JS, circa 2009. It was an awful experience for me, GitHub developers, and, ultimately, our users.
That said, I still believed Ajax could dramatically speed up a web page’s user interface and improve the overall experience. I just didn’t want to do it by writing lots of, or any, JavaScript. I liked the simple request/response paradigm that the web was built on.
Thus, Pjax was born. It sped up GitHub’s UI by loading new pages via Ajax instead of full page loads, correctly updating
URLs while not requiring any JS beyond the Pjax library itself. Our developers could just tag a link with [data-pjax]
and our backend application would then automatically render a page’s content without any layout, quickly getting you
just the data you need without asking the browser to reload any JS or CSS or HTML that didn’t need to change. It also (
mostly) worked with the back button, just like regular web pages, and it had a JS API if you did need to dip into the
dark side and write something custom.
The first commit to Pjax was Feb 26, 2011 and it was released publicly in late March 2011, after we had been using it to power GitHub.com for some time.
Q: I recall it being a big deal in the rails community. Did the advent of turbolinks hurt adoption there?
My goal wasn’t really adoption of the library. If it was, I probably would have put in the work to decouple it from jQuery. At the time, I was deep in building GitHub and wasn’t the best steward of my many existing open source projects.
What I wanted instead was adoption of the idea - I wanted people to know about pushState(), and I wanted people to
know there were ways to build websites other than just doing everything by hand in JavaScript. Rendering pages in whole
or in part on the server was still viable, and could be sped up using modern techniques.
Turbolinks being created and integrated into Rails was amazing to see, and not entirely unsurprising. I was a huge fan of Sam Stephenson’s work even pre-GitHub, and we had very similiar ideas about HTTP and the web. Part of my thinking was influenced by him and the Rails community, and part of what drew me to the Rails community was the shared ideas around what’s great about the web.
Besides being coupled to jQuery, pjax’s approach was quite limited. It was a simple library. I knew that other people could take it further, and I’m glad they did.
Q: How much “theory” was there to pjax? Did you think much about hypermedia, REST, etc. when you were building it? ( I backed into the theory after I had built intercooler, curious how it went for you!)
Not much. It started by appending ?pjax=1 to every request, but before release we switched it to send an X-PJAX
header instead. Very fancy.
Early GitHub developer Rick Olson (@technoweenie), also from the Rails community, was the person who introduced me to HATEOAS and drove that philosophy in GitHub’s API. So anything good about Pjax came from him and Josh Peek, another early Rails-er.
My focus was mostly on the user experience, the developer experience, and trying to stick to what made the web great.
img, video { max-width: 100%; margin: 10px; }
When I was in college, I wrote some customer service software that tied together some custom AI models I trained, the OpenAI API, a database, and some social media APIs to make the first version of Sidekick.
Over the next couple years I worked on adding more features and growing the user base. As a solo founder, I should have been focused on sales, marketing, and market discovery. Instead, as an engineer, I wanted to hand-craft the perfect web stack. I was firmly of the belief that the network gap between the frontend and the backend could be abstracted away, and I could make writing web apps as simple as writing native apps. Did this have anything to do with my business, product, or customers? Absolutely not, but as many technical founders do, I believed if I perfected the tech, the customers would materialize.
My design decisions were naive, but also reminiscent to what’s seen in industry today: I wanted the backend and frontend to share a language (Rust), I wanted compile-time checks across the network boundary, I wanted to write frontend code like it was an app (reactive), and I wanted nearly instant reload times. What I got out of it was a buggy mess.
I had invented a system where simple rust functions can be tagged with a macro to generate a backend route and a frontend request function, so you can call the function like it was a standard function, and it would run on the backend. A true poor-mans GraphQL. My desire to write Rust on the frontend required I compile a WASM bundle. My desire for instant load times required isomorphic SSR. All of this complexity, for what was essentially a simple CRUD site.
At this point Sidekick has grown and it now has a codebase which is responsible for not-insignificant volumes of traffic each day. There was this point where I looked into HTMX, multi-page websites, and HATEOAS, and realized the Sidekick codebase, which had grown into ~36k lines spread over 8 different crates, could be folded into a single crate, a single binary that ran the backend, which generated the frontend on demand through templating, and that HTMX could suffice for all the interactivity we required.
Large refactors typically have a bad track record so we wrote a quick and dirty simplified version of part of the site to convince ourselves it could work. After sufficient convincing, we undertook a full rewrite. All said and done, the rewrite took approximately 3 weeks of intense work. The results were dramatic:
36k LOC -> 8k LOC
8 crates -> 1 crate
~5 bug reports / week -> ~1 bug report / week
More full nights of sleep
The rewrite went far better than I could have imagined. It definitely won’t be representative of every experience, our app was definitely uniquely suited to HTMX. Axum and some custom middleware also went a long way for sharing common infrastructure across the site. Though we don’t have proper metrics, we’ve anecdotally noticed significantly improved load times.
I’ll finish by touching on the biggest benefit in my eyes: it’s tremendously easier to add new features as our customers request them. A feature that would have taken 2 weeks to fully implement, test and ship, now takes a day or two. As a small startup with a large number of customer demands, this is table stakes.
Sidekick hasn’t raised VC funding so I can’t afford to hire lots of devs. With HTMX we don’t need to.
htmx began life as intercooler.js, a library built around jQuery that added behavior based on HTML attributes.
For developers who are not familiar with it, jQuery is a venerable JavaScript library that made writing cross-platform JavaScript a lot easier during a time when browser implementations were very inconsistent, and JavaScript didn’t have many of the convenient APIs and features that it does now.
Today many web developers consider jQuery to be “legacy software.” With all due respect to this perspective, jQuery is currently used on 75% of all public websites, a number that dwarfs all other JavaScript tools.
Why has jQuery remained so ubiquitous?
Here are three technical reasons we believe contribute to its ongoing success:
It is very easy to add to a project (just a single, dependency-free link)
It has maintained a very consistent API, remaining largely backwards compatible over its life (intercooler.js works with jQuery v1, v2 and v3)
As a library, you can use as much or as little of it as you like: it stays out of the way otherwise and doesn’t dictate the structure of your application
Now, that’s a ridiculous (and arrogant) statement to make, of course, but it is an ideal that we on the htmx team are striving for.
In particular, we want to emulate these technical characteristics of jQuery that make it such a low-cost, high-value addition to the toolkits of web developers. Alex has discussed “Building The 100 Year Web Service” and we want htmx to be a useful tool for exactly that use case.
Websites that are built with jQuery stay online for a very long time, and websites built with htmx should be capable of the same (or better).
Going forward, htmx will be developed with its existing users in mind.
If you are an existing user of htmx—or are thinking about becoming one—here’s what that means.
We are going to work to ensure that htmx is extremely stable in both API & implementation. This means accepting and documenting the quirks of the current implementation.
Someone upgrading htmx (even from 1.x to 2.x) should expect things to continue working as before.
Where appropriate, we may add better configuration options, but we won’t change defaults.
We are going to be increasingly inclined to not accept new proposed features in the library core.
People shouldn’t feel pressure to upgrade htmx over time unless there are specific bugs that they want fixed, and they should feel comfortable that the htmx that they write in 2025 will look very similar to htmx they write in 2035 and beyond.
We will consider new core features when new browser features become available, for example we
are already using the experimental moveBefore() API on supported browsers.
However, we expect most new functionality to be explored and delivered via the htmx extensions API, and will work to make the extensions API more capable where appropriate.
Our release schedule is going to be roughly quarterly going forward.
There will be no death march upgrades associated with htmx, and there is no reason to monitor htmx releases for major functionality changes, just like with jQuery. If htmx 1.x is working fine for you, there is no reason to feel like you need to move to 2.x.
htmx does not aim to be a total solution for building web applications and services: it generalizes hypermedia controls, and that’s roughly about it.
This means that a very important way to improve htmx — and one with lots of work remaining — is by helping improve the tools and techniques that people use in conjunction with htmx.
Doing so makes htmx dramatically more useful without any changes to htmx itself.
While htmx gives you a few new tools in your HTML, it has no opinions about other important aspects of building your websites. A flagship feature of htmx is that it does not dictate what backend or database you use.
htmx is compatible with lots of backends, and we want to help make hypermedia-driven development work better for all of them.
One part of the hypermedia ecosystem that htmx has already helped improve is template engines. When we first wrote about how “template fragments” make defining partial page replacements much simpler, they were a relatively rare feature in template engines.
Not only are fragments much more common now, that essay is frequently cited as an inspiration for building the feature.
There are many other ways that the experience of writing hypermedia-based applications can be improved, and we will remain dedicated to identifying and promoting those efforts.
Although htmx will not be changing dramatically going forward, we will continue energetically evangelizing the ideas of hypermedia.
In particular, we are trying to push the ideas of htmx into the HTML standard itself, via the Triptych project. In an ideal world, htmx functionality disappears into the web platform itself.
htmx code written today will continue working forever, of course, but in the very long run perhaps there will be no need to include the library to achieve similar UI patterns via hypermedia.
At the end of the intercooler docs, we said this:
Many javascript projects are updated at a dizzying pace. Intercooler is not.
This is not because it is dead, but rather because it is (mostly) right: the basic idea is right, and the implementation at least right enough.
This means there will not be constant activity and churn on the project, but rather a stewardship relationship: the main goal now is to not screw it up. The documentation will be improved, tests will be added, small new declarative features will be added around the edges, but there will be no massive rewrite or constant updating. This is in contrast with the software industry in general and the front end world in particular, which has comical levels of churn.
Intercooler is a sturdy, reliable tool for web development.
Leaving aside the snark at the end of the third paragraph, this thinking is very much applicable to htmx. In fact, perhaps even more so since htmx is a standalone piece of software, benefiting from the experiences (and mistakes) of intercooler.js.
We hope to see htmx, in its own small way, join the likes of giants like jQuery as a sturdy and reliable tool for building your 100 year web services.
This is a “quirks” page, based on SQLite’s “Quirks, Caveats, and Gotchas In SQLite” page.
Many attributes in htmx are inherited: child elements can receive behavior from attributes located on parent elements.
As an example, here are two htmx-powered buttons that inherit their target from a parent div:
div hx-target="#output">
button hx-post="/items/100/like">Likebutton>
button hx-delete="/items/100">Deletebutton>
div>
output id="output">output>
This helps avoid repeating attributes, thus keeping code DRY.
On the other hand, as the attributes get further away elements, you lose Locality of Behavior and it becomes more difficult to understand what an element is doing.
It is also possible to inadvertently change the behavior of elements by adding attributes to parents.
Some people prefer to disable inheritance in htmx entirely, using the htmx.config.disableInheritance
configuration variable.
Here is a meta tag configuration that does so:
meta name="htmx-config" content='{"disableInheritance":true}'>
innerHTMLThe hx-swap attribute allows you to control how a swap is performed. The default strategy is
innerHTML, that is, to place the response HTML content within the target element.
Many people prefer to use the outerHTML strategy as the default instead.
You can change this behavior using the htmx.config.defaultSwapStyle
configuration variable.
Here is a meta tag configuration that does so:
meta name="htmx-config" content='{"defaultSwapStyle":"outerHTML"}'>
body Always Performs an innerHTML SwapFor historical reasons, if you target the body element, htmx will
always perform an innerHTML swap.
This means you cannot change attributes on the body tag via an htmx request.
4xx & 5xx Responses Do Not Swaphtmx has never swapped “error” status response codes (400s & 500s) by default.
This behavior annoys some people, and some server frameworks, in particular, will return a 422 - Unprocessable Entity
response code to indicate that a form was not filled out properly.
This can be very confusing when it is first encountered.
You can configure the response behavior of htmx via the htmx:beforeSwap
event or via the htmx.config.responseHandling config array.
Here is the default configuration:
{
"responseHandling": [
{"code":"204", "swap": false},
{"code":"[23]..", "swap": true},
{"code":"[45]..", "swap": false, "error":true},
{"code":"...", "swap": false}]
}
Note that 204 No Content also is not swapped.
If you want to swap everything regardless of response code, you can use this configuration:
{
"responseHandling": [
{"code":"...", "swap": true}]
}
If you want to specifically allow 422 responses to swap, you can use this configuration:
{
"responseHandling": [
{"code":"422", "swap": true},
{"code":"204", "swap": false},
{"code":"[23]..", "swap": true},
{"code":"[45]..", "swap": false, "error":true},
{"code":"...", "swap": false}]
}
Here is a meta tag allowing all responses to swap:
meta name="htmx-config" content='{"responseHandling": [{"code":"...", "swap": true}]}'>
GET Requests on Non-Form Elements Do Not Include Form Values by DefaultIf a non-form element makes a non-GET request (e.g. a PUT request) via htmx, the values of the enclosing form
of that element (if any) will be included in the request.
However, if the element issues a GET, the values of an enclosing form will
not be included.
If you wish to include the values of the enclosing form when issuing an GET you can use the
hx-include attribute like so:
button hx-get="/search"
hx-include="closest form">
Search
button>
htmx provides support for interacting with the browser’s history. This can be very powerful, but it can also be tricky, particularly if you are using 3rd party JavaScript libraries that modify the DOM.
There can also be security concerns when using htmx’s history support.
Most of these issues can be solved by disabling any local history cache and simply issuing a server request when a user navigates backwards in history, with the tradeoff that history navigation will be slower.
Here is a meta tag that disables history caching:
meta name="htmx-config" content='{"historyCacheSize": 0}'>
hx-boosthx-boost is an odd feature compared with most other aspects of htmx: it “magically” turns
all anchor tags and forms into AJAX requests.
This can speed the feel of these interactions up, and also allows the forms and anchors to continue working when JavaScript is disabled, however it comes with some tradeoffs:
The history issues mentioned above can show up
Only the body of the web page will be updated, so any styles and scripts in the new page head tag will be discarded
The global javascript scope is not refreshed, so it is possible to have strange interactions between pages. For example
a global let may start failing because a symbol is already defined.
Some members on the core htmx team feel that, due to these issues, as well as the fact that browsers have improved
quite a bit in page navigation, it is best to avoid hx-boost and
just use unboosted links and forms.
There is no doubt that hx-boost is an odd-man out when compared to other htmx attributes and suffers from the dictum
that “If something magically works, then it can also magically break.”
Despite this fact, I (Carson) still feel it is useful in many situations, and it is used on the https://htmx.org website.
htmx is designed to be loaded with a standard, blocking `` tag, not one that is a module or deferred. Although we make a best-effort attempt to initialize htmx regardless of when in the document lifecycle the script is loaded, there are some use-cases that slip through the cracks, typically ones that involve bundling or AJAX insertion of htmx itself.
Our past attempts to close this gap have all lead to unacceptable regressions. Therefore, although htmx can be loaded asynchronously, do so at your own risk.
Keep in mind, also, that if your DOM content loads before htmx does, all the htmx-provided functionality will be nonfunctional until htmx loads. Prefetching (or even “regular” fetching) htmx before you need it is one possible way to resolve this problem.
htmx is a hypermedia-oriented front end library. This means that htmx enhances HTML via attributes in the HTML , rather than providing an elaborate JavaScript API.
There is a JavaScript API, but it is not a focus of the library and, in most cases, should not be used heavily by htmx end users.
If you find yourself using it heavily, especially the htmx.ajax() method, it may be
worth asking yourself if there is a more htmx-ish approach to achieve what you are doing.
For better or for worse, htmx has collected a lot of lore, mainly around the twitter account.
Here are some explanations.
A common set of phrases used by htmx enthusiasts when, for example, @bunjavascript told me to delete my account
At one point there was a hostile takeover attempt of the htmx CEO position and, in a desperate poison pill, I declared everyone CEO of htmx.
Turk created https://htmx.ceo if you want to register as a CEO.
If someone emails hr@bigsky.software asking if you are CEO of htmx, I will tell them yes.
You can put it on your LinkedIn, because it’s true.
At some point I photoshopped lasers onto a horse mask, as kind of an homage to @horse_js.
For some reason it stuck and now it’s the official unofficial mascot of htmx.
Spieltrieb means “play instinct”, and is a big part of the htmx vibe.
At some point someone (I think @techsavvytravvy), generated a grug AI image, and there was a pickle smiling in a really bizarre way in it.
So we started riffing on pickles and now there’s a shirt.
Cry more, drizzle.
In July 2023, when htmx first got popular, there was a moral panic around cross site scripting. I may have overcooked my response to it.
@WarrenInTheBuff is the king of twitter and we regularly fight with him. This often ends in someone saying “shut up warren”.
You can see the htmx website do this by going to https://htmx.org?suw=true
In mid-January of 2024 I got really serious with the htmx twitter account and started quote tweeting things about microsoft. People started worrying. I announced a license change to get people freaked out about a rug pull.
This is the offer I got from microsoft (real).
I believe that this tweet is the origin of the (same thing) meme
In December 2023, I was trying to get some indonesian twitter users to take a look at htmx, so I created a “Montana & Indonesia, Stronger Together!” tweet w/an AI image.
This turned into a whole series of tweets.
Sometimes I am accused of being “unhinged” but, in fact, I own many hinges.
People often call htmx a framework, but it’s a library
A common one word response when I don’t feel like arguing with someone.
In December 2024, I added a marquee tag to the htmx website and started using the honorific (sic) in my twitter title.
I wrote an essay called htmx sucks in which I criticize htmx (some valid, some tongue in cheek, most both.) I also released a mug that I will often link to when people are criticizing htmx.
Jason Knight hates htmx and wrote a great post about it.
Please don’t harass him, I draw energy from his posts.
In July 2023, sparked by the accusation that htmx users could not create dropdowns, I did a deep-dive into web drop down technology and uncovered a bombshell
A phrase I will often quote tweet violent htmx-related imagery with.
The Process™ is the mechanism by which people initially hostile to htmx come to be enlightened.
In June 2023, @srasash accused htmx of being a government op, the first in many such increasingly ridiculous claims. I typically quote-tweet these claims and point out that “that’s ridiculous”
I created http://grugbrain.dev.
The htmx & intercooler.js twitter accounts often fight with one another. Sometimes its just me switching back and forth, but two other people have access to the intercooler account, so sometimes I have no idea who I am fighting with.
Nothing magically breaks.
I was very unfairly given a lifetime ban from /r/webdev/ for an obviously satirical post. Even the term “htmx” is banned (or semi-banned) on that sub, so people now use the htmeggs instead.
idk, I just think it’s funny
A common phrase used to mock people (including ourselves) with.
htmx is a villain in the front-end world, I’m good w/that
“Invert, always invert.” –Carl Jacobi, by way of Charlie Munger
prefer if statements to polymorphism
whenever you are tempted to create a class, ask yourself: “could this be an if statement instead?”
In grug-oriented programming, the closed–closed principle (CCP) states “software entities (classes, modules, functions, etc.) should be closed for extension, but also closed for modification”
they should just do something useful man
The Minimize Abstractions Principle (MAP) is a computer programming principle that states that “a module should minimize the number of abstractions it contains, both in API and in implementation. Stop navel gazing nerd.”
The “Try It Out” Substitution Principle states that you should try something out and, if that doesn’t work, think about why, and substitute something else for it instead.
It is common to need to substitute multiple things to hit on the right thing eventually.
The Useful Stuff Principle states that entities must depend on useful stuff, not overly abstract nonsense. It states that a high-level module can depend on a low-level module, because that’s how software works.
The The Existence of Dependencies Is Not An Excuse For Destroying The Codebase Principle states that the existence of dependencies is not an excuse for destroying the codebase
consider giving your developers an abstraction budget
when they exhaust that budget & ask for more, tell them they can have another abstraction when they remove an existing one
when they complain, look for classes with the term “Factory”, “Lookup” or “Visitor” in their names
if your function is only called in one place, consider inlining it & reducing the total number of method signatures in your module, to help people better understand it
studies show longer methods have fewer bugs per line of code, so favor longer methods over many smaller methods
consider creating “God” objects that wrap a lot of functionality up in a single package
consumers of your API don’t want to learn 50 different classes to get something done, so give them a few that provide the core functionality of your module with minimal fuss
copy&paste driven development is a development methodology where, when you need to reuse some code but in a slightly different manner, you copy & paste the code & then modify it to satisfy the new requirements
this contrasts with designing an elaborate object model, for example
implementation driven development is a development methodology where you first explore various implementations of your idea to determine the best one, then add tests for it
no test code may be written without first having some implementation code to drive that test
Mixing of Concerns is a design methodology whereby “concerns” of various kinds are mixed into a single code unit. This improves locality in that code unit by placing all relevant logic within it. An example of this is hypermedia, which mixes control & presentation information.
a macroservice architecture revolves around “macroservices”: network-deployed modules of code that provide a significant amount of functionality to the overall system
by adopting a macroservice-based architecture you minimize deployment complexity & maximize resource utilization
kinda had a manic break last night my bad
“Writing clean code is what you must do in order to call yourself a professional. There is no reasonable excuse for doing anything less than your best.” Clean Code
In this essay I want to talk about how I write code. I am going to call my approach “codin’ dirty” because I often go against the recommendations of Clean Code, a popular approach to writing code.
Now, I don’t really consider my code all that dirty: it’s a little gronky in places but for the most part I’m happy with it and find it easy enough to maintain with reasonable levels of quality.
I’m also not trying to convince you to code dirty with this essay. Rather, I want to show that it is possible to write reasonably successful software this way and, I hope, offer some balance around software methodology discussions.
I’ve been programming for a while now and I have seen a bunch of different approaches to building software work. Some people love Object-Oriented Programming (I like it), other very smart people hate it. Some folks love the expressiveness of dynamic languages, other people hate it. Some people ship successfully while strictly following Test Driven Development, others slap a few end-to-end tests on at the end of the project, and many people end up somewhere between these extremes.
I’ve seen projects using all of these different approaches ship and maintain successful software.
So, again, my goal here is not to convince you that my way of coding is the only way, but rather to show you (particularly younger developers, who are prone to being intimidated by terms like “Clean Code”) that you can have a successful programming career using a lot of different approaches, and that mine is one of them.
Three “dirty” coding practices I’m going to discuss in this essay are:
(Some) big functions are good, actually
Prefer integration tests to unit tests
Keep your class/interface/concept count down
If you want to skip the rest of the essay, that’s the takeaway.
I think that large functions are fine. In fact, I think that some big functions are usually a good thing in a codebase.
This is in contrast with Clean Code, which says:
“The first rule of functions is that they should be small. The second rule of functions is that they should be smaller than that.” Clean Code
Now, it always depends on the type of work that I’m doing, of course, but I usually tend to organize my functions into the following:
A few large “crux” functions, the real meat of the module. I set no bound on the Lines of Code (LOC) of these functions, although I start to feel a little bad when they get larger than maybe 200-300 LOC.
A fair number of “support” functions, which tend to be in the 10-20 LOC range
A fair number of “utility” functions, which tend to be in the 5-10 LOC range
As an example of a “crux” function, consider the issueAjaxRequest()
in htmx. This function is nearly 400 lines long!
Definitely not clean!
However, in this function there is a lot of context to keep around, and it lays out a series of specific steps that must proceed in a fairly linear manner. There isn’t any reuse to be found by splitting it up into other functions and I think it would hurt the clarity (and also importantly for me, the debuggability) of the function if I did so.
A big reason I like big functions is that I think that in software, all other things being equal, important things should be big, whereas unimportant things should be little.
Consider a visual representation of “Clean” code versus “Dirty” code:
When you split your functions into many equally sized, small implementations you end up smearing the important parts of your implementation around your module, even if they are expressed perfectly well in a larger function.
Everything ends up looking the same: a function signature definition, followed by an if statement or a for loop, maybe a function call or two, and a return.
If you allow your important “crux” functions to be larger it is easier to pick them out from the sea of functions, they are obviously important: just look at them, they are big!
There are also fewer functions in general in all categories, since much of the code has been merged into larger functions. Fewer lines of code are dedicated to particular type signatures (which can change over time) and it easier to keep the important and maybe even the medium-important function names and signatures in your head. You also tend to have fewer LOC overall when you do this.
I prefer coming into a new “dirty” code module: I will be able to understand it more quickly and will remember the important parts more easily.
What about the empirical (dread word in software!) evidence for the ideal function size?
In Chapter 7, Section 4 of Code Complete, Steve McConnell lays out some evidence for and against longer functions. The results are mixed, but many of the studies he cites show better errors-per-line metrics for larger, rather than smaller, functions.
There are newer studies as well that argue for smaller functions ( 200LOC, and a walk around the SQLite codebase will furnish many other examples of large functions. SQLite is noted for being extremely high quality and very well maintained.
Or consider the ChromeContentRendererClient::RenderFrameCreated()
function in the Google Chrome Web Browser. Also looks to be over 200 LOC. Again, poking around the codebase
will give you plenty of other long functions to look at. Chrome is solving one of the hardest problems in software:
being a good general purpose hypermedia client. And yet their code doesn’t look very “clean” to me.
Next, consider the kvstoreScan()
function in Redis. Smaller, on the order of 40LOC, but still far larger than Clean Code would
suggest. A quick scan through the Redis codebase will furnish many other “dirty” examples.
These are all C-based projects, so maybe the rule of small functions only applies to object-oriented languages, like Java?
OK, take a look at the update()
function in the CompilerAction class of IntelliJ, which is roughly 90LOC. Again,
poking around their codebase will reveal many other large functions well over 50LOC.
SQLite, Chrome, Redis & IntelliJ…
These are important, complicated, successful & well maintained pieces of software, and yet we can find large functions in all of them.
Now, I don’t want to imply that any of the engineers on these projects agree with this essay in any way, but I think that we have some fairly good evidence that longer functions are OK in software projects. It seems safe to say that breaking up functions just to keep them small is not necessary. Of course you can consider doing so for other reasons such as code reuse, but being small just for small’s sake seems unnecessary.
I am a huge fan of testing and highly recommend testing software as a key component of building maintainable systems.
htmx itself is only possible because we have a good test suite that helps us ensure that the library stays stable as we work on it.
If you take a look at the test suite one thing you might notice is the relative lack of Unit Tests. We have very few test that directly call functions on the htmx object. Instead, the tests are mostly Integration Tests: they set up a particular DOM configuration with some htmx attributes and then, for example, click a button and verify some things about the state of the DOM afterward.
This is in contrast with Clean Code’s recommendation of extensive unit testing, coupled with Test-First Development:
First Law You may not write production code until you have written a failing unit test. Second Law You may not write more of a unit test than is sufficient to fail, and not compiling is failing. Third Law You may not write more production code than is sufficient to pass the currently failing test.
I generally avoid doing this sort of thing, especially early on in projects. Early on you often have no idea what the right abstractions for your domain are, and you need to try a few different approaches to figure out what you are doing. If you adopt the test first approach you end up with a bunch of tests that are going to break as you explore the problem space, trying to find the right abstractions.
Further, unit testing encourages the exhaustive testing of every single function you write, so you often end up having more tests that are tied to a particular implementation of things, rather than the high level API or conceptual ideas of the module of code.
Of course, you can and should refactor your tests as you change things, but the reality is that a large and growing test suite takes on its own mass and momentum in a project, especially as other engineers join, making changes more and more difficult as they are added. You end up creating things like test helpers, mocks, etc. for your testing code.
All that code and complexity tends over time to lock you in to a particular implementation.
My preferred approach in many projects is to do some unit testing, but not a ton, early on in the project and wait until the core APIs and concepts of a module have crystallized.
At that point I then test the API exhaustively with integrations tests.
In my experience, these integration tests are much more useful than unit tests, because they remain stable and useful even as you change the implementation around. They aren’t as tied to the current codebase, but rather express higher level invariants that survive refactors much more readily.
I have also found that once you have a few higher-level integration tests, you can then do Test-Driven development, but at the higher level: you don’t think about units of code, but rather the API you want to achieve, write the tests for that API and then implement it however you see fit.
So, I think you should hold off on committing to a large test suite until later in the project, and that test suite should be done at a higher level than Test-First Development suggests.
Generally, if I can write a higher-level integration test to demonstrate a bug or feature I will try to do so, with the hope that the higher-level test will have a longer shelf life for the project.
A final coding strategy that I use is that I generally strive to minimize the number of classes/interfaces/concepts in my projects.
Clean Code does not explicitly say that you should maximize the # of classes in your system, but many recommendations it makes tend to lead to this outcome:
“Prefer Polymorphism to If/Else or Switch/Case”
“The first rule of classes is that they should be small. The second rule of classes is that they should be smaller than that.”
“The Single Responsibility Principle (SRP) states that a class or module should have one, and only one, reason to change.”
“The first thing you might notice is that the program got a lot longer. It went from a little over one page to nearly three pages in length.”
As with functions, I don’t think classes should be particularly small, or that you should prefer polymorphism to a simple (or even a long, janky) if/else statement, or that a given module or class should only have one reason to change.
And I think the last sentence here is a good hint why: you tend to end up with a lot more code which may be of little real benefit to the system.
You will often hear people criticise the idea of “God objects” and I can of course understand where this criticism comes from: an incoherent class or module with a morass of unrelated functions is obviously a bad thing.
However, I think that fear of “God objects” can tend to lead to an opposite problem: overly-decomposed software.
To balance out this fear, let’s look at one of my favorite software packages, Active Record.
Active Record provides a way for you to map ruby object to a database, it is what is called an Object/Relational Mapping tool.
And it does a great job of that, in my opinion: it makes the easy stuff easy, the medium stuff easy enough, and when push comes to shove you can kick out to raw SQL without much fuss.
(This is a great example of what I call “layering” an API.)
But that’s not all the Active Record objects are good at: they also provide excellent functionality for building HTML in the view layer of Rails. They don’t include HTML specific functionality, but they do offer functionality that is useful on the view side, such as providing an API to retrieve error messages, even at the field level.
When you are writing Ruby on Rails applications you simply pass your Active Record instances out to the view/templates.
Compare this with a more heavily factored implementation, where validation errors are handled as their own “concern”. Now you need to pass (or at least access) two different things in order to properly generate your HTML. It’s not uncommon in the Java community to adopt the DTO pattern and have another set of objects entirely distinct from the ORM layer that is passed out to the view.
I like the Active Record approach. It may not be separating concerns when looked at from a purist perspective, but my concern is often getting data from a database into an HTML document, and Active Record does that job admirably without me needing to deal with a bunch of other objects along the way.
This helps me minimize the total number of objects I need to deal with in the system.
Will some functionality creep into a model that is maybe a bit “view” flavored?
Sure, but that’s not the end of the world, and it reduces the number of layers and concepts I have to deal with. Having one class that handles retrieving data from the database, holding domain logic and serves as a vessel for presenting information to the view layer simplifies things tremendously for me.
I’ve given three examples of my codin’ dirty approach:
I think (some) big functions are good, actually
I prefer integration tests to unit tests
I like to keep my class/interface/concept count down
I’m presenting this, again, not to convince you to code the way I code, or to suggest that the way I code is “optimal” in any way.
Rather it is to give you, and especially you younger developers out there, a sense that you don’t have to write code the way that many thought leaders suggest in order to have a successful software career.
You shouldn’t be intimidated if someone calls your code “dirty”: lots of very successful software has been written that way and, if you focus on the core ideas of software engineering, you will likely be successful in spite of how “dirty” it is, and maybe even because of it!
People interested in htmx often ask us about component libraries. React and other JavaScript frameworks have great ecosystems of pre-built components that can be imported into your project; htmx doesn’t really have anything similar.
The first and most important thing to understand is that htmx doesn’t preclude you from using anything. Because htmx-based websites are often multi-page apps, each page is a blank canvas on which you can import as much or as little JavaScript as you like. If your app is largely hypermedia, but you want an interactive, React-based calendar for one page, just import it on that one page with a script tag.
We sometimes call this pattern “Islands of Interactivity”—it’s referenced in our explainers here, here, and here. Unlike JS frameworks, which are largely incompatible with each other, using islands with htmx won’t lock you into any specific paradigm.
But there’s a second way that you can re-use complex frontend functionality with htmx, and it’s Web Components!
Let’s say that you have a table that says what carnival rides everyone is signed up for:
Name
Carousel
Roller Coaster
Alex
Yes
No
Sophia
Yes
YesAlex is willing to go on the carousel but not the roller coaster, because he is scared; Sophia is not scared of either.
I built this as a regular HTML table (closing tags are omitted for clarity):
table>
tr>th>Name th>Carousel th>Roller Coaster
tr>td>Alex td>Yes td>No
tr>td>Sophia td>Yes td>Yes
table>
Now imagine we want to make those rows editable. This is a classic situation in which people reach for frameworks, but can we do it with hypermedia? Sure! Here’s a naive idea:
form hx-put=/carnival>
table>
tr>
th>Name
th>Carousel
th>Roller Coaster
tr>
tr>
td>Alex
td>select name="alex-carousel"> option selected>Yes option>No option> Maybeselect>
td>select name="alex-roller"> option>Yes option selected>No option> Maybeselect>
tr>
tr>
td>Sophia
td>select name="sophia-carousel"> option selected>Yes option>No option> Maybeselect>
td>select name="sophia-roller"> option selected>Yes option>No option> Maybeselect>
tr>
table>
button>Savebutton>
form>
That will give us this table:
Name
Carousel
Roller Coaster
Alex
Sophia
Save
That’s not too bad! The save button will submit all the data in the table, and the server will respond with a new table that reflects the updated state. We can also use CSS to make the ``s fit our design language. But it’s easy to see how this could start to get unwieldy—with more columns, more rows, and more options in each cell, sending all that information each time starts to get costly.
Let’s remove all that redundancy with a web component!
form hx-put=/carnival>
table>
tr>
th>Name
th>Carousel
th>Roller Coaster
tr>
tr>
td>Alex
td>edit-cell name="alex-carousel" value="Yes">edit-cell>
td>edit-cell name="alex-roller" value="No">edit-cell>
tr>
tr>
td>Sophia
td>edit-cell name="sophia-carousel" value="Yes">edit-cell>
td>edit-cell name="sophia-roller" value="Yes">edit-cell>
tr>
table>
button>Savebutton>
form>
We still have an entirely declarative HATEOAS interface—both current state (the value attribute) and possible actions on that state (the and elements) are efficiently encoded in the hypertext—only now we’ve expressed the same ideas a lot more concisely.
htmx can add or remove rows (or better yet, whole tables) with the web component as if were a built-in HTML element.
You’ve probably noticed that I didn’t include the implementation details for `` (although you can, of course, View Source this page to see them). That’s because they don’t matter! Whether the web component was written by you, or a teammate, or a library author, it can be used exactly like a built-in HTML element and htmx will handle it just fine.
A lot of the problems that JavaScript frameworks have supporting Web Components don’t apply to htmx.
Web Components have DOM-based lifecycles, so they are difficult for JavaScript frameworks, which often manipulate elements outside of the DOM, to work with. Frameworks have to account for some bizarre and arguably buggy APIs that behave differently for native DOM elements than they do for custom ones. Here at htmx, we agree with SvelteJS creator Rich Harris: “web components are [not] useful primitives on which to build web frameworks.”
The good news is that htmx is not really a JavaScript web framework.
The DOM-based lifecycles of custom elements work great in htmx, because everything in htmx has a DOM-based lifecycle—we get stuff from the server, and we add it to the DOM.
The default htmx swap style is to just set .innerHTML, and that works great for the vast majority of users.
That’s not to say that htmx doesn’t have to accommodate weird Web Component edge cases. Our community member and resident WC expert Katrina Scialdone merged Shadow DOM support for htmx 2.0, which lets htmx process the implementation details of a Web Component, and supporting that is occasionally frustrating. But being able to work with both the Shadow DOM and the “Light DOM” is a nice feature for htmx, and it carries a relatively minimal support burden because htmx just isn’t doing all that much.
A couple of years ago, W3C Contributor (and Web Component proponent, I think) Lea Verou wrote the following, in a blog post about “The failed promise of Web Components”:
the main problem is that HTML is not treated with the appropriate respect in the design of these components. They are not designed as closely as possible to standard HTML elements, but expect JS to be written for them to do anything. HTML is simply treated as a shorthand, or worse, as merely a marker to indicate where the element goes in the DOM, with all parameters passed in via JS.
Lea is identifying an issue that, from the perspective of 2020, would have seemed impossible to solve: the cutting-edge web developers targeted by Web Components were not writing HTML, they were writing JSX, usually with React (or Vue, or what have you). The idea that behavior belongs in the HTML was, in the zeitgeist, considered a violation of separation of concerns; disrespecting HTML was best practice.
The relatively recent success of htmx—itself now a participant in the zeitgeist—offers an alternative path: take HTML seriously again. If your website is one whose functionality can be primarily described with large-grain hypermedia transfers (we believe most of them can), then the value of being able to express more complex patterns through hypermedia increases dramatically. As more developers use htmx (and multi-page architectures generally) to structure their websites, perhaps the demand for Web Components will increase along with it.
Do Web Components “just work” everywhere? Maybe, maybe not. But they do work here.
class EditCell extends HTMLElement { connectedCallback() { this.value = this.getAttribute("value") this.name = this.getAttribute("name")
this.innerHTML = `
Yes
No
Maybe
`} }
customElements.define('edit-cell', EditCell)
Over 6 years ago, I created an open source URL shortener with Next.js and after years of working on it, I found Next.js to be much more of a burden than a help. Over the years, Next.js has changed, and so did my code so it can be compatible with those changes.
My Next.js codebase grew bigger, and its complexity increased by greater size. I had dozens of components and a list of dependencies to manage. I ended up maintaining the code constantly just to keep it alive. Sure, Next.js helped here and there, but at what cost?
I asked myself, what am I doing on my website that is so complex that needs all that JavaScript code to decide what to render and how to render on my webpage? Next.js was trying to render the webpage from the server side, so why won’t I send the HTML directly myself?
So I decided to try a new route—some might say the good ol’ route—and choose plain HTML and use the help of htmx for that.
Watch me go full in details here:
Video
Replacing my components with the equivalent HTML elements powered by htmx wasn’t exactly an easy task, but one that was worth the time. I had to view things from a different angle, and I sometimes felt strict in what user interactions I can implement, but what I created was reliable and fast.
All the build steps were gone; no more transpiling and compiling the code. What you see is what you get. Most of the dependencies became redundant and have been removed. All the main logic of the website was moved to the server side, holding one source for the truth.
In the Next.js version I had isolated components, global states, and all that JavaScript to handle the forms or update the content, and yet, everything was more intuitive with htmx. After trying it, sending and receiving HTML suddenly made sense.
Dependencies are reduced by 87% (24 to 3!)
I wrote less code by 17% (9500 LOC to 7900 LOC.) In reality the total LOC of the code base is reduced by more than 50%, since much less code is imported from the dependencies.
Web build time was reduced by 100% (there’s no build step anymore.)
Size of the website reduced by more than 85% (~800KB to ~100KB!)
These numbers signify a great improvement, however, what is important for me at the end is the user and the developer experience, which to me htmx won at both.
At Gumroad, we recently embarked on a new project called Helper. As the CEO, I was initially quite optimistic about using htmx for this project, even though some team members were less enthusiastic.
My optimism stemmed from previous experiences with React, which often felt like overkill for our needs. I thought htmx could be a good solution to keep our front-end super light.
Source with htmx - Click Image To View
In fact, I shared this sentiment with our team in Slack:
“https://htmx.org/ may be a way of adding simple interactions to start”
And initially, it seemed promising! As one of our engineers at Gumroad eloquently put it:
“HTMX is (officially) a meme to make fun of how overly complicated the JS landscape has gotten - much like tailwind is just a different syntax for inline CSS, HTMX is a different syntax for inline JS.”
However, unlike Tailwind, which has found its place in our toolkit, htmx didn’t scale for our purposes and didn’t lead to the best user experience for our customers–at least for our use case.
Here’s why:
Intuition and Developer Experience: While it would have been possible to do the right thing in htmx, we found it much more intuitive and fun to get everything working with Next.js. The development process felt natural with Next.js, whereas with htmx, it often felt unnatural and forced. For example, when building complex forms with dynamic validation and conditional fields, we found ourselves writing convoluted server-side logic to handle what would be straightforward client-side operations in React.
UX Limitations: htmx ended up pushing our app towards a Rails/CRUD approach, which led to a really poor (or at least, boring and generic) user experience by default. We found ourselves constantly fighting against this tendency, which was counterproductive. For instance, implementing a drag-and-drop interface for our workflow builder proved to be a significant challenge with htmx, requiring workarounds that felt clunky compared to the smooth experience we could achieve with React libraries.
AI and Tooling Support: It’s worth noting that AI tools are intimately familiar with Next.js and not so much with htmx, due to the lack of open-source training data. This is similar to the issue Rails faces. While not a dealbreaker, it did impact our development speed and the ease of finding solutions to problems. When we encountered issues, the wealth of resources available for React/Next.js made troubleshooting much faster.
Scalability Concerns: As our project grew in complexity, we found htmx struggling to keep up with our needs. The simplicity that initially attracted us began to feel limiting as we tried to implement more sophisticated interactions and state management. For example, as we added features like real-time collaboration and complex data visualization, managing state across multiple components became increasingly difficult with htmx’s server-centric approach.
Community and Ecosystem: The React/Next.js ecosystem is vast and mature, offering solutions to almost any problem we encountered. With htmx, we often found ourselves reinventing the wheel or compromising on functionality. This became particularly evident when we needed to integrate third-party services and libraries, which often had React bindings but no htmx equivalents.
Source with Next.js - Click Image To View
Ultimately, we ended up moving to React/Next.js, which has been a really great fit for building the complex UX we’ve been looking for. We’re happy with this decision–for now. It’s allowed us to move faster, create more engaging user experiences, and leverage a wealth of existing tools and libraries.
Gumroad Helper Before & After - Click Image To View
This experience has reinforced a valuable lesson: while it’s important to consider lightweight alternatives, it’s equally crucial to choose technologies that can grow with your project and support your long-term vision. For Helper, React and Next.js have proven to be that choice.
Since we’ve moved there, we’ve been able to seriously upgrade our app’s user experience for our core customers.
Drag-and-Drop Functionality: One of the key features of our workflow builder is the ability to reorder steps through drag-and-drop. While it’s possible to implement drag-and-drop with htmx, we found that the available solutions felt clunky and required significant custom JavaScript. In contrast, React ecosystem offers libraries like react-beautiful-dnd that provide smooth, accessible drag-and-drop with minimal setup.
Complex State Management: Each workflow step has its own set of configurations and conditional logic. As users edit these, we need to update the UI in real-time to reflect changes and their implications on other steps. With htmx, this would require numerous server roundtrips or complex client-side state management that goes against htmx’s server-centric philosophy. React’s state management solutions (like useState or more advanced options like Redux) made this much more straightforward.
Dynamic Form Generation: The configuration for each step type is different and can change based on user input. Generating these dynamic forms and handling their state was more intuitive with React’s component model. With htmx, we found ourselves writing more complex server-side logic to generate and validate these forms.
Real-time Collaboration: While not visible in this screenshot, we implemented features allowing multiple users to edit a workflow simultaneously. Implementing this with WebSockets and React was relatively straightforward, whereas with htmx, it would have required more complex server-side logic and custom JavaScript to handle real-time updates.
Performance Optimization: As workflows grew larger and more complex, we needed fine-grained control over rendering optimizations. React’s virtual DOM and hooks like useMemo and useCallback allowed us to optimize performance in ways that weren’t as readily available or intuitive with htmx.
It’s important to note that while these challenges aren’t insurmountable with htmx, we found that addressing them often led us away from htmx’s strengths and towards solutions that felt more natural in a JavaScript-heavy environment. This realization was a key factor in our decision to switch to React and Next.js.
We acknowledge that htmx may be a great fit for many projects, especially those with simpler interaction models or those built on top of existing server-rendered applications. Our experience doesn’t invalidate the benefits others have found in htmx. The key is understanding your project’s specific needs and choosing the tool that best aligns with those requirements.
In our case, the complex, stateful nature of Helper’s interface made React and Next.js a better fit. However, we continue to appreciate htmx’s approach and may consider it for future projects where its strengths align better with our needs.
That said, we’re always open to reevaluating our tech stack as our needs evolve and new technologies emerge. Who knows what the future might bring?
img, video { max-width: 100%; margin: 10px; }
I often encounter discussions on Reddit and YCombinator where newer developers seek tech stack advice. Inevitably, someone claims it’s impossible to build a high-quality application without using a single-page application (SPA) framework like React or AngularJS. This strikes me as odd because, even before the SPA revolution, many popular multi-page web applications offered excellent user experiences.
Two years ago, I set out to build an observability platform and chose to experiment with a multi-page application (MPA) approach using HTMX. I wondered: Would a server-rendered MPA be inadequate for a data-heavy application, considering that most observability platforms are built on ReactJS?
What I discovered is that you can create outstanding server-rendered applications if you pay attention to certain details.
Here are some common MPA myths and what I’ve learned about them.
The perception that MPA page transitions are slow is widespread—and not entirely unfounded—since this is the default behavior of browsers. However, browsers have made significant improvements over the past decade to mitigate this issue.
To illustrate, in the video below, a full page reload with the cache disabled takes 2.90 seconds until the DOMContentLoaded event fires. I recorded this at a café with poor Wi-Fi, but let’s use this as a reference point. Keep that number in mind.
It is common to reduce load times in MPAs using libraries such as PJAX, Turbolinks, and even HTMX Boost. These libraries hijack the page reload using Javascript and swap out only the HTML body element between transitions. That way, most of the page’s head section assets don’t need to be reloaded or re-downloaded.
But there’s a lesser known way of reducing how much assets are re-downloaded or evaluated during page transitions.
Frontend developers who have built Progressive Web Applications (PWA) with SPA frameworks might know about service workers.
For those of us who are not frontend or PWA developers, service workers are a built-in feature of browsers. They let you write Javascript code that sits between your users and the network, intercepting requests and deciding how the browser handles them.
Due to its association with the PWA trend, service workers are only ordinary among SPA developers, and developers need to realize that this technology can also be used for regular Multi-Page Applications.
In the video demonstration, we enable a service worker to cache and refresh the current page. You’ll notice that there’s no flicker when clicking the link to reload the page, resulting in a smoother user experience.
Moreover, instead of transmitting over 2 MB of static assets as before, the browser now only fetches 84 KB of HTML
content—the actual page data. This optimization reduces the DOMContentLoaded event time from 2.9 seconds to under 500
milliseconds. Impressively, this improvement is achieved without using HTMX Boost, PJAX, or Turbolinks.
You might be wondering how to replicate these performance gains in your own MPA. Here’s a simple guide:
Create a sw.js File: This is your service worker script that will manage caching and network requests.
List Files to Cache: Within the service worker, specify all the assets (HTML, CSS, JavaScript, images) that should be cached.
Define Caching Strategies: Indicate how each type of asset should be cached—for example, whether they should be cached permanently or refreshed periodically.
By implementing a service worker, you effectively tell the browser how to handle network requests and caching, leading to faster load times and a more seamless user experience.
While it’s possible to write service workers by hand—and there are excellent resources like this MDN article to help you—I prefer using Google’s Workbox library to automate the process.
Install Workbox: Install Workbox via npm or your preferred package manager:
npm install workbox-cli --global
Generate a Workbox Configuration file: Run the following command to create a configuration file:
workbox wizard
Configure Asset Handling: In the generated workbox-config.js file, define how different assets should be
cached. Use the urlPattern property—a regular expression—to match specific HTTP requests. For each matching
request, specify a caching strategy, such as CacheFirst or NetworkFirst.
Build the Service Worker: Run the Workbox build command to generate the sw.js file based on your configuration:
workbox generateSW workbox-config.js
Register the Service Worker in Your Application: Add the following script to your HTML pages to register the service worker:
script>
if ('serviceWorker' in navigator) {
window.addEventListener('load', function() {
navigator.serviceWorker.register('/sw.js').then(function(registration) {
console.log('ServiceWorker registration successful with scope: ', registration.scope);
}, function(err) {
console.log('ServiceWorker registration failed: ', err);
});
});
}
script>
By following these steps, you instruct the browser to serve cached assets whenever possible, drastically reducing load times and improving the overall performance of your multi-page application.
Image showing the registered service worker from the Chrome browser console.
Speculation Rules API: Prerender pages for instant page navigation.If you have used htmx-preload or instantpage.js, you’re familiar with prerendering and the problem the “Speculation Rules API” aims to solve. The Speculation Rules API is designed to improve performance for future navigations. It has an expressive syntax for specifying which links should be prefetched or prerendered on the current page.
Speculation rules configuration example
The script above is an example of how speculation rules are configured. It is a Javascript object, and without going into detail, you can see that it uses keywords such as “where,” “and,” “not,” etc. to describe what elements should either be prefetched or prerendered.
Example impact of prerendering (Chrome Team)
From the last sections, you know that service workers can cache everything and make our apps operate entirely offline. But what if we want to save offline POST requests and retry them when there is internet?
The configuration javascript file above shows how to configure Workbox to support two common offline scenarios. Here, you see background Sync, where we ask the service worker to cache any failed requests due to the internet and retry it for up to 24 hours.
Below, we define an offline catch Handler, triggered when a request is made offline. We can return a template partial with HTML or a JSON response or dynamically build a response based on the request input. The sky is the limit here.
In the service worker videos, we already saw that this will not happen if we configure caching and prerendering. However, this myth was not generally true until 2019. Since 2019, most browsers withhold painting the next screen until all the required assets for the next page are available or a timeout is reached, resulting in no flash of white while transitioning between both pages. This only works when navigating within the same origin/domain.
Paint holding documentation on chrome.com.
The advent of single-page application frameworks made custom transitions between pages more popular. The allure of different navigation styles comes from completely taking control of page navigation from the browsers. In practice, such transitions have mostly been popular within the demos at web dev conference talks.
Cross Document Transitions documentation on chrome.com.
This remains a common argument for single-page applications, especially on Reddit and Hacker News comment sections. However, browsers have been working towards solving this problem natively for the last couple of years. Chrome 126 rolled out cross-document view transitions. This means we can build our MPAs to include those fancy animations and transitions between pages using CSS only or CSS and Javascript.
My favorite bit is that we might be able to create lovely cross-document transitions with CSS only:
You can quickly learn more on the Google Chrome announcement page
This link hosts a multi-page application demo, where you can play around with a rudimentary server-rendered application using the cross-document view transitions API to simulate a stack-based animation.
I’ve heard this a lot when HTMX is discussed. So, there might be some confusion caused by the HTMX positioning. But you don’t have to do everything server-side. Many HTMX and regular MPA users continue to use Javascript, Alpine, or Hyperscript where appropriate.
In situations where robust interactivity is helpful, you can lean into the component islands architecture using WebComponents or any javascript framework (React, Angular, etc.) of your choice. That way, instead of your entire application being an SPA, you can leverage those frameworks specifically for the bits of your application that need that interactivity.
The example above shows a very interactive search component in the APItoolkit. It’s a web component implemented with lit-element, a zero-compile-step library for writing web components. So, the entire web component event fits in a Javascript file.
The speed of direct DOM operations was a major motivation for building ReactJS on and popularizing the virtual DOM technology. While virtual DOM operations can be faster than direct DOM operations, this is only true for applications that perform many complex operations and refresh in milliseconds, where that performance might be noticeable. But most of us are not building such software.
The Svelte team wrote an excellent article titled “Virtual DOM is pure Overhead.” I recommend reading it, as it better explains why Virtual DOM doesn’t matter for most applications.
With the advancements in browser tech, you can avoid writing a lot of client-side Javascript in the first place. For
example, a standard action on the web is to show and hide things based on a button click or toggle. These days, you can
show and hide elements with only CSS and HTML, for example, by using an HTML input checkbox to track state. We can style
an HTML label as a button and give it a for="checkboxID“ attribute, so clicking the label toggles the checkbox.
toggle content
Content to be toggled when label/btn is clicked
We can combine such a checkbox with HTMX intersect to fetch content from an endpoint when the button is clicked.
input id="published" class="peer" type="checkbox" name="status"/>
div
class="hidden peer-checked:block"
hx-trigger="intersect once"
hx-get="/log-item"
>Shell/Loading text etc
div>
All the classes above are vanilla Tailwind CSS classes, but you can also write the CSS by hand. Below is a video of that code being used to hide or reveal log items in the log explorer.
This may or may not be true.
I like to argue that some of the most productive days of the web were the PHP and JQuery spaghetti days. A lot of software was built at that time, including many of the popular internet brands we know today. Most of them were built as so-called spaghetti codes, which helped them ship their products early and survive long enough to refactor and not be spaghetti.
The entire point of this talk is to show you that a lot is possible with browsers in 2024. While we were not looking, browsers have closed the gap and borrowed the best ideas from the single-page application revolution. For example, WebComponents exist thanks to the lessons we learned from single-page applications.
So now, we can build very interactive, even offline web applications using mostly browser tools—HTML, CSS, maybe some Javascript—and still not sacrifice much in terms of user experience.
I’m very happy to announce the release of htmx 2.0. This release ends support for Internet Explorer and tightens up some defaults, but does not change most of the core functionality or the core API of the library.
Note that we are not marking 2.0 as latest in NPM
because we do not want to force-upgrade users who are relying on non-versioned CDN URLs for htmx. Instead, 1.x will
remain latest and the 2.0 line will remain next until Jan 1, 2025. The website, however, will reference 2.0.
Most 1.x extensions will work with 2.x, however the SSE extension did have a break and must be upgraded.
The older extensions remain in the /dist/ext directory so as to not break the URLs of CDNs like unpkg, but please
move to the new extension URLs going forward
We removed the deprecated hx-sse and hx-ws attributes in favor of the extensions, which were available and
recommended in 1.x.
HTTP DELETE requests now use parameters, rather than form encoded bodies, for their payload (This is in accordance w/ the spec.)
We now provide specific files in /dist for the various JavaScript module styles:
ESM Modules: /dist/htmx.esm.js
AMD Modules: /dist/htmx.amd.js
CJS Modules: /dist/htmx.cjs.js
The /dist/htmx.js file continues to be browser-loadable
The hx-on attribute, with its special syntax, has been removed in favor of the less-hacky hx-on: syntax.
htmx.config.scrollBehavior was changed to 'instant' from 'smooth'
As mentioned previously, DELETE requests now use query parameters, rather than a form-encoded body. This can
be reverted by setting htmx.methodsThatUseUrlParams to the value ['get'],
htmx.config.selfRequestsOnly now defaults to true rather than false
Not much, really:
The selectAndSwap() internal API method was replaced with the public (and much better) swap() method
Web Component support has been improved dramatically
And the biggest feature of this release: the website now supports dark mode! (Thanks @pokonski!)
A complete upgrade guide can be found here:
htmx 1.x -> 2.x Migration Guide
If you require IE compatibility, the 1.x will continue to be supported for the foreseeable future.
htmx 2.0 can be installed via a package manager referencing version 2.0.0, or can be linked via a CDN:
script src="https://unpkg.com/htmx.org@2.0.0/dist/htmx.min.js">script>
or Downloaded