Pulumi Blog

For about two years, the unit of work with a coding agent was the prompt. You wrote a good one, you gave it enough context, you read what came back, and you wrote the next one. The agent was a tool, and you were holding it the entire time, one turn after another.

That part is ending. Addy Osmani, a director of AI at Google Cloud, has a name for what replaces it, and I have not stopped thinking about it since: loop engineering. You stop being the person who prompts the agent. You design the loop that prompts it for you.

In my phrasing: you stop being the thing that runs, and start designing the thing that runs. The leverage moves up a layer. What I want to do here is take an honest look at the pieces, and at the part nobody automates.

The leverage moved up a layer

The people building these tools have already made the jump. Peter Steinberger has been posting it as a monthly reminder.

Peter Steinberger (@steipete) on X.

Boris Cherny, who heads Claude Code at Anthropic, says the same thing about his own job. He does not prompt Claude anymore. He has loops running that prompt Claude and decide what to do next, scanning the issue tracker, the team chat, and the timeline for what to build. “My job is to write loops.”

A loop is a goal that prompts itself. You set the purpose, and the system keeps iterating until it’s met. In practice it finds the work, hands it out, checks the result, writes down what got finished, and decides the next thing, then it pokes the agent instead of you. You build that small system once and let it run.

Look closer, and a loop is really two loops nested. The inner one does the work against a spec. The outer one decides what the work should be: it watches an issue tracker, an error feed, a changelog, then writes the next spec and hands it down. Most people are still running that outer loop by hand, in their head, and calling it a backlog.

The part that surprised me is that this is barely a tooling problem anymore. A year ago a loop meant a pile of bash you wrote and maintained forever. Now the pieces ship inside the products, and the same shapes show up in Claude Code and in Codex. Osmani puts loop engineering one floor above the harness, the context and tooling you wire around a single agent. I wrote about that harness a couple of weeks ago. The loop is the thing that runs on top of it: it runs on a timer, it spawns helpers, and it feeds itself.

The five pieces, and the one that holds them together

Strip loop engineering down and you get roughly five building blocks, plus one place to remember things. Both Claude Code and Codex have all five now. The names differ here and there; the capability is the same.

1. Automations are the heartbeat. They are what make a loop an actual loop and not one run you did once. A prompt or command on a cadence, a scheduled task, a hook that fires at a point in the agent’s lifecycle, or a job on CI that keeps running after you close the laptop. Discovery and triage run themselves, and the findings that matter come to you.
2. Worktrees keep parallel from turning into chaos. The second you run more than one agent, the files start colliding. Two agents writing the same file is the same headache as two engineers committing to the same lines with nobody talking first. A git worktree is a separate working directory on its own branch, so one agent’s edits cannot touch another’s checkout.
3. Skills are intent, written down. An agent starts every session cold and fills any hole in your intent with a confident guess. A skill is that intent written on the outside: the conventions, the build steps, the “we don’t do it like this because of that one incident,” recorded once where the agent reads it every run. Without skills, the loop re-derives your whole project from zero every cycle.
4. Connectors let the loop touch your real tools. Built on MCP, they let the agent read the issue tracker, query a database, hit a staging API, or drop a message in chat. This is the difference between an agent that says “here is the fix” and a loop that opens the pull request, links the ticket, and pings the channel once CI goes green.
5. Sub-agents keep the maker away from the checker. The model that wrote the code is far too generous grading its own homework. A second agent with different instructions, and sometimes a different model, catches the things the first one talked itself into. Worktrees and a cold-context reviewer are two pieces I have written about before, back when the question was running agents in parallel without them trampling each other.

Then the sixth thing: memory. A markdown file, a Linear board, a state file, anything that lives outside the single conversation and holds what is done and what is next. It sounds too dumb to matter, and it’s the whole game. The model forgets everything between runs, so the memory has to live on disk, not in the context window. The agent forgets. The repo does not.

What makes a loop hold together

A loop running unattended is also a loop making mistakes unattended. The one thing that keeps it honest is verification, and verification needs an oracle, something outside the model that returns a hard yes or no. Passing tests, a clean build, a green pipeline, a real production signal. Without an oracle, the loop compounds confidently wrong work, faster than you can read it.

The cleanest version of this already ships in the tools. Claude Code’s /goal keeps working across turns until a condition you actually wrote holds, something like “every test in auth/ passes and lint is clean,” and after every turn a separate, faster model reads the transcript and decides whether you are there yet. The agent that wrote the code is not the one that grades it. That is the maker-and-checker split applied to the stop condition itself. Codex’s /goal reaches the same finish line a different way: the agent audits its own work against the evidence before it can call the goal done.

What the loop still won’t do for you

The loop changes the shape of the work. It does not take it off your desk. And a few things get sharper as the loop gets better, not softer.

Verification is still on you. The split reviewer is what makes “it’s done” mean something, but “done” is a claim, not a proof. Your job is still to ship code you confirmed works, which is harder to remember when the diff arrived while you were at lunch.

The bill comes in two currencies. Tokens and attention. A single unattended run can burn through millions of tokens, and that is only worth it when the tokens buy something worth more than they cost. The quieter trap is the second currency: memory is what lets a loop compound over time, and slop compounds right alongside it. A loop pointed at a vague goal does not get tired and stop. It gets faster.

Your understanding rots if you let it. The faster the loop ships code you did not write, the wider the gap between what exists in the repo and what you actually understand. A smooth loop grows that gap faster, not slower, unless you read what it made. The comfortable posture, where you stop having an opinion and take whatever the loop gives back, is the risky one. Two engineers can build the exact same loop and get opposite results, one moving faster on work they understand deeply, the other avoiding the work entirely. The loop cannot tell which one you are.

When the loop reaches production

Most of this thinking grew up around application code, where a bad run costs you a revert. When the loop reaches into infrastructure, the blast radius is a production outage rather than a revert, and the verification bar has to rise to meet it. The upside is that infrastructure hands the loop a better oracle than application code does. A plan diff is deterministic and machine-readable, a policy check returns a hard verdict, and drift and cost are numbers you can put a threshold on. A reviewer, whether human or agent, can read the change cold, with no memory of the prompt that produced it. That cold-context check is exactly what an unattended loop needs, and it’s the reason an infrastructure loop can be built to hold together while you sleep. Pulumi Neo reasons over the state graph directly, so the checker grounds every claim in what the change actually does, not in what the writer says it does.

Where to start

Pick the loop you can actually trust first. In order:

1. Start where “done” is unambiguous. CI triage, dependency bumps, a flaky-test hunt, a failing job you keep re-running by hand. Loops need an oracle, so begin where the oracle already exists.
2. Write the memory file before the loop. One markdown file, or a board. What is done, what is next, what was tried and failed. This is the spine, and everything else hangs off it.
3. Split the checker from the maker. Use /goal with a verifiable condition, or a second agent with its own instructions. Never let the agent that did the work be the one that decides the work is finished.
4. Cap it, then read everything. A max-iteration count, a token budget, a teardown step. Run it once, end to end, then read every line it shipped. The first run is the measurement, not the payoff.

Then look at what you built. You designed it once, and it ran without you steering each step. That is the real shift. But the leverage only holds if you wire the loop like an engineer, not like someone looking for permission to stop thinking. Read what it ships. Keep an opinion. Judgment is the one part that does not move up a layer.

The loop will do the typing. The thinking is the work.

See how Pulumi Neo closes the loop on your infrastructure

A git tag is how many teams mark a release as ready. Pulumi Deployments can now act on that signal directly: configure a tag-based trigger, push a version tag like v1.2.0, and Pulumi automatically runs pulumi up for your stack. No extra pipeline glue, no manual click — your release tag is the deployment.

Why tags?

Push to Deploy has long let you preview changes on a pull request and update a stack when commits merge to a branch. That branch-based model is a great fit for continuous delivery to shared development and QA environments, where every merge should flow straight through.

But promotion to production is often deliberate, not continuous. You merge throughout the day, then decide — separately — that a particular commit is the release. The conventional way to record that decision is a git tag: v1.2.0, 2026.06.0, release-2026-06-04. Tagging is already part of most teams’ release rituals.

Tag-based triggers connect that ritual to your infrastructure. Instead of wiring up a separate CI job to call the Pulumi Deployments REST API on a tag event, you configure the trigger once in your stack’s deployment settings and let Pulumi handle the rest.

How it works

Tag triggers are controlled by two settings on your stack’s deployment configuration:

• Run updates for pushed tags — a toggle that enables running pulumi up when a matching tag is pushed.
• Tag filters — a list of glob patterns that decide which tag names qualify.

Tag filters use the same model as the path filters you may already know, except the patterns match against the tag name rather than changed file paths. A few examples:

• v* — deploy on any tag beginning with v, such as v1.0.0 and v2.3.1.
• v* plus !*-rc* — deploy on release tags but skip release candidates like v1.2.0-rc1.
• 2026.* — deploy on calendar-versioned releases such as 2026.06.0.

Filters prefixed with ! are exclusions, and an exclusion always wins over an include. With no filters configured and the toggle on, every tag push deploys. Deleting a tag never triggers a deployment.

When a tag push kicks off a deployment, Pulumi sets the PULUMI_CI_TAG_NAME environment variable to the tag name. Your pre-run commands or your Pulumi program can read it — for example, to stamp the release version onto a resource tag or an application config value.

Works across every VCS integration

Tag triggers are available across all five version control integrations: GitHub, GitLab, Bitbucket, Azure DevOps, and Custom VCS.

Get started

You can configure tag triggers wherever you manage deployment settings today — the Pulumi Cloud console, the REST API, or as code with the pulumiservice.DeploymentSettings resource.

To try it out:

1. Open a stack’s Settings > Deploy tab in the Pulumi Cloud console.
2. Enable Run updates for pushed tags and add a tag filter such as v*.
3. Push a tag — git tag v1.0.0 && git push origin v1.0.0 — and watch the deployment run.

For the full details, see the deployment triggers and tag filtering documentation. We’d love to hear how you put tag-based deployments to work.

If you run AI tools and agents, you’ve probably accepted three tradeoffs: your data leaves your network, you can’t work offline, and your bill scales with usage.

Open-weight models now run well on consumer hardware. Once the model is on your machine, your data stays local, inference works offline, and tokens cost nothing. If you own a modern Mac, you can run a high-quality model yourself.

Gemma 4 is an open-weights model family from Google. This post focuses on Gemma 4 12 B, released in June 2026, using Unsloth’s Q8_0 GGUF. The 12 B model fits comfortably on a modern Mac while leaving enough headroom for local llama.cpp and a chat UI.

We’ll use llama.cpp for host-native inference, k3d for a local Kubernetes cluster, Pulumi for infrastructure as code, and Tailscale for secure access.

Prerequisites

This setup was validated on the following hardware:

• macOS 26 Tahoe, version 26.5
• MacBook Pro with Apple M3 Max
• 36 GB RAM

On this machine, llama.cpp reported about 20 output tokens per second for a 160-token validation response with unsloth/gemma-4-12b-it-GGUF, gemma-4-12b-it-Q8_0.gguf, and a 131,072-token context. Sustained throughput varies by prompt length, thermal state, and llama.cpp settings.

You’ll need brew, docker, pulumi, and tailscale installed. We’ll also install k3d during the process.

Run Gemma 4 with host-native llama.cpp

We use llama.cpp directly on macOS to leverage Apple Metal acceleration. Running the LLM on the host is more efficient than trying to pass GPU access into a local Kubernetes VM.

Install the build tools:

<span class="line"><span class="cl">brew install cmake git
</span></span>

Then build llama.cpp from source and download the multimodal projector. In validation, Homebrew llama.cpp 9430 could run text inference, but it could not load the new Gemma 4 12 B projector and failed with unknown projector type: gemma4uv. Building current llama.cpp from source fixed that.

<span class="line"><span class="cl"><span class="nv">llm_home</span><span class="o">=</span><span class="s2">"</span><span class="nv">$HOME</span><span class="s2">/pulumi-gemma4-llm"</span>
</span></span><span class="line"><span class="cl">mkdir -p <span class="s2">"</span><span class="nv">$llm_home</span><span class="s2">/models"</span> <span class="s2">"</span><span class="nv">$llm_home</span><span class="s2">/logs"</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">if</span> <span class="o">[</span> ! -d <span class="s2">"</span><span class="nv">$llm_home</span><span class="s2">/llama.cpp/.git"</span> <span class="o">]</span><span class="p">;</span> <span class="k">then</span>
</span></span><span class="line"><span class="cl"> git clone --depth <span class="m">1</span> https://github.com/ggml-org/llama.cpp.git <span class="s2">"</span><span class="nv">$llm_home</span><span class="s2">/llama.cpp"</span>
</span></span><span class="line"><span class="cl"><span class="k">fi</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">cmake -S <span class="s2">"</span><span class="nv">$llm_home</span><span class="s2">/llama.cpp"</span> <span class="se">\
</span></span></span><span class="line"><span class="cl"> -B <span class="s2">"</span><span class="nv">$llm_home</span><span class="s2">/llama.cpp/build"</span> <span class="se">\
</span></span></span><span class="line"><span class="cl"> -DGGML_METAL<span class="o">=</span>ON <span class="se">\
</span></span></span><span class="line"><span class="cl"> -DGGML_BLAS<span class="o">=</span>ON <span class="se">\
</span></span></span><span class="line"><span class="cl"> -DCMAKE_BUILD_TYPE<span class="o">=</span>Release
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">cmake --build <span class="s2">"</span><span class="nv">$llm_home</span><span class="s2">/llama.cpp/build"</span> --target llama-server -j <span class="m">10</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">curl -L --fail <span class="se">\
</span></span></span><span class="line"><span class="cl"> --output <span class="s2">"</span><span class="nv">$llm_home</span><span class="s2">/models/mmproj-F16.gguf"</span> <span class="se">\
</span></span></span><span class="line"><span class="cl"> https://huggingface.co/unsloth/gemma-4-12b-it-GGUF/resolve/main/mmproj-F16.gguf
</span></span>

Then download and run the model with this command:

<span class="line"><span class="cl"><span class="s2">"</span><span class="nv">$HOME</span><span class="s2">/pulumi-gemma4-llm/llama.cpp/build/bin/llama-server"</span> <span class="se">\
</span></span></span><span class="line"><span class="cl"> --hf-repo unsloth/gemma-4-12b-it-GGUF <span class="se">\
</span></span></span><span class="line"><span class="cl"> --hf-file gemma-4-12b-it-Q8_0.gguf <span class="se">\
</span></span></span><span class="line"><span class="cl"> --mmproj <span class="s2">"</span><span class="nv">$HOME</span><span class="s2">/pulumi-gemma4-llm/models/mmproj-F16.gguf"</span> <span class="se">\
</span></span></span><span class="line"><span class="cl"> --host 127.0.0.1 <span class="se">\
</span></span></span><span class="line"><span class="cl"> --port <span class="m">18080</span> <span class="se">\
</span></span></span><span class="line"><span class="cl"> --ctx-size <span class="m">131072</span> <span class="se">\
</span></span></span><span class="line"><span class="cl"> --parallel <span class="m">1</span> <span class="se">\
</span></span></span><span class="line"><span class="cl"> --jinja <span class="se">\
</span></span></span><span class="line"><span class="cl"> --reasoning off
</span></span>

We use port 18080 because 8080 is commonly used and is likely to conflict with another service you may already have running locally. If your port 8080 is free, you can use it and adjust the Pulumi config later.

The model file is about 12.65 GB, and the projector is about 116 MB. Gemma 4 12 B advertises a 131,072-token context, and this Mac loaded that full context with --parallel 1. llama.cpp projected about 15.1 GiB of Apple Metal device memory for the text model and about 258 MiB worst-case memory for the projector, leaving enough headroom for Open WebUI and the rest of the local stack. The --reasoning off flag keeps OpenAI-compatible chat responses visible in clients that do not read separate reasoning fields.

With --mmproj, /v1/models advertised capabilities: ["completion","multimodal"]. In local validation, Open WebUI accepted an uploaded Pulumi logo image and Gemma 4 described it correctly. A small WAV file also worked through the OpenAI-compatible input_audio request shape, though llama.cpp logs still mark audio input as experimental.

Verify the LLM API

Open a new terminal and check if llama.cpp is responding:

<span class="line"><span class="cl">curl http://127.0.0.1:18080/v1/models
</span></span>

The /v1/models endpoint should return the model ID unsloth/gemma-4-12b-it-GGUF. Now try a chat completion:

<span class="line"><span class="cl">curl http://127.0.0.1:18080/v1/chat/completions <span class="se">\
</span></span></span><span class="line"><span class="cl"> -H <span class="s2">"Content-Type: application/json"</span> <span class="se">\
</span></span></span><span class="line"><span class="cl"> -d <span class="s1">'{
</span></span></span><span class="line"><span class="cl"><span class="s1"> "model": "unsloth/gemma-4-12b-it-GGUF",
</span></span></span><span class="line"><span class="cl"><span class="s1"> "messages": [{"role": "user", "content": "Reply with exactly: OK"}],
</span></span></span><span class="line"><span class="cl"><span class="s1"> "temperature": 0,
</span></span></span><span class="line"><span class="cl"><span class="s1"> "max_tokens": 32
</span></span></span><span class="line"><span class="cl"><span class="s1"> }'</span>
</span></span>

The chat prompt Reply with exactly: OK should return content OK. In validation, llama.cpp reported an output token velocity of about 20 tokens per second for a longer 160-token response.

Keep llama.cpp running after reboot

For a permanent setup, put the llama.cpp startup script and logs under a folder in your home directory and let launchd restart it when you sign in:

<span class="line"><span class="cl"><span class="nv">llm_home</span><span class="o">=</span><span class="s2">"</span><span class="nv">$HOME</span><span class="s2">/pulumi-gemma4-llm"</span>
</span></span><span class="line"><span class="cl">mkdir -p <span class="s2">"</span><span class="nv">$llm_home</span><span class="s2">/logs"</span> <span class="s2">"</span><span class="nv">$HOME</span><span class="s2">/Library/LaunchAgents"</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">cat > <span class="s2">"</span><span class="nv">$llm_home</span><span class="s2">/start-llama-server.sh"</span> <span class="s"><<'EOF'
</span></span></span><span class="line"><span class="cl"><span class="s">#!/bin/zsh
</span></span></span><span class="line"><span class="cl"><span class="s">set -euo pipefail
</span></span></span><span class="line"><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="cl"><span class="s">export PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"
</span></span></span><span class="line"><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="cl"><span class="s">exec "$HOME/pulumi-gemma4-llm/llama.cpp/build/bin/llama-server" \
</span></span></span><span class="line"><span class="cl"><span class="s"> --hf-repo unsloth/gemma-4-12b-it-GGUF \
</span></span></span><span class="line"><span class="cl"><span class="s"> --hf-file gemma-4-12b-it-Q8_0.gguf \
</span></span></span><span class="line"><span class="cl"><span class="s"> --mmproj "$HOME/pulumi-gemma4-llm/models/mmproj-F16.gguf" \
</span></span></span><span class="line"><span class="cl"><span class="s"> --host 127.0.0.1 \
</span></span></span><span class="line"><span class="cl"><span class="s"> --port 18080 \
</span></span></span><span class="line"><span class="cl"><span class="s"> --ctx-size 131072 \
</span></span></span><span class="line"><span class="cl"><span class="s"> --parallel 1 \
</span></span></span><span class="line"><span class="cl"><span class="s"> --jinja \
</span></span></span><span class="line"><span class="cl"><span class="s"> --reasoning off
</span></span></span><span class="line"><span class="cl"><span class="s">EOF</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">chmod +x <span class="s2">"</span><span class="nv">$llm_home</span><span class="s2">/start-llama-server.sh"</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">cat > <span class="s2">"</span><span class="nv">$HOME</span><span class="s2">/Library/LaunchAgents/com.pulumi.gemma4.llama-server.plist"</span> <span class="s"><<EOF
</span></span></span><span class="line"><span class="cl"><span class="s"><?xml version="1.0" encoding="UTF-8"?>
</span></span></span><span class="line"><span class="cl"><span class="s"><!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
</span></span></span><span class="line"><span class="cl"><span class="s"><plist version="1.0">
</span></span></span><span class="line"><span class="cl"><span class="s"><dict>
</span></span></span><span class="line"><span class="cl"><span class="s"> <key>Label</key>
</span></span></span><span class="line"><span class="cl"><span class="s"> <string>com.pulumi.gemma4.llama-server</string>
</span></span></span><span class="line"><span class="cl"><span class="s"> <key>ProgramArguments</key>
</span></span></span><span class="line"><span class="cl"><span class="s"> <array>
</span></span></span><span class="line"><span class="cl"><span class="s"> <string>$llm_home/start-llama-server.sh</string>
</span></span></span><span class="line"><span class="cl"><span class="s"> </array>
</span></span></span><span class="line"><span class="cl"><span class="s"> <key>WorkingDirectory</key>
</span></span></span><span class="line"><span class="cl"><span class="s"> <string>$llm_home</string>
</span></span></span><span class="line"><span class="cl"><span class="s"> <key>RunAtLoad</key>
</span></span></span><span class="line"><span class="cl"><span class="s"> <true/>
</span></span></span><span class="line"><span class="cl"><span class="s"> <key>KeepAlive</key>
</span></span></span><span class="line"><span class="cl"><span class="s"> <true/>
</span></span></span><span class="line"><span class="cl"><span class="s"> <key>StandardOutPath</key>
</span></span></span><span class="line"><span class="cl"><span class="s"> <string>$llm_home/logs/llama-server.out.log</string>
</span></span></span><span class="line"><span class="cl"><span class="s"> <key>StandardErrorPath</key>
</span></span></span><span class="line"><span class="cl"><span class="s"> <string>$llm_home/logs/llama-server.err.log</string>
</span></span></span><span class="line"><span class="cl"><span class="s"></dict>
</span></span></span><span class="line"><span class="cl"><span class="s"></plist>
</span></span></span><span class="line"><span class="cl"><span class="s">EOF</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">launchctl bootout gui/<span class="k">$(</span>id -u<span class="k">)</span>/com.pulumi.gemma4.llama-server 2>/dev/null <span class="o">||</span> <span class="nb">true</span>
</span></span><span class="line"><span class="cl">launchctl bootstrap gui/<span class="k">$(</span>id -u<span class="k">)</span> <span class="s2">"</span><span class="nv">$HOME</span><span class="s2">/Library/LaunchAgents/com.pulumi.gemma4.llama-server.plist"</span>
</span></span><span class="line"><span class="cl">launchctl kickstart -k gui/<span class="k">$(</span>id -u<span class="k">)</span>/com.pulumi.gemma4.llama-server
</span></span>

Check the launchd service and llama.cpp logs:

<span class="line"><span class="cl">launchctl print gui/<span class="k">$(</span>id -u<span class="k">)</span>/com.pulumi.gemma4.llama-server
</span></span><span class="line"><span class="cl">tail -f <span class="s2">"</span><span class="nv">$HOME</span><span class="s2">/pulumi-gemma4-llm/logs/llama-server.err.log"</span>
</span></span>

If you want to stop llama.cpp later, unload the launchd service:

<span class="line"><span class="cl">launchctl bootout gui/<span class="k">$(</span>id -u<span class="k">)</span>/com.pulumi.gemma4.llama-server
</span></span>

Deploy Open WebUI with Pulumi and k3d

Now we’ll deploy Open WebUI into a local Kubernetes cluster. This provides a polished chat interface that connects to our host-native LLM.

First, install k3d if you haven’t already:

<span class="line"><span class="cl">brew install k3d
</span></span>

Create a new cluster for this project:

<span class="line"><span class="cl">k3d cluster create pulumi-gemma4-blog-qa
</span></span>

We’ll use the Pulumi program in pulumi/examples. This program defaults to runtimeMode=host, which creates a Kubernetes ExternalName service pointing to your host machine.

Why not run the LLM inside Kubernetes on this Mac? Pulumi can do that, and the example supports it with runtimeMode=cluster, but that path is meant for Linux hosts with NVIDIA or AMD GPU device plugins.

On macOS, llama.cpp enables Metal by default, and Metal acceleration is available to native macOS processes. k3d runs Linux containers through Docker Desktop, so those pods do not get direct access to the Mac’s Metal device. Docker’s own vLLM Metal announcement calls out the same boundary: Metal-backed inference runs natively on the host because there is no Metal GPU passthrough for containers. That is why this setup keeps inference host-native and lets Pulumi manage the Kubernetes UI, service wiring, and optional Tailscale access around it.

Clone the examples repo, navigate to the program directory, and initialize a new stack:

<span class="line"><span class="cl">git clone https://github.com/pulumi/examples.git
</span></span><span class="line"><span class="cl"><span class="nb">cd</span> examples/kubernetes-py-self-host-gemma4-llm
</span></span><span class="line"><span class="cl">pulumi stack init gemma4-local
</span></span>

Configure the program to match your local setup:

<span class="line"><span class="cl">pulumi config <span class="nb">set</span> hostLlmPort <span class="m">18080</span>
</span></span><span class="line"><span class="cl">pulumi config <span class="nb">set</span> llmBaseUrl http://llm-server:18080/v1
</span></span>

The Kubernetes service named llm-server maps to host.k3d.internal. In our validation, we confirmed that a disposable k3d pod could reach the Mac’s llama.cpp API at http://llm-server:18080/v1/models after a CoreDNS restart.

<span class="line"><span class="cl">kubectl rollout restart deployment coredns -n kube-system
</span></span>

Run pulumi up to deploy Open WebUI and connect it to host-native llama.cpp:

<span class="line"><span class="cl">pulumi up
</span></span>

In our validation environment, this command successfully reached the resource synthesis phase without requiring Tailscale credentials because Tailscale exposure is opt-in.

Access Open WebUI through Tailscale

Tailscale allows you to access your private Open WebUI instance from any device on your tailnet. Note that we only expose the web interface, not the raw LLM API, to keep the system secure.

The base Open WebUI deployment works without Tailscale credentials. To expose the web UI on your tailnet, enable Tailscale resources and provide an explicit api_key or OAuth/identity token:

<span class="line"><span class="cl">pulumi config <span class="nb">set</span> enableTailscale <span class="nb">true</span>
</span></span><span class="line"><span class="cl">pulumi config <span class="nb">set</span> tailscale:apiKey YOUR_API_KEY --secret
</span></span>

Once configured, Pulumi will create a Tailscale device or proxy that routes traffic to your Open WebUI service.

Use the model with Pi

Open WebUI gives you a browser-based chat interface, but local models are also useful from coding agents. Pi is the local coding agent used for this validation; if you do not use Pi, treat this section as an example of how any OpenAI-compatible client can point at the same local endpoint. Pi can point at the same OpenAI-compatible llama.cpp endpoint and use the model running on your Mac.

For a fresh Pi config, create ~/.pi/agent/models.json with a local provider that points at the llama.cpp API:

<span class="line"><span class="cl"><span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="nt">"providers"</span><span class="p">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="nt">"local-llama"</span><span class="p">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="nt">"baseUrl"</span><span class="p">:</span> <span class="s2">"http://127.0.0.1:18080/v1"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nt">"api"</span><span class="p">:</span> <span class="s2">"openai-completions"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nt">"apiKey"</span><span class="p">:</span> <span class="s2">"local"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nt">"compat"</span><span class="p">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="nt">"supportsDeveloperRole"</span><span class="p">:</span> <span class="kc">false</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nt">"supportsReasoningEffort"</span><span class="p">:</span> <span class="kc">false</span>
</span></span><span class="line"><span class="cl"> <span class="p">},</span>
</span></span><span class="line"><span class="cl"> <span class="nt">"models"</span><span class="p">:</span> <span class="p">[</span>
</span></span><span class="line"><span class="cl"> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="nt">"id"</span><span class="p">:</span> <span class="s2">"unsloth/gemma-4-12b-it-GGUF"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nt">"name"</span><span class="p">:</span> <span class="s2">"Gemma 4 12B Q8 (local llama.cpp)"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nt">"reasoning"</span><span class="p">:</span> <span class="kc">false</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nt">"input"</span><span class="p">:</span> <span class="p">[</span><span class="s2">"text"</span><span class="p">],</span>
</span></span><span class="line"><span class="cl"> <span class="nt">"contextWindow"</span><span class="p">:</span> <span class="mi">131072</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nt">"maxTokens"</span><span class="p">:</span> <span class="mi">1024</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nt">"cost"</span><span class="p">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="nt">"input"</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nt">"output"</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nt">"cacheRead"</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nt">"cacheWrite"</span><span class="p">:</span> <span class="mi">0</span>
</span></span><span class="line"><span class="cl"> <span class="p">}</span>
</span></span><span class="line"><span class="cl"> <span class="p">}</span>
</span></span><span class="line"><span class="cl"> <span class="p">]</span>
</span></span><span class="line"><span class="cl"> <span class="p">}</span>
</span></span><span class="line"><span class="cl"> <span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span>

Then set Pi to use that provider and model by default in ~/.pi/agent/settings.json:

<span class="line"><span class="cl"><span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="nt">"defaultProvider"</span><span class="p">:</span> <span class="s2">"local-llama"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nt">"defaultModel"</span><span class="p">:</span> <span class="s2">"unsloth/gemma-4-12b-it-GGUF"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nt">"defaultThinkingLevel"</span><span class="p">:</span> <span class="s2">"off"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nt">"hideThinkingBlock"</span><span class="p">:</span> <span class="kc">true</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span>

If you already have Pi configuration files, merge the local-llama provider and defaults into your existing JSON instead of replacing the files.

Advanced: Linux GPU in-cluster serving

If you’re running on a Linux host with an NVIDIA or AMD GPU, you can run the LLM directly inside the Kubernetes cluster. This requires the NVIDIA or ROCm device plugins.

The Pulumi program supports this through runtimeMode=cluster. In this mode, it deploys a LlmServer pod that manages the llama.cpp process within the cluster, using GPU resource requests to ensure hardware acceleration.

Cleanup

When you’re done, you can tear down the resources:

<span class="line"><span class="cl">pulumi destroy
</span></span><span class="line"><span class="cl">k3d cluster delete pulumi-gemma4-blog-qa
</span></span><span class="line"><span class="cl"><span class="c1"># Stop llama.cpp using the PID from your terminal</span>
</span></span><span class="line"><span class="cl"><span class="nb">kill</span> <PID>
</span></span>

AWS reports in an AWS Architecture Blog case study that Deloitte’s move to a virtual cluster model on Amazon EKS resulted in 89% faster testing environment provisioning. By consolidating dozens of disparate clusters into a single host cluster with over 50 vCluster instances, the case study says Deloitte saved about 500 QA hours per year. This “Environment Factory” pattern allows platform teams to provide isolated, ephemeral Kubernetes environments on demand without the cost or lag of full cluster provisioning.

This post adapts that general architecture with Pulumi to orchestrate Amazon EKS Auto Mode and vCluster.

The problem: environment sprawl and provisioning lag

Traditional development workflows often rely on one full EKS cluster per developer or feature branch. While this provides strong isolation, it introduces major pain points. Provisioning a full cluster can take 15 minutes or more, which slows down CI/CD pipelines. Managing dozens of clusters also leads to high costs and significant operational overhead.

Platform teams need a “soft multi-tenancy” model. This model should feel like a dedicated cluster to the developer but run on shared infrastructure to keep costs low and startup times fast.

Architecture overview: the host and the tenants

The environment factory architecture consists of two main layers.

1. Host cluster: A single, reliable EKS cluster managed with EKS Auto Mode. This cluster provides the underlying compute, networking, and storage.
2. Tenant environments: Virtual clusters (vCluster) running as pods within host namespaces.

According to the vCluster architecture, the virtual control plane handles API requests while a syncer maps virtual resources to the host cluster. This separation allows tenants to manage their own CRDs, namespaces, and RBAC while platform teams use quotas, NetworkPolicies, pod security, IAM boundaries, and node isolation controls to protect the host and other tenants.

Implementation: the EKS Auto Mode host

EKS Auto Mode simplifies the host cluster by automating infrastructure management. It handles node provisioning, scaling, and updates based on pod requirements.

The following snippet shows how to define an EKS cluster with Auto Mode enabled using Pulumi.

<span class="line"><span class="cl"><span class="kr">import</span> <span class="o">*</span> <span class="kr">as</span> <span class="nx">awsx</span> <span class="kr">from</span> <span class="s2">"@pulumi/awsx"</span><span class="p">;</span>
</span></span><span class="line"><span class="cl"><span class="kr">import</span> <span class="o">*</span> <span class="kr">as</span> <span class="nx">eks</span> <span class="kr">from</span> <span class="s2">"@pulumi/eks"</span><span class="p">;</span>
</span></span><span class="line"><span class="cl"><span class="kr">import</span> <span class="o">*</span> <span class="kr">as</span> <span class="nx">k8s</span> <span class="kr">from</span> <span class="s2">"@pulumi/kubernetes"</span><span class="p">;</span>
</span></span><span class="line"><span class="cl"><span class="kr">import</span> <span class="p">{</span> <span class="nx">SubnetType</span> <span class="p">}</span> <span class="kr">from</span> <span class="s2">"@pulumi/awsx/ec2"</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">clusterName</span> <span class="o">=</span> <span class="s2">"environment-factory"</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">vpc</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">awsx</span><span class="p">.</span><span class="nx">ec2</span><span class="p">.</span><span class="nx">Vpc</span><span class="p">(</span><span class="s2">"environment-factory"</span><span class="p">,</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="nx">enableDnsHostnames</span>: <span class="kt">true</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nx">cidrBlock</span><span class="o">:</span> <span class="s2">"10.0.0.0/16"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nx">subnetSpecs</span><span class="o">:</span> <span class="p">[</span>
</span></span><span class="line"><span class="cl"> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="kr">type</span><span class="o">:</span> <span class="nx">SubnetType</span><span class="p">.</span><span class="nx">Public</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nx">tags</span><span class="o">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="p">[</span><span class="sb">`kubernetes.io/cluster/</span><span class="si">${</span><span class="nx">clusterName</span><span class="si">}</span><span class="sb">`</span><span class="p">]</span><span class="o">:</span> <span class="s2">"shared"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="s2">"kubernetes.io/role/elb"</span><span class="o">:</span> <span class="s2">"1"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="p">},</span>
</span></span><span class="line"><span class="cl"> <span class="p">},</span>
</span></span><span class="line"><span class="cl"> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="kr">type</span><span class="o">:</span> <span class="nx">SubnetType</span><span class="p">.</span><span class="nx">Private</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nx">tags</span><span class="o">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="p">[</span><span class="sb">`kubernetes.io/cluster/</span><span class="si">${</span><span class="nx">clusterName</span><span class="si">}</span><span class="sb">`</span><span class="p">]</span><span class="o">:</span> <span class="s2">"shared"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="s2">"kubernetes.io/role/internal-elb"</span><span class="o">:</span> <span class="s2">"1"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="p">},</span>
</span></span><span class="line"><span class="cl"> <span class="p">},</span>
</span></span><span class="line"><span class="cl"> <span class="p">],</span>
</span></span><span class="line"><span class="cl"> <span class="nx">subnetStrategy</span><span class="o">:</span> <span class="s2">"Auto"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"><span class="p">});</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1">// Create an EKS cluster with Auto Mode enabled.
</span></span></span><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">hostCluster</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">eks</span><span class="p">.</span><span class="nx">Cluster</span><span class="p">(</span><span class="s2">"host-cluster"</span><span class="p">,</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="nx">name</span>: <span class="kt">clusterName</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nx">authenticationMode</span>: <span class="kt">eks.AuthenticationMode.Api</span><span class="p">,</span> <span class="c1">// Use API authentication mode for EKS access entries.
</span></span></span><span class="line"><span class="cl"> <span class="nx">vpcId</span>: <span class="kt">vpc.vpcId</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nx">publicSubnetIds</span>: <span class="kt">vpc.publicSubnetIds</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nx">privateSubnetIds</span>: <span class="kt">vpc.privateSubnetIds</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nx">autoMode</span><span class="o">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="nx">enabled</span>: <span class="kt">true</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="p">},</span>
</span></span><span class="line"><span class="cl"><span class="p">});</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">hostProvider</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">k8s</span><span class="p">.</span><span class="nx">Provider</span><span class="p">(</span><span class="s2">"host-provider"</span><span class="p">,</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="nx">kubeconfig</span>: <span class="kt">hostCluster.kubeconfig</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"><span class="p">});</span>
</span></span>

Implementation: the environment factory

Once the host cluster is ready, we can build the factory that stamps out tenant environments. Each tenant needs a dedicated namespace, resource quotas, and the vCluster itself.

Tenant guardrails

Before installing vCluster, we set up a namespace and resource quotas to ensure one tenant cannot consume all host resources.

<span class="line"><span class="cl"><span class="kr">import</span> <span class="o">*</span> <span class="kr">as</span> <span class="nx">k8s</span> <span class="kr">from</span> <span class="s2">"@pulumi/kubernetes"</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1">// Define a tenant namespace on the host cluster.
</span></span></span><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">tenantNamespace</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">k8s</span><span class="p">.</span><span class="nx">core</span><span class="p">.</span><span class="nx">v1</span><span class="p">.</span><span class="nx">Namespace</span><span class="p">(</span><span class="s2">"tenant-alpha"</span><span class="p">,</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="nx">metadata</span><span class="o">:</span> <span class="p">{</span> <span class="nx">name</span><span class="o">:</span> <span class="s2">"tenant-alpha"</span> <span class="p">},</span>
</span></span><span class="line"><span class="cl"><span class="p">},</span> <span class="p">{</span> <span class="nx">provider</span>: <span class="kt">hostProvider</span> <span class="p">});</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1">// Apply resource quotas to the tenant namespace.
</span></span></span><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">quota</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">k8s</span><span class="p">.</span><span class="nx">core</span><span class="p">.</span><span class="nx">v1</span><span class="p">.</span><span class="nx">ResourceQuota</span><span class="p">(</span><span class="s2">"tenant-quota"</span><span class="p">,</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="nx">metadata</span><span class="o">:</span> <span class="p">{</span> <span class="kr">namespace</span><span class="o">:</span> <span class="nx">tenantNamespace</span><span class="p">.</span><span class="nx">metadata</span><span class="p">.</span><span class="nx">name</span> <span class="p">},</span>
</span></span><span class="line"><span class="cl"> <span class="nx">spec</span><span class="o">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="nx">hard</span><span class="o">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="nx">pods</span><span class="o">:</span> <span class="s2">"20"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="s2">"requests.cpu"</span><span class="o">:</span> <span class="s2">"4"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="s2">"requests.memory"</span><span class="o">:</span> <span class="s2">"8Gi"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="s2">"limits.cpu"</span><span class="o">:</span> <span class="s2">"8"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="s2">"limits.memory"</span><span class="o">:</span> <span class="s2">"16Gi"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="p">},</span>
</span></span><span class="line"><span class="cl"> <span class="p">},</span>
</span></span><span class="line"><span class="cl"><span class="p">},</span> <span class="p">{</span> <span class="nx">provider</span>: <span class="kt">hostProvider</span> <span class="p">});</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1">// Define a Role for the tenant within their namespace.
</span></span></span><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">tenantRole</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">k8s</span><span class="p">.</span><span class="nx">rbac</span><span class="p">.</span><span class="nx">v1</span><span class="p">.</span><span class="nx">Role</span><span class="p">(</span><span class="s2">"tenant-role"</span><span class="p">,</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="nx">metadata</span><span class="o">:</span> <span class="p">{</span> <span class="kr">namespace</span><span class="o">:</span> <span class="nx">tenantNamespace</span><span class="p">.</span><span class="nx">metadata</span><span class="p">.</span><span class="nx">name</span> <span class="p">},</span>
</span></span><span class="line"><span class="cl"> <span class="nx">rules</span><span class="o">:</span> <span class="p">[{</span>
</span></span><span class="line"><span class="cl"> <span class="nx">apiGroups</span><span class="o">:</span> <span class="p">[</span><span class="s2">""</span><span class="p">],</span>
</span></span><span class="line"><span class="cl"> <span class="nx">resources</span><span class="o">:</span> <span class="p">[</span><span class="s2">"pods"</span><span class="p">,</span> <span class="s2">"services"</span><span class="p">,</span> <span class="s2">"configmaps"</span><span class="p">,</span> <span class="s2">"secrets"</span><span class="p">],</span>
</span></span><span class="line"><span class="cl"> <span class="nx">verbs</span><span class="o">:</span> <span class="p">[</span><span class="s2">"get"</span><span class="p">,</span> <span class="s2">"list"</span><span class="p">,</span> <span class="s2">"watch"</span><span class="p">,</span> <span class="s2">"create"</span><span class="p">,</span> <span class="s2">"update"</span><span class="p">,</span> <span class="s2">"patch"</span><span class="p">,</span> <span class="s2">"delete"</span><span class="p">],</span>
</span></span><span class="line"><span class="cl"> <span class="p">}],</span>
</span></span><span class="line"><span class="cl"><span class="p">},</span> <span class="p">{</span> <span class="nx">provider</span>: <span class="kt">hostProvider</span> <span class="p">});</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1">// Bind the Role to a tenant user or group.
</span></span></span><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">tenantRoleBinding</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">k8s</span><span class="p">.</span><span class="nx">rbac</span><span class="p">.</span><span class="nx">v1</span><span class="p">.</span><span class="nx">RoleBinding</span><span class="p">(</span><span class="s2">"tenant-role-binding"</span><span class="p">,</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="nx">metadata</span><span class="o">:</span> <span class="p">{</span> <span class="kr">namespace</span><span class="o">:</span> <span class="nx">tenantNamespace</span><span class="p">.</span><span class="nx">metadata</span><span class="p">.</span><span class="nx">name</span> <span class="p">},</span>
</span></span><span class="line"><span class="cl"> <span class="nx">subjects</span><span class="o">:</span> <span class="p">[{</span>
</span></span><span class="line"><span class="cl"> <span class="nx">kind</span><span class="o">:</span> <span class="s2">"User"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="c1">// Replace "tenant-user" with the IAM-mapped user or group for this tenant.
</span></span></span><span class="line"><span class="cl"> <span class="nx">name</span><span class="o">:</span> <span class="s2">"tenant-user"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nx">apiGroup</span><span class="o">:</span> <span class="s2">"rbac.authorization.k8s.io"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="p">}],</span>
</span></span><span class="line"><span class="cl"> <span class="nx">roleRef</span><span class="o">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="nx">kind</span><span class="o">:</span> <span class="s2">"Role"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nx">name</span>: <span class="kt">tenantRole.metadata.name</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nx">apiGroup</span><span class="o">:</span> <span class="s2">"rbac.authorization.k8s.io"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="p">},</span>
</span></span><span class="line"><span class="cl"><span class="p">},</span> <span class="p">{</span> <span class="nx">provider</span>: <span class="kt">hostProvider</span> <span class="p">});</span>
</span></span>

For production use, map these Kubernetes identities to IAM principals using EKS Access Entries, with the legacy aws-auth ConfigMap still appearing in older clusters.

Deploying vCluster with Helm

We use the kubernetes.helm.v3.Release resource to install vCluster. This resource provides controlled Helm lifecycle management for the vCluster release. The values block should be adjusted for each tenant profile to control resource synchronization and control plane behavior. Review the vCluster release notes when changing chart versions because values schema and generated secret names can change across releases.

<span class="line"><span class="cl"><span class="kr">import</span> <span class="o">*</span> <span class="kr">as</span> <span class="nx">k8s</span> <span class="kr">from</span> <span class="s2">"@pulumi/kubernetes"</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1">// Install vCluster using the Helm Release resource.
</span></span></span><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">vcluster</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">k8s</span><span class="p">.</span><span class="nx">helm</span><span class="p">.</span><span class="nx">v3</span><span class="p">.</span><span class="nx">Release</span><span class="p">(</span><span class="s2">"vcluster-alpha"</span><span class="p">,</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="nx">name</span><span class="o">:</span> <span class="s2">"vcluster-alpha"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nx">chart</span><span class="o">:</span> <span class="s2">"vcluster"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nx">version</span><span class="o">:</span> <span class="s2">"0.20.0"</span><span class="p">,</span> <span class="c1">// Tested with vCluster 0.20.x; review release notes before changing versions.
</span></span></span><span class="line"><span class="cl"> <span class="nx">repositoryOpts</span><span class="o">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="nx">repo</span><span class="o">:</span> <span class="s2">"https://charts.loft.sh"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="p">},</span>
</span></span><span class="line"><span class="cl"> <span class="kr">namespace</span><span class="o">:</span> <span class="nx">tenantNamespace</span><span class="p">.</span><span class="nx">metadata</span><span class="p">.</span><span class="nx">name</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nx">values</span><span class="o">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="c1">// Explicit sync configuration; adjust per tenant profile.
</span></span></span><span class="line"><span class="cl"> <span class="nx">sync</span><span class="o">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="nx">toHost</span><span class="o">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="nx">pods</span><span class="o">:</span> <span class="p">{</span> <span class="nx">enabled</span>: <span class="kt">true</span> <span class="p">},</span>
</span></span><span class="line"><span class="cl"> <span class="p">},</span>
</span></span><span class="line"><span class="cl"> <span class="p">},</span>
</span></span><span class="line"><span class="cl"> <span class="p">},</span>
</span></span><span class="line"><span class="cl"><span class="p">},</span> <span class="p">{</span> <span class="nx">provider</span>: <span class="kt">hostProvider</span> <span class="p">});</span>
</span></span>

Accessing the virtual cluster

The vCluster generates a kubeconfig that allows developers to interact with the virtual API server. We must treat this kubeconfig as a secret, and the endpoint in that kubeconfig must be reachable from the Pulumi runner before using it to create resources inside the virtual cluster.

<span class="line"><span class="cl"><span class="kr">import</span> <span class="o">*</span> <span class="kr">as</span> <span class="nx">pulumi</span> <span class="kr">from</span> <span class="s2">"@pulumi/pulumi"</span><span class="p">;</span>
</span></span><span class="line"><span class="cl"><span class="kr">import</span> <span class="o">*</span> <span class="kr">as</span> <span class="nx">k8s</span> <span class="kr">from</span> <span class="s2">"@pulumi/kubernetes"</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1">// Retrieve the vCluster kubeconfig from the generated secret.
</span></span></span><span class="line"><span class="cl"><span class="c1">// The vCluster-generated secret can lag behind Helm release readiness on first creation,
</span></span></span><span class="line"><span class="cl"><span class="c1">// so teams may choose an explicit readiness check or rerun after the virtual control plane initializes.
</span></span></span><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">vclusterKubeconfig</span> <span class="o">=</span> <span class="nx">k8s</span><span class="p">.</span><span class="nx">core</span><span class="p">.</span><span class="nx">v1</span><span class="p">.</span><span class="nx">Secret</span><span class="p">.</span><span class="kr">get</span><span class="p">(</span><span class="s2">"vcluster-secret"</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nx">pulumi</span><span class="p">.</span><span class="nx">interpolate</span><span class="sb">`</span><span class="si">${</span><span class="nx">tenantNamespace</span><span class="p">.</span><span class="nx">metadata</span><span class="p">.</span><span class="nx">name</span><span class="si">}</span><span class="sb">/vc-vcluster-alpha`</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="nx">provider</span>: <span class="kt">hostProvider</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"> <span class="nx">dependsOn</span><span class="o">:</span> <span class="p">[</span><span class="nx">vcluster</span><span class="p">],</span>
</span></span><span class="line"><span class="cl"> <span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="p">).</span><span class="nx">data</span><span class="p">.</span><span class="nx">apply</span><span class="p">(</span><span class="nx">data</span> <span class="o">=></span> <span class="nx">Buffer</span><span class="p">.</span><span class="kr">from</span><span class="p">(</span><span class="nx">data</span><span class="p">[</span><span class="s2">"config"</span><span class="p">],</span> <span class="s2">"base64"</span><span class="p">).</span><span class="nx">toString</span><span class="p">());</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1">// Export the kubeconfig as a secret.
</span></span></span><span class="line"><span class="cl"><span class="kr">export</span> <span class="kr">const</span> <span class="nx">tenantKubeconfig</span> <span class="o">=</span> <span class="nx">pulumi</span><span class="p">.</span><span class="nx">secret</span><span class="p">(</span><span class="nx">vclusterKubeconfig</span><span class="p">);</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1">// Create a provider for the virtual cluster using the secret kubeconfig.
</span></span></span><span class="line"><span class="cl"><span class="kr">const</span> <span class="nx">vclusterProvider</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">k8s</span><span class="p">.</span><span class="nx">Provider</span><span class="p">(</span><span class="s2">"vcluster-provider"</span><span class="p">,</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"> <span class="nx">kubeconfig</span>: <span class="kt">tenantKubeconfig</span><span class="p">,</span>
</span></span><span class="line"><span class="cl"><span class="p">});</span>
</span></span>

Operational caveats

• RBAC and permissions: vCluster generates default RBAC rules that work for most scenarios. However, if your host cluster is heavily locked down, you may need to provide additional permissions to the vCluster service account.
• Helm release previews: When using kubernetes.helm.v3.Release, Pulumi previews may not show every detail of the rendered Kubernetes resources. It primarily tracks the state of the Helm release itself.
• EKS Auto Mode node lifetime: EKS Auto Mode uses immutable AMIs and has a 21-day node lifetime. Kubernetes reschedules vCluster pods and tenant workloads when nodes are replaced, so configure replicas, PodDisruptionBudgets, requests, and persistent storage for disruption tolerance.

Conclusion: ephemeral environments at scale

By combining Pulumi with EKS Auto Mode and vCluster, you can build a scalable environment factory. This approach provides the isolation developers need while maintaining the speed and cost-efficiency required by platform teams.

The snippets provided here are adapted for illustration. In a production environment, you would likely wrap these resources into a Pulumi ComponentResource to provide a clean, reusable API for your internal developers. When a feature branch is merged, deleting the Pulumi stack removes the resources managed by that stack, but validate namespace finalizers, persistent volume reclaim policies, and external cloud artifacts as part of cleanup.

For more on managing EKS with Pulumi, see the EKS guide.