Stopped is the default state: what a scale-to-zero orchestrator changes about how you build

Every project on RapidNative can run as a real cloud dev environment: a mobile runner, a web runner, an API server, and a Supabase-compatible database, each on its own subdomain. Multiply that by every project ever created and you get a fleet of small workloads that are almost all idle, almost all of the time.

Nobody wants to pay for idle. Nobody wants to run Kubernetes for a fleet of dev servers either. That gap is what orchd fills. It's open source, self-hosted, and the part of our stack I own. I wrote a short overview of it on my projects page too. Here is how it works, mostly in pictures, and what building on it taught me.

The bet: wake latency is cheap enough to make idleness free

orchd is one Go daemon. It provisions per-tenant workloads, gives each a hostname, isolates it with Docker + gVisor, suspends it when idle, and wakes it on the next request. No cluster, no YAML. The docs state the bet plainly: if a suspended workload comes back in about a second, "stopped" is an acceptable default, and the cost of a tenant collapses from running memory to disk.

Property	Measured
Cold provision, including database init, under gVisor	~2.8 s
Wake from scale-to-zero	~0.9 s
Memory per running instance	~75-85 MB
Memory per idle instance	0, the container is stopped

That last row is the whole product.

The model: project → workload → route

A workload is the unit that gets scheduled, isolated and suspended. A route is a hostname pointing at one. A project's orchd.json declares its workloads, and a workload is just an image, a port and a volume.

orchd has no opinion about what it runs. RapidNative is an adopter, not a part of it: the daemon reads orchd.json and runs whatever each workload declares. The orchestrator stays generic, and every integration problem lives on the application side.

What a request does

The gateway resolves the request's host against an exact-match route table, wakes the workload if it is suspended, and proxies through. When traffic stops for long enough, the workload is suspended again. Control plane and gateway are the same daemon.

One interface, three substrates

Workloads run untrusted code, so containers alone are not enough. gVisor gives VM-grade isolation without nested virtualization, which is why orchd runs on an ordinary cloud VM. Firecracker, which needs /dev/kvm, is the planned faster tier. Swapping substrate is a driver change, not a rewrite.

What building on it taught me

The orchestrator is the clean part. These are the habits that scale-to-zero forced on the application side.

Don't wake things the user didn't ask for. Scale-to-zero only pays off if you also stop scaling up speculatively. Anything that provisions a workload "just in case" on a page open turns the cheapest state into the most common one you never use. Wake on an explicit action.

"Not up yet" is a state, not an error. Once wake is a normal event, every caller that touches a workload has to budget for it: keep the loading state up until the route actually answers, retry the lookup rather than the frame, and never treat the gateway's first response to a cold workload as final.

Lifecycle hygiene is a feature. Deleting a project has to release its whole stack, or idle workloads pile up forever. Hostname conflicts have to verify ownership before anything is reused. Each is a small fix. Each, missed, is a slow bleed of money or a security question.

Why I like this design

Most orchestration stories are about scheduling. This one is about defaults. orchd's default is off, its unit is a workload with a hostname, and its contract with the application is a JSON file. That contract is small enough for one engineer to own the integration end to end, and small enough that when something breaks, it is almost always on the side you can see.

The daemon is open source and standalone. If you have a fleet of small, mostly idle things that each need a URL and their own data, it's worth a look.