Run sessions on your own workers

What a customer-run worker is

A worker is the machine a session runs on. Every worker serves exactly one organization: it is enrolled into it, and it can only ever be given that organization's sessions.

With your own workers:

• your host runs the sessions, their containers, the workspace files, the session's egress proxy and its read-only package registry cache;
• checkpoints go to your own S3-compatible bucket, or stay on the host's disk;
• the host only makes outbound connections. Ciresk never connects to it: its health endpoint (127.0.0.1:9471) and its registry cache (127.0.0.1:9473) listen on loopback only.

Ciresk's control plane in eu1 still decides: sign-in, policy, the session queue, placement, leases, tunnel tickets, the session journal and the audit log.

What leaves your network and what Ciresk sees

The relay and the browser gateway are Ciresk's today. The bytes your developers' tunnels and previews carry (sync, logs, exec, the browser view, VS Code in the browser) pass through them, readable by the relay except ciresk dev ssh and ciresk dev code streams, which are SSH end to end; the browser gateway shows the diff and the terminal, so it sees them in clear. So the workspace and its checkpoints stay in your network; the traffic between a developer and the session does not yet.

Models are not available on your own workers: a session there gets no model access, whatever models.allow says.

Requirements

The host:

• Linux, arm64 or x86_64, with systemd. Ciresk prepares its own hosts on Ubuntu 24.04.
• The worker runs as root.
• podman, with the systemd cgroup manager (podman info --format '{{.Host.CgroupManager}}' prints systemd), so session containers survive a worker restart.
• Overlay filesystems and network namespaces.
• The data directory, /var/lib/ciresk-ci by default, on XFS mounted with prjquota.
• Two rows in both /etc/subuid and /etc/subgid: containers:2147483647:2147483648 for podman, and ciresk-sessions:1342177280:131072, the range the worker gives each session its own host user from. The worker refuses to start sessions when another row or account overlaps that range.
• keyutils, and fs.inotify.max_user_watches of at least 524288 so each session can run a dev server.

A host that lacks one of root, podman with the systemd cgroup manager, network namespaces, overlay or XFS project quotas registers but takes no sessions, and its log says so.

Network:

• Outbound HTTPS (443) to api.eu.ciresk.com and get.ciresk.com, directly or through your proxy. Nothing inbound.
• Your image registry, to pull the session image by digest.
• Your checkpoint bucket's endpoint, when you set one.

Enrol a worker

Enrolling, listing, draining and revoking workers needs the dev:workers.admin scope, which owners and admins hold. A person or an API key carrying the scope may do it; an agent never can.

1. Keep your sessions on your own workers

Set workers to customer in the dev policy before your first session:

ciresk dev policy show --json     # copy the "policy" object to policy.json, raise "version" by one
ciresk dev policy set policy.json

{
  "version": 2,
  "workers": "customer"
}

Changing the policy needs dev:policy.write (see admin.md). Switching to customer revokes nothing: a Ciresk host already enrolled for your organization keeps its sessions until it idles out or you revoke it (Revoke a worker).

2. Install the worker

On the host, install the worker binary and its session guest from the signed release. The installer verifies the release's signature and writes the systemd unit ciresk-ci-worker:

curl -fsSL https://get.ciresk.com/install/worker.sh | sudo sh

Leave out --enrol: that flag joins Ciresk CI, which this host does not use. The installer writes an empty /etc/ciresk-ci/egress.yaml, so nothing leaves the host until you add the rows in step 5.

3. Issue the enrolment secret

From any machine signed in as an admin, issue the host's enrolment secret. The name is lowercase letters, digits and hyphens, at most 63 characters, and must be the name in the host's worker.toml:

ciresk dev admin enrol-worker acme-dc1-01

It prints the dve_ secret once, and as its next line the command that stores it on the host. The console's Workers page does the same. Run that line on the worker's host and paste the secret at its silent prompt, so the secret reaches neither a shell history nor a command line:

sudo install -m 0600 /dev/null /etc/ciresk-ci/dev-enrolment && read -rs DVE && printf %s "$DVE" | sudo tee /etc/ciresk-ci/dev-enrolment >/dev/null && unset DVE

With --json, data.file names that file and data.store holds the same line.

Both reach only your organization's sessions. Enrolling a name again rotates its secret and keeps the same worker; a registration under a different name than the secret was issued for is refused.

4. Write worker.toml

name = "acme-dc1-01"
ci_enabled = false
release_url = "https://get.ciresk.com/worker"

[dev]
sessions = true
api_url = "https://api.eu.ciresk.com"
control_plane = "remote"
enrolment_secret = "file:/etc/ciresk-ci/dev-enrolment"
base_image = "registry.acme.internal/ciresk/dev-base@sha256:<digest>"
memory_bytes = 60129542144
cpus_milli = 14000
sessions_max = 4

The images are not public. Mirror ciresk/dev-base (and ciresk/dev-ide) into your own registry, or ask Ciresk to admit your AWS account to pull from its registry.

Checkpoints (optional) go to your bucket when the unit's environment names it, with the host's own credentials (an instance profile, or the AWS SDK's usual chain):

[Service]
Environment=CIRESK_CI_STORE_BACKEND=s3
Environment=CIRESK_CI_STORE_BUCKET=acme-dev-checkpoints
Environment=CIRESK_CI_STORE_REGION=eu-central-1

CIRESK_CI_STORE_ENDPOINT points at an S3-compatible store other than AWS. Without a bucket, checkpoints stay on the host, and a session moved to another host cannot resume from its checkpoint (checkpoint_unavailable). A bucket that is set but does not open stops the worker; it never falls back to the disk silently.

5. Allow what the host must reach

Add these rows to /etc/ciresk-ci/egress.yaml, and allow the same destinations on outbound 443 in your firewall:

- host: api.eu.ciresk.com
  purpose: the dev control plane (registration, lease, heartbeat, reports, session secrets) and the relay
  products: [ci]
- host: get.ciresk.com
  purpose: the signed release the session guest is verified against
  products: [ci-worker]

Behind a mandatory proxy, add one proxy row; the worker's HTTPS calls and the relay connection go through it, and a TLS-inspecting proxy needs its CA in the host's trust store:

- proxy: proxy.acme.internal
  ports: [3128]
  purpose: the only way out of acme's network

What a session itself may reach is your rows for product dev-session, narrowed by the template's egress rows and your organization's ceilings (see egress.md). With no dev-session rows, sessions reach nothing outside. No session reaches api.eu.ciresk.com: that row names ci, never dev-session. With the registry cache on, it reads through the public registries (index.crates.io, static.crates.io, registry.npmjs.org, proxy.golang.org, sum.golang.org, pypi.org, files.pythonhosted.org, repo1.maven.org), so allow them in your firewall where you need them.

The worker reads /etc/ciresk-ci/egress.yaml unless CIRESK_EGRESS_FILE in the unit's environment names another file.

6. Start it and check

sudo systemctl enable --now ciresk-ci-worker
journalctl -u ciresk-ci-worker -f

The log says dev-only host: no ci registration, then registered with the dev control plane. A refused egress row names the destination and the product. A host that cannot run sessions says why, and the unit fails, so systemd restarts it.

Then, from a developer's machine in your organization, run ciresk dev up in a repository with a template; ciresk dev status names the worker and its release.

How sessions are placed

A pending session goes to one of your organization's workers that is online, takes sessions, is not drained and has room for the session's size: memory within memory_bytes, CPUs within cpus_milli times the overcommit, disk above the free-space floor (workspace_min_free_bytes, 20 GiB by default), and fewer sessions than sessions_max. Among those, the worker that held the session before comes first, then one that already has the repository warm, then the one with the most free memory.

When none can take it, ciresk dev up says why:

Operate a worker

A drained worker stays drained across its own restarts until you undrain it. The console's Workers page lists every enrolled name with its state (online, offline, drained, revoked), last heartbeat and release, and enrols and revokes too.

Upgrades: nothing updates your host by itself. Run the installer again for the new release and restart the unit. Sessions survive a worker restart: their containers outlive it and the worker takes them back.

If the host loses contact with Ciresk, each session's egress stops 30 seconds after its last renewed heartbeat, and the session waits; its content stays on your host.

Revoke a worker

ciresk dev admin enrol-worker acme-dc1-01 --revoke

In one step:

• the worker's dve_ and dvw_ stop working, and it is marked offline;
• every live session it held moves to a new epoch, so the relay refuses every ticket, stream and worker connection of the old one at once;
• a session started from the repository (ciresk dev up --remote) with a checkpoint goes back to the queue and resumes from that checkpoint on another of your workers;
• any other session is marked lost (reason worker_revoked); its owner's next ciresk dev up starts it again on another worker and syncs the worktree again;
• a session that was already stopping ends;
• the audit log records dev.worker.revoke with the sessions it moved.

People's device grants stay, and their next tickets carry the new epoch. The worker shows as revoked until you enrol its name again, which lets it back in under the same worker.

Ciresk can also cut off every worker of your organization at once, for an emergency rotation of credentials; you then enrol each host again.

Not available yet

• The relay and the browser gateway in your network: today developer traffic to your sessions passes through Ciresk's in eu1.
• Models in sessions on your own workers.
• Ciresk CI tasks on the same host, worker pools with settings of their own, single-use enrolment secrets and tokens bound to the host's key.
• Automatic updates of your workers, and a factory that starts your hosts on demand.
• Listing workers from the CLI: use the console's Workers page or the API (GET /v1/dev/workers/enrolments, see /api/dev).
• A self-hosted control plane and air-gapped installs.