Join Machines as Nodes

Orchestration runs agents on your machines. A machine joins your hosted nfltr.xyz account once, as a node; the node starts Claude Code agents on demand when a hub spawns work there, and stops them when the work ends. To join them to a relay you run yourself instead, see Run your own relay (beta).

Join a machine

Run this on the machine, with NFLTR_API_KEY in the environment or a key saved with nfltr config add-api-key (never on the command line), and the Claude token in the environment (see Credentials):

$ nfltr node join --max-agents 4 --allow-all-tools \
    --workspace ~/src/app \
    --allow-repo 'github.com/your-org/*' \
    --machine-description "GPU box on the corp VPN; has the eval dataset in /data" \
    --machine-resources gpu.a100,vpn.corp,dataset.eval

Leave it running (a service manager, tmux or screen). --max-agents is required: the most agent processes this node runs at once, a number or cores.

Or join one over SSH

From your terminal, nfltr orch nodes add ssh starts a node on a remote host over ssh; it copies this nfltr binary when the remote OS and architecture match (or use --remote-bin). Node flags go after --.

$ nfltr orch nodes add ssh --max-agents 2 user@build-host -- --workspace /srv/app --allow-all-tools
$ nfltr orch nodes list
$ nfltr orch nodes stop

nodes list shows every node, its bound and its running agents. nodes stop stops the local node the zero-setup hub started on this machine; stop a joined node by ending its nfltr node join process. --ssh-command "ssh -p 2222" passes ssh options; --wait bounds how long to wait for the node to join.

nfltr node join flags

Pairing key: keep the relay out of the key exchange

End-to-end encryption alone stops a relay that only watches. The keys each end sends are not signed, so a relay that swapped them could read the traffic. A pairing key stops that: each end proves it holds the same key before any brief or result is sent, and the key is mixed into the session key. The relay never sees the key; your account's API key cannot serve this purpose, because the relay sees it on every request.

$ openssl rand -hex 32 > ~/.config/nfltr/e2ee.key && chmod 600 ~/.config/nfltr/e2ee.key
# copy the same file to every machine yourself (scp, a password manager), never through the relay
$ nfltr node join --max-agents 4 --e2ee-key-file ~/.config/nfltr/e2ee.key
$ nfltr orch --e2ee-key-file ~/.config/nfltr/e2ee.key "<goal>"

• Every hub, node and worker that talks to the others needs the same file. An end with a key refuses one without it, naming the flag; there is no silent fallback. An older nfltr fails the handshake.
• Use random bytes (at least 32), not a passphrase: a guessable key could be tested offline.
• It proves the peer is one of your machines, not which one. Anyone with the file can join as any of them.
• A key treats the relay as untrusted: workers holding it refuse work the relay itself sends (a task POSTed to the relay's task API, a label-selector fleet invocation). Agents a hub spawns (spawn_agent, nfltr orch "<goal>") are not affected.
• The agent process a node launches gets the file too (it talks to the hub); its model CLI does not, unless --agent-relay-access. The key keeps the relay out; it does not tell your machines apart.
• Rotating it means replacing the file everywhere and restarting the hub and nodes.

Tools are opt-in

Without --allow-all-tools, claude agents can edit files in their checkout but every command that needs approval is refused, and the turn completes as tools_not_allowed, naming the node and the flag. Codex agents run in codex's workspace-write sandbox; a task cannot widen it. Join with --allow-all-tools only on machines where you want agents to run commands. A node without it takes no repository tasks: repository work needs a shell, so --allow-repo needs --allow-all-tools.

What agents can reach

• No account key. The agent process connects to the relay with the node's key; the model CLI it runs (claude, codex) and that CLI's tools get no nfltr credential. An agent that should run its own hub (nfltr mcp --toolset hub) needs a node joined with --agent-relay-access. The agent still runs as the node's OS user, so files that user can read (a key saved with nfltr config add-api-key, other processes' environments) stay readable; run the node as a dedicated user for a hard boundary.
• Its checkout only. A task's working directory must be inside the node's --workspace or the agent's task directory, after symlinks; anything else is refused with workspace_provision_failed.
• A clean Claude config. Agents run as the node's OS user. On a node joined with --inherit-host-claude-config, an agent (and a claude it runs itself) uses that user's ~/.claude: its settings, hooks, plugins and sessions, and it can leave sessions there. Otherwise it gets a clean config (see Credentials and a clean Claude config).
• No repository settings. A repository cannot grant its agent tools or run hooks through .claude/settings*.json unless the node joined with --load-project-settings.
• Its own node. An agent runs as the node's OS user, so it can stop its own node (pkill -f 'nfltr node', or any pattern that matches the node's command line). The node keeps that command line short (its description and resources are not on it); tell agents to stop only the processes they started.
• Who drives the node. A node acts on launch, status and monitor requests only from a hub or tool of your account: the relay routes only within one account, a node refuses requests from the agents nodes launch and from the relay's own agent, and with a pairing key only machines holding it get through. Agents likewise take tasks only from a hub or the relay's dispatcher, never from another agent.

A shared machine

A node's agents run as the node's OS user, beside each other, the node, its monitors and your own processes. Each agent is told so in its system prompt, together with its own pid and process group: the processes it starts descend from that pid, and any other process is not its own. The node decides nothing for it. What you can add:

• A tool policy. --tool-policy-file denies tool calls you name, for every claude agent here, with the hook that enforces a spawn's tool_policy. For example, to keep agents from signalling processes by name:

  {"pre_tool_use":[{"tool":"Bash","input_field":"command","deny_regex":"(^|[;&|]\\s*)(pkill|killall)\\b","reason":"this machine is shared: signal only your own processes, by pid"}]}

  The pattern is yours to choose; nfltr ships none.
• A PID namespace (Linux). --agent-pid-namespace runs each agent under unshare --user --map-current-user --pid --fork --mount-proc --kill-child (util-linux 2.38 or later): its ps, pkill and kill reach only its own processes, and everything in the namespace ends with the agent. The agent keeps your user id, so files and sockets behave as before; sudo does not work inside. macOS has no PID namespaces; there, run the node as a dedicated user for a hard boundary.
• Long commands. Claude Code moves a Bash command that outlasts its timeout (120 s by default) to the background, where the rest of it keeps running and can change the checkout while the agent goes on. Set BASH_DEFAULT_TIMEOUT_MS and BASH_MAX_TIMEOUT_MS in the node's environment to give commands longer in the foreground, or CLAUDE_CODE_DISABLE_BACKGROUND_TASKS=1 to have a command past its timeout killed instead of moved. They reach every agent here unchanged; nfltr sets none. An agent's final answer ends its turn and stops what it still runs in the background; the hub sees those tasks in the completion's background-tasks-at-answer artifact, and in its long-commands artifact the commands that were moved to the background or ran at least the default timeout, with their wall time and exit status.
• Temporary files. Each agent's TMPDIR is its own directory, emptied when its process exits. /tmp itself stays shared by every agent here, and agents are told so.

Approvals

A node that can change an environment (a cluster, a deployment) can gate those changes on your approval, per turn. Make a key pair once, on your own machine, and give nodes the public half:

nfltr orch hub approval-key --out ~/.config/nfltr/approval.key   # prints the public key
nfltr node join --max-agents 2 --labels env=k8s-prod --tool-policy-file k8s-policy.json --approval-public-key nfltr-approval-pub-v1:...

In the policy, mark the rules a grant may lift:

{"pre_tool_use":[{"tool":"Bash","input_field":"command","deny_regex":"(^|[;&|]\s*)kubectl\s+(apply|delete|patch|rollout|scale)\b","reason":"prod writes need the owner's approval","unless_approved":true}]}

When the hub asks you to approve a fix, it calls request_approval, which gives the node, the prompt's SHA-256 and the command to run; nfltr orch "<goal>" prints that request, the prompt verbatim between marker lines, on its terminal. Save the prompt to a file and sign it; --expect-digest signs nothing unless the file is exactly that prompt:

nfltr orch hub approve --key-file ~/.config/nfltr/approval.key --node node-k8s --prompt-file fix.txt --ttl 15m --expect-digest <prompt_sha256>

Keep the key away from every agent: never join a node as the OS user that holds the approval key or can reach the Docker socket, on any machine. On your own laptop, run the node as another user or in a container, or keep the key on a separate device; an agent with --allow-all-tools can read whatever its user can.

Give the printed grant back to the hub, which passes it as constraints.approval with that prompt. The node checks your signature, its own id, the prompt and the expiry; only then does that one turn run without the marked rules, and it takes no steers. Without a valid grant the call is denied, and the denial says why. The hub never holds your key, and an agent on the node cannot make a grant.

Repositories

A spawn can name a repository and ref. It goes only to a node whose --allow-repo globs admit it (matched on the normalized URL, so git@, ssh:// and https:// spellings of one remote match); nodes advertise the globs as nfltr.workspace_repo_allow. There the node provides, and the agent does the repository work itself (clone, branch, commit, push, choosing how):

• An empty task directory of the agent's own (under the node's state directory), kept across its turns, a stopped one included, and kept by the node after the agent's process exits for its next turn on this node, until the hub archives the agent or the node restarts. Anything not pushed then goes with it. Its path is not unique to the agent; see Builds across agents.
• A bare mirror of the repository on the node (git clone --mirror into the node's state directory), refreshed when a spawn names it, with the origin's branches and tags (objects are never garbage-collected). It refuses pushes: work reaches the origin only when pushed to the origin URL. For an https repository, the agent's own git clone and git fetch of the origin URL are served from the mirror: each one first brings the mirror up to date from the origin, so it sees every branch pushed there, and only what is new crosses the network; if that update fails, git goes to the origin itself. Pushes go to the origin. A shallow mirror gets the full history as soon as its origin has it. Only https, ssh and file transports; no redirects, no submodules.
• Git credentials for the allow-list only. The node gives its agents a git credential helper that answers only for https URLs a glob admits, by asking this machine's own git credential helpers.
• Your git identity, if you set one. When it joins, the node asks git which identity its user commits with (GIT_AUTHOR_*/GIT_COMMITTER_* in its environment, else user.name and user.email in its git config), with git's guessing turned off. If there is one, every agent it launches gets it as GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, GIT_COMMITTER_NAME and GIT_COMMITTER_EMAIL, and its facts name it. If there is none, the node sets none and the facts say so; it never makes one up. To have agents commit as someone, configure that identity for the node's user (for example git config --global user.name and user.email) and restart the node.
• The facts, in the agent's system prompt and as NFLTR_TASK_DIR, NFLTR_REPO_ORIGIN and NFLTR_REPO_MIRROR: the directory, the origin URL and ref, and the mirror (a local path, when it was fetched for this task, and whether it is shallow). The agent reports its delivery as a branch and commit (branch_ref, sha). The node runs no build, test or hook on a delivery; the agent and the hub choose any checks.

A spawn naming a repository no glob admits is not launched on the node: it waits, its waiting naming each node's allow-list, and ends at its timeout_ms (else after about 30 seconds) with the same reason. It never gets a mirror or a credential. The credential scope has limits: ssh keys and agents, and tokens in the environment (such as GH_TOKEN), are not scoped, and an agent runs as the node's OS user, so it can read that user's credential files. An agent with a shell can also fetch from any host the machine reaches; the allow-list bounds what the node itself fetches and what its credentials answer for, not the agent's network.

Describe the machine

The hub routes by what you tell it. Put what is only on this machine in --machine-description and --machine-resources: "has the production logs", "on the staging VPN", "runs the exporter service". A goal can then say "on the machine with the logs" without naming it.

What a hub can read about this machine

A hub of your account can read this node's live resources with inspect_node: CPU cores and load, memory, free disk where the node works, GPUs (model, memory, and utilization where the driver reports it) and its free agent slots. With --allow-inspect-files it can also list and find files by name under the workspace, the repository mirrors, the agents' task directories and each --data-dir, so it can send work to the machine that has the data. It never reads file contents, never follows a link out of those directories, and every answer is bounded in entries, depth and time. Only hubs of your own account can ask.

list_nodes carries each node's live load too, read by the node at most every 5 seconds: CPU cores, load average, available memory and busy and free agent slots. In a container the load average and memory are the host's, shared by every container on it; on Linux the row adds the node's own cgroup CPU limit and use and memory limit and use.

Builds across agents

Agents on one node share its build caches (one OS user: Go's, ccache's, Bazel's, and so on). Many tools key a cache entry by the source path, so the node keeps paths stable: it runs each agent process on one of its slots (up to --max-agents), and the agent's task directories and TMPDIR live in that slot's directory. Each agent still starts with an empty directory, and the node empties it when the agent's process exits; the next agent on that slot gets the same paths, so its build reuses what an earlier agent's build left in the cache instead of adding a copy keyed by a new path. Agents are told this.

For Go, a build in a directory no earlier agent on the slot used, such as a clone in a new subdirectory, still compiles the repository's packages again, unless the build uses -trimpath. That flag is your choice for the node: set GOFLAGS=-trimpath in its environment and its agents inherit it. The caveat: -trimpath also removes file system paths from binaries, so runtime.Caller, stack traces and debug info name module-relative paths, and tests that locate their testdata through runtime.Caller break under it.

Idle maintenance

Caches still grow, and the node does not decide what to clean. On a long-lived node, give it your own maintenance command: --idle-hook '<command>' --idle-hook-interval <duration>. The node runs the command with this machine's shell, as its user, in the workspace, with its environment minus the account key, only while none of its agents runs and at most once per interval (first when it joins). A run is stopped after one interval. If a spawn needs an agent while the command runs, the node stops the command's processes first, then starts the agent, so the two never run at once. Each run's outcome, exit code and duration are logged, its output goes to idle-hook.log in the node's log directory, and the last run shows as idle_hook in list_nodes and nfltr orch nodes list --json (the command itself is not shown). A failing run is logged and the node carries on. Without the flags nothing runs. Examples you might choose, not defaults:

nfltr node join --max-agents 4 --idle-hook 'go clean -cache' --idle-hook-interval 24h
nfltr node join --max-agents 4 --idle-hook 'cargo cache -a' --idle-hook-interval 12h
nfltr node join --max-agents 4 --idle-hook 'docker system prune -f' --idle-hook-interval 6h

The node waits only for its own agents: your own commands, monitors and other nodes of the same user may use those caches while the command runs. A node that is never idle never runs it.

A running agent's activity (in list_agents) includes its own task list only when Claude Code offers it one: in claude -p, as agents run, version 2.1 does not unless CLAUDE_CODE_ENABLE_TODO_TOOLS=1 is in the node's environment. That too is your choice for the node.

How a hub schedules work across your nodes is the hub's decision, and you may state your own policy for it: --hub-guidance-file <path> (or NFLTR_HUB_GUIDANCE_FILE) on nfltr orch, nfltr orch hub or nfltr mcp --toolset hub carries the file's text verbatim to the hub model as your scheduling guidance (UTF-8, at most 4 KiB; refused at startup otherwise): in the spawn_agents tool's description, and for a session nfltr orch starts also in its system prompt. Omitted, nothing is added: nfltr ships no guidance text. An example you might write, not a default:

Split a task with independent parts across free slots.
Ask each agent to keep a task list with TaskCreate/TaskUpdate.
When slots free up while an agent still has open items, hand some to new agents.

The task-list line assumes CLAUDE_CODE_ENABLE_TODO_TOOLS=1 on the nodes.

Credentials and a clean Claude config

Agents run with a clean Claude config: none of the machine's user settings, hooks, CLAUDE.md, memory, plugins or MCP servers load, and neither do the task repository's .claude settings unless --load-project-settings. So a keychain-only claude login is not read; put CLAUDE_CODE_OAUTH_TOKEN (from claude setup-token) or ANTHROPIC_API_KEY in the node's environment. The node checks this before joining and refuses with clean Claude config not established otherwise. --inherit-host-claude-config opts into the machine's own setup.

What an agent can read: the Claude credential it runs with, and whatever the node's OS user can read. Not the node's account key: on Linux no nfltr process exposes it to other processes of that user, and the agent's own processes never hold it. Give the node its key in its environment rather than a saved config file, which the agent could read; on macOS an agent can read the environment of the node's processes, so run a node whose key must stay from its agents in a Linux container or as a separate user.

Session isolation

Agents cannot reach the machine's other Claude Code sessions (yours included): the node verifies this before any task and refuses to join if it cannot. --allow-cross-session-messaging opts out, only for machines where agents must talk to those sessions.

Agent lifetime and continuations

By default an agent process exits when its task ends; the node keeps the agent's task directory, with everything in it, uncommitted files included. Continuing a finished agent (send_message) then starts a new process on the same machine, which takes that directory back. The node drops a kept directory once the hub archives its agent (the hub keeps its newest 128 finished agents; an archived agent cannot be continued) and empties them all when it restarts; a continuation that finds none starts in an empty directory and says so, and only what the agent pushed carries over. --agent-linger 10m keeps the process, its session and its task directory around, so a continuation resumes there. A session cannot move between machines.

When a node dies

The node owns its agents; if it is killed, its running agents end as node_lost, which the hub receives as a completion. If it stops (Ctrl-C, SIGTERM), it stops its agents and each ends as node_lost at once. If it loses its connection and does not come back within 30 seconds, each agent it was running ends as node_lost about 30 seconds after the node left; a node back sooner loses nothing. Task directories the agents held are removed; branches they pushed stay. Run the same nfltr node join to bring the machine back: a spawn waiting for that machine then runs there. If the hub itself goes away instead, agents keep running and deliver their results when it reattaches.

$ nfltr orch nodes list

In a container

A node or hub starts child processes and reaps them as they exit. When the container's first process does not reap orphans (for example sleep infinity), exited ones stay as zombies; start the container with docker run --init (or tini as its entry point).

Related: Many agents across machines · Troubleshooting · Hub tools reference.