# daedal: the composition engine

**Lachlan Douglas**

An engineer declares a system as components and the relationships between them, and daedal resolves that declaration to one determinate form before anything runs. The same declaration gives the same form on any machine. A plan states what a change will do before anyone applies it. daedal hands the building and running to the substrates a team already uses.

## 1 What daedal is

daedal is a composition engine: a single statically linked Rust binary that runs in process, with no daemon. An engineer runs it from a solution directory. daedal compiles the declared composition to a normal form, plans the change against what it last applied and applies the change by handing each step to a substrate. daedal keeps a solution's state in the solution directory.

Terraform manages infrastructure state, Kubernetes manages cluster state and Restate runs durable workflows. Each is a control plane in its own domain. daedal works at the level of the solution: which components make it up, how they connect and what each expects of the others. daedal composes the solution's components and delegates each one to the tool that realises it. A composition can declare a database cluster and the release that uses the cluster. One apply realises both, in the order their relationships imply.

A **substrate** is anything that realises part of the resolved form: a cloud provider, a container runtime, a scheduler, a durable-execution engine, an inference agent or a person working through an interface. daedal addresses each substrate by the role it fills in the composition. Two substrates that fill the same role are interchangeable. An engineer swaps one for the other by changing a binding, and every consumer of the role keeps its declaration.

*No Feedback* gives the account of a made thing whose form is settled by its declared parts before it operates. *Free Assembly* gives the calculus of the composition operator that daedal's resolver implements. *Free Assembly* also proves that the resolved form is the same in any order of assembly. Both papers are at [/foundations/](/foundations/#reading).

## 2 The composition model

A **solution** is a tree of components under one root, each declared in a short manifest, `daedal.component.yaml`. Every component takes one of seven shapes:

| Shape | Role |
|---|---|
| `root` | the solution's scope root; owns its children and carries the top-level wiring and grants |
| `branch` | an interior container that owns a scope |
| `module` | a sub-root that daedal compiles as its own unit, connected to the enclosing composition only through a declared interface |
| `content` | a static leaf that emits data at resolve time and runs no code |
| `step` | a component whose code runs during an apply and hands any lasting work to a substrate |
| `scaffold` | a background helper opened before its first consumer and torn down after its last, even when an apply fails |
| `aspect` | a cross-cutting emitter whose offers fan out to every step that accepts them |

Components relate through typed relationships, which daedal resolves by reading the declarations:

| Relationship | Meaning |
|---|---|
| configuration | static key-values sent to a component; each key has one configurer, and a second configurer of the same key is a compile error |
| contribution | a structured fragment attributed to its emitter and collected by kind; daedal aggregates it at build time or pushes it to a substrate at run time |
| provision | a provider supplying a consumer; daedal resolves it directly when static and routes it from a step's output when dynamic |
| offer | an aspect's fragment, fanned out to every step that accepts it; an accepted offer with no binding is a compile error |
| register | the edge that anchors a step to the substrate that owns it |
| bind | a scope owner's choice of which component fills a role, for registrants and aspects |
| grant | the operator's permissions for a component (section 6) |

## 3 Resolution and change

daedal resolves a composition by assembling its declared parts and deriving their wiring across the whole composition at once. The result is a single normal-form graph. daedal wires a requirement to its provider when there is one, and reports an error when there are two.

**Planning.** daedal compares each step's freshly resolved configuration with its last applied baseline and classifies the change as create, no-op, update or replace. `daedal plan` also runs each step's own plan phase, so a step can report what it would change in the world.

**Immutable keys.** A step can mark configuration keys immutable. Changing one forces a replace.

**Guarded destruction.** When a replace would run a step's recorded obliterators, daedal refuses the apply until the operator authorises the replace with an obliterate flag. daedal also refuses an apply whose obliterate flag authorises nothing.

## 4 Execution

**Scheduling.** daedal runs the steps of a resolved graph over their dependency order, in parallel up to a bound that defaults to the machine's core count.

**Converge always.** daedal runs every step on every apply. Each step checks whether the world already matches its input. daedal tears down a component marked for removal.

**Failure.** By default daedal lets the running steps finish, then halts. In persevere mode it continues the independent branches and stops only what depends on the failure. After a partial apply, daedal records new state for the steps it processed and keeps the prior state of the rest. A later teardown can therefore remove everything that earlier applies created. The next apply runs every step again, and each step converges.

**Suspension.** A step can return a suspended result. daedal records the step as suspended, pauses that branch and ends the run cleanly. The next apply runs the step again.

**Locking.** Mutating operations on a solution's state take an on-disk lock, and daedal reclaims the lock when its holder has died.

## 5 Substrates

daedal separates two choices for each step: the language its code is written in and the place the step runs.

### The step protocol

A step exchanges JSON with daedal once, statelessly, over standard streams. daedal sends the assembled input and a phase, `plan` or `apply`. The step does its work and returns a structured result, streaming log lines and heartbeats as it goes. An operator can set an optional timeout on silence. daedal then stops a step that sends no log line or heartbeat within that time, and a long-running step that keeps reporting progress runs to completion.

### Language runners

A step's file extension selects its launcher. Shell steps run directly and speak the protocol themselves. Python and Go steps have SDKs. daedal builds a Go step to a static binary before running it. Elixir steps run through an escript runner. Any program that speaks the protocol can serve as a step.

### Executors

The operator chooses where each step runs:

- **host**, the default, runs the step on the operator's machine with the operator's privileges;
- **container** runs the step in podman with no network, a read-only root filesystem, its code mounted read-only and one control channel as its route to the host. Shell, Python and Go steps run under the container executor; Elixir steps run on the host.

## 6 Governance

The composition declares every permission a system holds, so the operator can read them all before anything runs.

### Grants

A step declares the external programs it requires. The operator grants each component its permissions, under four keys: `commands`, `secrets`, `paths` and `executor`. A path grant names an entry from the root's `import: paths:`, the one place an engineer declares a host directory. The operator confers grants at a scope. daedal refuses a grant made by an imported component, so an imported package holds only the authority its importer gives it. A module can grant its components at most the authority the module holds.

### The control channel

A step under the container executor reaches the host only through the control channel. The step asks daedal to run a program, and daedal runs that program only when the operator has granted it to that step. The container executor makes the grants a perimeter. A step under the host executor runs directly, as an ordinary process with the operator's privileges, under grants the composition declares and the run log records. daedal injects only the step's granted secrets and gates the commands the step sends through the control channel.

### Secrets

daedal resolves a secret in memory from the first of three sources that holds it: an operator mapping to an environment variable or file, the encrypted secrets file or an environment variable of the same name. daedal places the value in the environment of a granted program. A step under the container executor names a secret in a control-channel request, and daedal places the value in the granted program's environment, outside the step. A step under the host executor receives its granted secrets in its own environment.

### Pre-flight

Before every plan, daedal collects every external program the composition requires, checks that each is present and reports every missing one in a single message.

## 7 Certification

`daedal certify` issues a composition certificate as JSON: a digest of the normal form, which is independent of the host, a digest of the resolved form, with the host's variables and paths bound, the engine version and a content digest for every input. `daedal verify` recomputes the composition and checks it against a certificate. Verification passes when the normal form and every input digest match. On a failure, `daedal verify` names each input whose digest changed.

## 8 Packaging and distribution

**The package loop.** `pack` marks a component as a package in a bundle, `export` projects the package into the local bundle store and `import` places a package into a scope by name, with its whole subtree, references, child packages and code. On import daedal mints a new identity and derives the wiring in the new context. `update` refreshes imported components from their recorded source and keeps their identity.

**Deterministic artefacts.** `bundle` builds a bundle as a standard OCI image layout with sorted entries and zeroed timestamps and ownership, so the same content gives the same digest on any machine.

**Transports.** `download` fetches a bundle from a local file, a git repository or an OCI registry. The git and OCI transports come with the `transport` build feature, which a build enables explicitly. `install` downloads a bundle and imports a package in one step. For private registries daedal uses the operator's Docker credentials and credential helpers. Pushing a bundle is the operator's step, with the operator's own tooling.

## 9 State and teardown

**State.** A small embedded SQLite database records each applied step's path, status and apply sequence. daedal stores each step's recorded output, including the teardown layers the step registered, in that step's `output.json`, next to the database. The database and the recorded outputs together describe the system as last applied.

**History.** daedal writes an append-only log for every run, indexed by digest. `daedal logs` replays a run. `daedal compact` deletes runs outside the operator's retention setting.

**Teardown.** `daedal apply --down` runs the destroyers each step recorded, in reverse order. Obliterators perform irreversible cleanup such as deleting data, and daedal runs them only when the operator passes an obliterate flag. `daedal remove` marks a component for removal, and the next apply tears the component down. Until that apply, `daedal restore` clears the mark.

## 10 Commands

| Group | Commands |
|---|---|
| Compose and inspect | `compile`, `show`, `steps` |
| Plan and apply | `plan`, `apply` (with `--down` for teardown) |
| Query | `state`, `logs`, `compact` |
| Certify | `certify`, `verify` |
| Author | `init`, `copy`, `move`, `rename`, `remove`, `restore`, `secrets` |
| Package | `pack`, `export`, `import`, `update`, `bundle`, `download`, `install` |

A global `-C` flag runs any command against another directory.

## 11 Authoring tools

**daedal-studio** draws a solution as a live graph, redrawing as the manifests change, with a panel that shows any component's relationships. **daedal-plugin** is a Claude Code plugin that teaches an assistant to author daedal compositions: the shapes, the relationships and the conventions.

## 12 Neighbours

Much of daedal's design has prior art. Nix resolves a graph of immutable declarations before it builds. Architecture description languages compose systems from components and connectors. Workflow engines such as Temporal compose human and machine tasks. Separation logic supplies the algebraic structure Free Assembly builds on. daedal composes infrastructure, application code, durable workflow, human work and inference agents in one composition. daedal addresses each substrate by role and fixes every grant when the composition resolves.