Alabama Wire Protocol

1. Introduction

Alabama is the satellite-to-core wire. A satellite is any device or process that captures input for, or renders output from, the Aleph assistant: a microphone-and-speaker board, a phone, a desktop app, a chat bridge. The core is the aleph server: it routes audio, runs the conversation, and owns settings and identity. Everything a satellite needs to do — stream a microphone, play speech, show an image, expose a device action as a tool, emit a button press — happens over this one connection.

Aleph ships many satellites itself, but this wire is specified, not left an internal detail, so that anyone can build a satellite — first- or third-party —​that plugs into the same feature set.

The guiding principle is that a satellite sends raw input and renders raw output; it never makes a core decision. It does not resolve who is speaking, does not decide whether an utterance is a command, does not interpret an event. It streams bytes and declares what it can do. The intelligence lives in the core.

1.1. Design principles

Raw in, rendered out

A satellite reports observations (audio, text, events) and performs instructions (render, play, invoke). Interpretation is the core’s job.

The wire is the one compatibility surface

The rest of the system is rebuilt freely; the wire is held to additive, non-breaking discipline, because satellites are not in the server’s deploy unit — firmware does not ship in lockstep with the server.

Fail loud, recover automatically

Failures surface loudly in logs and on the wire; where recovery is possible it is automatic, never a user chore. Where it is not — a device too old for the framing floor — the protocol fails legibly and points at the fix.

1.2. Document conventions

Wire schema in this document is **generated from the canonical .proto source** under docs/protocol/proto/; the proto blocks below are included at build time, never hand-copied, so the schema and the prose cannot drift. Sequence and state diagrams are authored in Mermaid and rendered in the browser.

2. Conformance

MUST, MUST NOT, SHOULD, MAY and their kin carry their BCP 14 [RFC2119] [RFC8174] meaning where they appear in all capitals.

A satellite implements every Tier 0 and Tier 1 operation, plus each Tier 2 operation gated by a capability, tool, event, or setting it advertises. A core implements every operation it may exchange with an admitted satellite.

2.1. Tiers

Operations are partitioned into three tiers, by how universal they are and how they are gated (see § 13 Version gate and compatibility for how this stays compatible across versions).

Tier 0 — universal, mandatory

Hello, HelloResponse, Error, Ping, PingResponse, Paired. Without these a peer is not a satellite. wire_version anchors this universal baseline.

Tier 1 — stream substrate

StreamChunk, StreamEnd, StreamCancel. Generic and shared; every streaming capability draws on it.

Tier 2 — capability-gated

Every other operation, each behind the capability, tool, event, or setting that declares it. A satellite that does not advertise a capability never receives its operation.

Tier 0 and Tier 1 are an application-agnostic substrate — framing, transport, identity, streams, and liveness, which could carry any typed device protocol —​and Tier 2 is the aleph voice-assistant semantics layered on it (§ 14 Prior art and alternatives). Keeping the substrate and the voice layer in one protocol keeps the Envelope oneof flat and fully typed; the substrate could be lifted out if a second application needed it.

2.2. Extensibility

The wire is append-only, in every tier. A new operation is a new oneof arm (plus, in Tier 2, the capability value that gates it); a new option is a new field or enum value. A field number, once assigned, is never reused or repurposed — not even with the same type. Implementations MUST ignore fields they do not recognise and tolerate fields that are absent (protobuf’s native behaviour: an unknown field’s wire type carries its own length so a parser can skip it, [PROTOBUF]). Two peers at different versions therefore always interoperate on their common subset: a newer peer’s additions are ignored by an older one, and an older peer’s smaller surface is handled by the newer one. This holds for Tier 0 and Tier 1 as much as Tier 2 — the substrate only grows, it never breaks (see § 13 Version gate and compatibility).

Because the wire is append-only and every Tier 2 operation is gated, a capability can be specified before any device implements it — an unexercised arm costs nothing. Several here are specified but not yet built: DEFERRED store-and-forward (CaptureAck, capture_ts), observe-copies (TOOL_ACTIVITY), and Presence. Each ships when a concrete satellite needs it.

3. Architecture overview

3.1. Roles

The runtime is two kinds of process: satellites (many, heterogeneous, independently deployed) and the core (one server). Alabama is spoken only between satellites and the core; everything else the core does is internal to it and never on this wire.

3.2. The two identities

Alabama carries two distinct credentials:

• the device identity — which device — established once, at connect, as a property of the transport key (see § 7 Identity and admission);

• the speaker hint — who produced this input — carried per-input on the optional from field (see § 11 Speaker identity).

A device is not a speaker: a shared living-room board is one device with many speakers; a personal phone is one device bound to one user.

3.3. Connection lifecycle

sequenceDiagram
    autonumber
    participant Sat as Satellite
    participant Core
    Sat->>Core: transport connect
    rect rgb(238,238,238)
    note over Sat,Core: encrypted handshake — device key authenticated
    end
    note over Core: device_id = fingerprint(pubkey), admission lookup
    Sat->>Core: Hello {capabilities, tools, events, settings}
    Core->>Sat: HelloResponse {Accepted}
    note over Sat,Core: surface active — operations flow both ways
    Sat-->>Core: CaptureOpen + StreamChunk… (input)
    Core-->>Sat: MediaRenderOpen + StreamChunk… (output)

4. Protocol by example

Before the per-operation reference (§ 10 Operations), here is the protocol in motion. Each line below is one frame: its direction, the operation, and the Envelope it carries. +audio / +tts marks a raw payload tail — present only for streaming StreamChunks; discrete content such as [ image/png ] rides inline in the Envelope itself (a protobuf bytes field, no tail). Frame and id rules are in § 6 Wire encoding and framing — recall that a satellite mints odd ids and the core even ones.

4.1. A voice exchange, end to end

A living-room board wakes on its wakeword and streams microphone audio up; the core transcribes it, decides, speaks a reply, and closes the exchange with a terminal on the CaptureOpen id.

sequenceDiagram
    autonumber
    participant Sat as Satellite
    participant Core
    note over Sat,Core: wakeword fires
    Sat->>Core: CaptureOpen (id 1, trigger WAKEWORD)
    Sat->>Core: StreamChunk (seq 0) + audio
    Sat->>Core: StreamChunk (seq 1) + audio
    Sat->>Core: StreamChunk (seq 2) + audio
    note over Core: endpoint detector hears end of speech
    Core->>Sat: StreamEnd (END_DETECTED)
    note over Sat: mic stops capturing
    note over Core: transcribe, decide, synthesize
    Core->>Sat: MediaRenderOpen (id 2)
    Core->>Sat: StreamChunk (seq 0) + tts
    Core->>Sat: StreamChunk (seq 1) + tts
    Sat->>Core: PlaybackDone (id 2)
    Core->>Sat: Response (id 1) — exchange complete

On the wire, frame by frame:

Sat → Core   CaptureOpen      {id 1, AUDIO, trigger WAKEWORD, mode LIVE}
Sat → Core   StreamChunk      {id 1, seq 0}   +audio
Sat → Core   StreamChunk      {id 1, seq 1}   +audio
   ⋯
Core → Sat   StreamEnd        {id 1, END_DETECTED}              ← core's endpoint detector closes the mic
   (core transcribes → decides → synthesizes)
Core → Sat   MediaRenderOpen  {id 2, SPEAKER}
Core → Sat   StreamChunk      {id 2, seq 0}   +tts
   ⋯
Core → Sat   StreamEnd        {id 2, COMPLETE}
Sat → Core   PlaybackDone     {id 2, frames 9600}               ← render terminal
Core → Sat   Response         {id 1}                            ← terminal on the CaptureOpen id (empty)

What to notice:

• The satellite streams raw audio and never decides the input is over — the core’s endpoint detector does, and closes the mic with StreamEnd{END_DETECTED} ("raw in, core decides", § 1.1 Design principles). Push-to-talk is the exception: there the satellite sends StreamEnd{PTT_RELEASE}, since the button release is a local event it alone can see.

• id 1 correlates the whole exchange — CaptureOpen all the way to Response — and threads the mic stream within it. Every stream frame carries it: StreamChunk{id 1}, then StreamEnd{id 1} closes the mic while the id stays open until the Response. The stream ends; the exchange keeps going (§ 6.3 Identifiers, § 10.4.1 What the satellite observes).

• id 1 is odd because the satellite minted it; the core’s render uses even id 2. A stream’s frames carry their Open’s id, so parity tells you who minted the stream.

• Response carries no content — the reply already went out as TTS over the render stream — so it is a pure completion signal on the id.

• Live capture is fire-and-forward: the chunks are LOSSY_OK and get no per-chunk acknowledgement. The core’s only acknowledgement is acting on the stream — closing the mic, then rendering (§ 10.4 Input operations covers the CaptureAck that only a DEFERRED upload sees).

4.2. A tool call mid-exchange

If the core needs something only a device can do mid-exchange — read a screen, snap a photo, toggle a local control — it invokes a satellite-hosted tool and waits for the result before continuing, all under the same CaptureOpen id, which stays open:

   (… mic stream closed, as above …)
Core → Sat   Invoke       {id 4, name "screen.capture"}
Sat → Core   ToolResult   {id 4, content [ image/png ]}
   (core continues → MediaRenderOpen → … → Response {id 1})

Invoke is core-initiated, so its id 4 is even; the satellite replies with a ToolResult on the same id, and the exchange’s id 1 is untouched. The target need not be the satellite hosting the input — the core invokes a tool on whichever satellite hosts it, so a voice exchange on the living-room board can pull a screenshot from the user’s desktop.

4.3. Barge-in

The user talks over the assistant. The satellite opens a new capture, and the core cancels the TTS stream still in flight:

   (core mid-render: MediaRenderOpen id 2 — TTS playing)
Sat → Core   CaptureOpen    {id 3, AUDIO, trigger WAKEWORD}   ← user cuts in
Core → Sat   StreamCancel   {id 2, BARGE_IN}                  ← core drops the TTS render
Sat → Core   PlaybackDone   {id 2, frames 3200}               ← render ends early
   (the new exchange id 3 proceeds as normal)

StreamCancel with reason = BARGE_IN is how the core aborts its own render stream; the satellite stops playback and reports what it managed to play. The new exchange (id 3, odd — the satellite’s next) runs exactly like the first.

4.4. Tap-to-stop

Barge-in above replaces the turn with a new one. When the user instead just stops — a screen tap, a stop button — with no replacement command, the satellite sends Stop: a stateless turn-abort that carries no stream id and depends on no open stream.

   (core mid-turn: thinking — mic already closed, TTS not yet open)
Sat → Core   Stop   {no id}   ← user taps stop while it is thinking
   (core cancels the in-flight activation and lands the device idle)

Stop exists because a turn in the processing phase has nothing on the wire to cancel — the capture already ended and no render has opened — so a stream-scoped StreamCancel would reach nothing. Stop and StreamCancel are orthogonal: Stop ends the turn, StreamCancel discards one stream. Aborting a turn that still has an open capture or render sends both — Stop to cancel the activation, a StreamCancel on each open id to tear it down. Stopping a standalone media-sink (music) stream sends StreamCancel on its id alone, with no Stop: that is a transport stop, not a turn abort. StreamCancel never cancels an activation on its own; Stop is the only turn-abort signal.

4.5. An error along the way

When a request fails, its reply is a typed Error instead of the success terminal — on the same id. Here the screen.capture invoke fails because the display is asleep; the core adapts and still finishes the exchange:

   (… mid-exchange, the core invokes a tool …)
Core → Sat   Invoke           {id 4, name "screen.capture"}
Sat → Core   Error            {id 4, code UNAVAILABLE, message "display asleep"}   ← not a ToolResult
   (core adapts: replies "I can't see your screen right now")
Core → Sat   MediaRenderOpen  {id 2, SPEAKER}  → … → Response {id 1}

An Error carries {code, message, data} and terminates whatever request it answers — here the Invoke’s id 4. UNAVAILABLE means the capability is advertised but momentarily out of reach (display asleep, speaker muted): the operation is rejected on its own id, never a crash and never a silent drop, so the exchange carries on. The same shape can end a whole exchange — an Error on the CaptureOpen id (say INTERNAL_ERROR) aborts it, the failure counterpart to Response.

5. Transport

Alabama is a session layer: the framing, the operation catalogue, identity, and admission are defined independently of the substrate that carries them. Alabama does not care which transport carries it, only that the transport meets a small binding contract. A transport binding MUST:

• encrypt and authenticate every frame;

• authenticate the device’s static public key, because the device identity is that key (device_id = fingerprint(pubkey), see § 7 Identity and admission); and

• delimit the Alabama frame stream — a byte-stream transport carries the length-prefixed frames of § 6 Wire encoding and framing; a message-oriented transport carries one frame per message.

Each binding pins its own cryptography and its own message framing; the session layer above is identical on every one. Version 1 ships a single binding, Noise over TCP, and a future binding (QUIC [RFC9000], whose handshake is secured with TLS 1.3 [RFC9001]) slots in beside it as an additional listener with no change to anything above the transport.

5.1. Noise over TCP

The version 1 binding. [NOISE] The Noise static keypair is the device identity; the handshake proves possession of the key and encrypts the whole channel. Noise has a small internal footprint that coexists with the on-device audio front end on RAM-constrained hardware (ESP32), and it carries phones and desktops over the same binding. The handshake patterns — IK when the satellite already knows the server’s static key, XX when it does not — are in § 7.2 Noise handshake patterns.

5.1.1. Cipher suite

The suite is fixed:

**Noise_IK_25519_ChaChaPoly_SHA256** — when the satellite already knows the server’s static key (provisioned out of band).

**Noise_XX_25519_ChaChaPoly_SHA256** — when it does not, exchanging and pinning both keys on first use.

Both share the same primitives: Curve25519 [RFC7748] for the Diffie-Hellman, ChaCha20-Poly1305 [RFC8439] for the AEAD, and SHA-256 for the hash. These are the canonical 25519 / ChaChaPoly / SHA256 Noise tokens, and the suite name follows the Noise Noise_<pattern>_<DH>_<cipher>_<hash> form ([NOISE] §8). The pattern (IK or XX) is the satellite’s choice per § 7.2 Noise handshake patterns; the responder accepts either. A peer that offers any other suite is rejected at the handshake.

ChaCha20-Poly1305 is designed to run in constant time in pure software — it uses only add/rotate/XOR with no key-dependent table lookups ([RFC8439] §1) — with no dependence on an AES hardware path that a heterogeneous third-party device may not have, and SHA-256 is hardware-accelerated on the ESP32-S3 the firmware targets ([ESP32-S3-DATASHEET], SHA accelerator) and matches the ESPHome substrate Alabama draws on (§ 14.2 ESPHome native API).

5.1.2. Message framing

TCP is a byte stream with no message boundaries, and a Noise transport message is itself bounded — at most 65535 bytes ([NOISE] §3, the 16-bit length field), of which 16 are the Poly1305 tag, leaving 65519 plaintext bytes. So each Noise message — every handshake message and every transport message — is written to the TCP stream as a 2-byte big-endian length followed by exactly that many bytes of Noise message. The reader takes the length, reads the message, and decrypts it.

This framing is the Noise binding’s, distinct from the application frame’s own total_len (§ 6 Wire encoding and framing). The decrypted plaintext of successive transport messages concatenates into one continuous stream, and Alabama application frames are parsed from that stream by their total_len prefix — independent of where the Noise message boundaries fell. One application frame MAY span several Noise messages, and several small frames MAY share one. The two length layers never need to align: a StreamChunk larger than 65519 bytes is carried across as many Noise messages as it takes, while the negotiated max_frame_bytes (§ 6.1 Frame size limits) bounds the application frame as a whole.

6. Wire encoding and framing

Every frame is binary protobuf [PROTOBUF], length-prefixed.

┌───────────────┬────────────────┬─────────────────────────┬───────────────────┐
│   total_len   │   header_len   │   Envelope (protobuf)   │    raw payload    │
│     varint    │     varint     │     header_len bytes    │   optional tail   │
└───────────────┴────────────────┴─────────────────────────┴───────────────────┘

total_len

varint — the byte count of everything after it (the header_len varint, the Envelope header, and any payload). Bounded per § 6.1 Frame size limits.

header_len

varint — the byte count of the Envelope header that follows.

Envelope

the binary-protobuf frame header, carrying exactly one operation (see § 6.2 The Envelope).

raw payload

present only on a StreamChunk frame — zero-copy audio or video a device passes straight to its output without decoding the header.

The length prefix is intrinsic to a byte-stream transport such as the version 1 Noise-over-TCP substrate, where frame boundaries are not otherwise marked. A future message-framed transport (a QUIC DATAGRAM, say) is already self-delimiting and would omit the prefix.

6.1. Frame size limits

A length-prefixed frame is only safe if the reader can bound what the prefix claims before it allocates: an unbounded total_len is a denial-of-service against a device with kilobytes of RAM. Two limits bound it.

A fixed handshake ceiling bounds the frames that arrive before anything is negotiated: a Hello and its HelloResponse MUST each fit in a single Noise transport message — at most 65519 plaintext bytes, the 65535-byte Noise message limit less its 16-byte authentication tag (§ 5 Transport). They are pure control — a surface declaration and an outcome — so a single message suffices, and it keeps the pre-negotiation reader free of any reassembly state: it decrypts one message and holds the whole frame. A receiver rejects a larger handshake frame outright, since it cannot trust a length it has not yet been told to expect.

Past the handshake the frame size is negotiated, per direction. Each peer advertises the largest frame it is willing to receive: Hello.max_frame_bytes (the satellite’s receive limit) and Accepted.max_frame_bytes (the core’s). The sender MUST NOT emit a frame whose total_len exceeds the peer’s advertised limit — it chunks a StreamChunk tail, streams a large image over ImageRenderOpen (§ 10.5 Render operations) or large input media over UploadOpen (§ 10.4 Input operations) rather than inlining it, or shrinks an inline ContentBlock to stay under it. The limit governs the whole frame, header and tail together, so it bounds both a streamed media tail and a large inline block carried in the Envelope itself. A value of 0 means unset, and a protocol default of 65516 bytes applies — the largest frame that still rides a single Noise transport message, so a default-configured peer never reassembles a frame across Noise messages. A satellite with a tight RAM budget — an ESP32-S3 with the AEC and wakeword stack resident — advertises a small limit and the core paces under it; a capable device raises it.

A receiver sent an over-limit frame anyway MAY reject it with Error{code = FRAME_TOO_LARGE} on the frame’s id, or — when the header itself exceeds what it can decode — close the connection, since it can no longer locate the next frame boundary.

6.2. The Envelope

Exactly one operation rides each frame, in a flat oneof. The envelope id correlates a request with its reply.

6.3. Identifiers

The envelope id correlates a request with its terminal, and threads the frames of the stream that request opens (§ 10.3 Streaming operations).

It is 32-bit unsigned and allocated by the peer that originates the request. The low bit encodes the originator: a satellite allocates odd values, the core allocates even values — so the two never collide, and the parity of any id tells you who minted it. The satellite, as the connection initiator, takes the odd half, matching HTTP/2’s client-odd / server-even split (§ 14.4 gRPC and HTTP/2).

An Open (CaptureOpen, MediaRenderOpen, TextRenderOpen) mints a fresh id. Every frame of the stream it opens — StreamChunk, StreamEnd, StreamCancel, and the stream’s MediaControl — carries that id, and the terminal (Response, PlaybackDone, a ToolResult for an Invoke) answers on it. A StreamCancel is the exception that needs no separate terminal: it is one, retiring its id unilaterally on both sides (§ 10.3 Streaming operations). An id outlives its own stream: a mic stream ends at StreamEnd while the id stays open until the Response (§ 10.4.1 What the satellite observes). The stream is a phase of the id. Audio and video that belong together travel muxed in one container stream (§ 10.5 Render operations).

Within its parity class an allocator counts monotonically, stepping by two so the low bit is preserved, and wraps on overflow: ids are short-lived (an exchange, a stream), so a value is long retired before its number comes round again. The value 0 is the absent id — a frame with no id is a pure notification (a spontaneous SettingState, Presence, an observe copy) — so it is never allocated, and the core’s even sequence starts at 2.

7. Identity and admission

7.1. Device identity is the transport key

Every satellite generates a static keypair on first start. The public key is the device identity; the core derives an internal device_id = fingerprint(pubkey) that keys the registry, settings, and the tool namespace. The Hello carries no identity fields — the core reads the authenticated device key from the transport.

7.2. Noise handshake patterns

The satellite chooses its pattern by whether it already knows the server key; the core (responder) accepts both.

sequenceDiagram
    autonumber
    participant Sat as Satellite
    participant Core
    Sat->>Core: TCP connect
    alt server key known (provisioned via QR/BLE/ggwave)
        note over Sat,Core: Noise IK — anti-MITM from the first byte
    else cold device, address only
        note over Sat,Core: Noise XX — both keys exchanged, both TOFU
    end
    Sat-->>Core: static pubkey (authenticated in-handshake)
    Core-->>Sat: server pubkey + ephemerals → channel encrypted

7.3. Admission

The handshake (§ 7.2 Noise handshake patterns) authenticates the device’s static public key before the first Hello, and the core derives device_id = fingerprint(pubkey) and looks it up in its registry.

A device_id the core has not seen is pinned on first use and admitted as a guest: the core MUST reply Accepted and MUST enforce the guest surface server-side. A returning device whose authenticated key matches its pin is admitted at the trust the core already holds for it. The device is told nothing of its trust level — there is no trust field on the wire — so it runs identically as a guest or a fully paired device. Trust is elevated through one of three paths:

Token path (recommended for new devices)

The admin generates a short-lived pairing token on the server (e.g. by clicking "Add Device") and delivers it to the satellite out-of-band —​ggwave [GGWAVE] data-over-sound, a QR code, or BLE. The satellite includes the token in Hello.pairing_token. The core validates the token (checks it exists in the pending set and has not expired) and, if valid, MUST automatically send Paired (§ 10.2 Control operations) on the established session without any further admin action. Two proofs compose: the Noise handshake proves the device holds the matching private key; the token proves physical proximity at provisioning time (the device had to receive the out-of-band broadcast). Together they guarantee "this specific device was intentionally provisioned."

Manual in-band path

The satellite connects as a guest via Noise XX. An admin inspects the device in the management UI — identified by its key fingerprint and self-reported display name — and manually approves it. The core sends Paired on the live session.

Pre-provisioned path

Out-of-band provisioning (QR code, BLE, ggwave) delivers the server’s static public key to the satellite before its first connection. The satellite arrives already using Noise IK and is admitted at whatever trust the admin pre-configured. Paired is never sent on this path —​the satellite already has the key pinned.

Because the identity is the key, a different key is simply a different device_id, admitted as a new guest — the handshake proves possession of the matching private key, so impersonation is impossible by construction. A device that regenerates its keypair becomes a new identity and re-pairs from scratch; a satellite is cheap to re-enrol.

The one key mismatch that matters is the satellite’s: under the IK pattern it has pinned the server’s static key and MUST abort if the server presents a different one — this is what makes IK anti-MITM from the first byte.

A satellite that has not yet pinned a server key connects with XX (TOFU). It MUST NOT persist the server key on XX alone — the key is pinned only when the core sends a Paired message on the established session (§ 10.2 Control operations). Until then the satellite is a guest: it holds the server key for the lifetime of the current session only and uses XX again on reconnect. Once Paired is received the satellite MUST persist the server’s static public key and use IK for all subsequent connections. A satellite that has pinned a server key MUST NOT fall back to XX on IK failure — doing so would silently accept a different core and discard the pairing. If address discovery returns multiple candidates (e.g. via DNS-SD when a dev and a production instance share a network), the satellite SHOULD attempt IK against each candidate in turn and connect to the first one whose key matches the pin. A satellite MUST NOT replace a pinned server key without an explicit factory-reset or re-provisioning flow initiated by the user.

A server MAY operate in closed mode, in which it rejects Hello from any device_id it has not previously registered. In closed mode the core replies HelloResponse{rejected: Rejected{reason: REJECTED_CLOSED}} and closes the connection. The satellite MUST NOT retry automatically; the device requires user-initiated factory-reset or explicit provisioning to reach a server that will admit it. Closed mode is a server configuration knob, not a wire negotiation — a satellite has no way to query or change it.

HelloResponse carries Accepted, VersionMismatch, or Rejected (§ 13 Version gate and compatibility). In the normal path — open-mode server, compatible version —​admission never denies an authenticated peer a session; it only bounds what the session can do.

8. Surface declaration

A satellite declares its surface in the Hello across four orthogonal planes. The field tables for every message here are in Appendix A: Full wire schema.

8.1. Capabilities versus tools

The dividing line: a capability is something the core must understand in advance to take initiative — it is a closed, typed enum with exact parameters, and the core has hard-wired behaviour per value (it resamples to SpeakerParams.rate, gates a MIME against accepts). A tool is a discrete, on-demand action with an open JSON schema, called by name. Input that only the satellite initiates is not declared at all.

8.2. Tools, events, settings

The tools plane carries only name, description, and input_schema. The events plane is the inverse of a tool — a device-emitted signal, whose meaning the user configures in the management UI, never on the wire. The settings plane renders the device’s own settings generically.

Each plane carries normative weight:

• Capabilities. A CapabilityDescriptor’s params MUST be the variant that matches its kind (a CAPABILITY_SPEAKER carries SpeakerParams, and so on); TOOL_ACTIVITY is parameterless. Declared formats are exact, not maxima — the core resamples and transcodes every output precisely to the declared rate, channels, width, and accepts. A satellite MUST NOT be sent the operation a capability gates (§ 10.9 Capability mapping) unless it declared that capability; a core that receives an operation for an undeclared capability answers Error{code = CAPABILITY_MISSING}.

• Tools. A ToolDescriptor.name MUST be unique within the device’s Hello, and input_schema is a JSON Schema string. Qualified naming and collision resolution across backends are server-internal (§ 12 Tool federation); the wire name is the device-local one.

• Events. An EventDescriptor.id is a stable, device-defined string; the user-assigned meaning and display name live in the management UI and never travel on the wire. An Event’s metadata MUST validate against the descriptor’s metadata_schema when one is declared.

• Settings. The core MUST address a SettingsWrite only to a declared SettingDescriptor.key, and the device replies with a SettingState on the same id. A SettingState with no id is a spontaneous report the core observes (§ 10.7 Device operations).

9. Content blocks

The typed, discrete content unit carried in tool results and one-shot render pushes. Streaming media rides the frame payload instead. Field tables are in Appendix A: Full wire schema.

10. Operations

Every wire verb is a typed arm of the Envelope oneof. This section walks them grouped by function — each with its tier and, for Tier 2, the capability, tool, event, or setting that gates it; the field-level detail for every message is in Appendix A: Full wire schema. § 10.9 Capability mapping inverts the view, mapping each declared part of the surface to the operations it unlocks.

10.1. Delivery classes

Each operation has an intrinsic delivery class — a property of its type, not a wire field — with one mode-dependent exception for capture. LOSSY_OK covers the stream chunks of MediaRenderOpen and of a **LIVE** CaptureOpen: individually droppable media frames, where a missing frame is normal and not an error. RELIABLE covers everything else — every stream open, end, and cancel, every control, result, and lifecycle frame, *and the chunks of a DEFERRED capture*. A DEFERRED upload is store-and-forward precisely so a recording made while disconnected survives intact; dropping its frames would defeat the one capture mode whose entire purpose is durability, so its chunks are reliable even though a LIVE capture’s are not. Delivery class thus keys on the capture’s mode, not on the StreamChunk type alone.

On version 1’s single reliable Noise-over-TCP connection nothing is dropped in transit, and the transport offers no help here: TCP exposes only connection-global flow control — no per-stream backpressure, and no way to drop a frame — so left to the transport, a slow consumer would stall the whole multiplexed connection and delay RELIABLE frames behind buffered media. The core therefore reads LOSSY_OK as its licence to run its own application-level backpressure: rather than let media back up, it elides the stale frames (drop-oldest, pacing) the transport itself would never drop. So LOSSY_OK authorises the core to drop on purpose; under a future datagram transport (§ 5 Transport) it becomes literal transit loss, mapping to unreliable QUIC DATAGRAM frames ([RFC9221], which are not retransmitted on loss). A RELIABLE stream — a DEFERRED capture among them — rides a reliable QUIC stream under that transport, never a datagram, so a stored recording is never lost in transit.

A satellite declares its playout-buffer depth, buffer_ms, on a render capability (SpeakerParams, MediaSinkParams), and the core paces a live render within it. How the core schedules and bounds its own sending is server-internal and out of scope here; the wire contract is only that LOSSY_OK frames may be dropped, RELIABLE frames may not, and that the satellite advertises the buffer depth the core paces to.

The single connection multiplexes every stream and every control frame, so a large RELIABLE transfer — a chunked image, a DEFERRED upload — MUST NOT starve liveness. A sender MUST interleave Tier 0 frames (Ping, PingResponse) and other control between the chunks of a bulk transfer rather than emitting the whole transfer ahead of them, so head-of-line delay for a control frame is bounded by one max_frame_bytes frame, not by the size of the transfer; it SHOULD keep its socket writes frame-granular so the kernel send buffer cannot queue bulk bytes ahead of a waiting Ping. This is discipline, not a transport guarantee: a single TCP byte-stream offers no real priority, so a liveness timeout SHOULD allow for that one-frame queueing, and a clock-offset round taken under heavy load is approximate — the four-timestamp math cancels symmetric delay but not the asymmetric queueing a saturated link adds. A future QUIC binding (§ 5 Transport) removes the coupling outright by giving each stream its own flow-controlled lane; on version 1’s single TCP stream the interleaving discipline is what keeps liveness honest.

10.2. Control operations

Tier 0 — universal and mandatory, gated by nothing: without them a peer is not a satellite. Hello opens the connection and declares the device surface (§ 8 Surface declaration); the core replies with HelloResponse — Accepted (normal path), VersionMismatch (§ 13 Version gate and compatibility), or Rejected (server in closed mode, § 7.3 Admission). Ping/PingResponse carry liveness and a clock-offset round: four microsecond timestamps yield offset and round-trip time, mapping the satellite’s monotonic clock onto the server wall-clock, and nothing else; PingResponse answers on the Ping’s id. Error is a typed failure correlated by envelope id; a transiently unavailable capability (screen off, speaker muted) is rejected per-operation with code = UNAVAILABLE, and an Error on a CaptureOpen id means the input on that id was cancelled or failed. Paired is sent core→satellite on the token path (valid Hello.pairing_token) or the manual in-band path (admin approves via the management UI) — see § 7.3 Admission. On receipt the satellite pins the server’s static public key and switches to IK for all future connections. Paired carries no payload; its meaning is fully encoded by the direction and the authenticated channel it arrives on. Devices that arrived via Noise IK (pre-provisioned path) never receive Paired — they already have the server key pinned.

10.3. Streaming operations

Tier 1 — the generic stream substrate, shared and drawn in by any streaming capability rather than gated per-capability. Concurrent streams are multiplexed by the envelope id — each Open’s id is its stream (§ 6.3 Identifiers); a StreamChunk carries one frame, with the raw audio or video payload on the frame tail (§ 6 Wire encoding and framing). StreamEnd closes a stream gracefully (finalize and process); StreamCancel aborts it and discards on both sides, subsuming barge-in. A LIVE capture streams in real time; a DEFERRED capture is store-and-forward, its chunks carrying a capture_ts (microseconds) from the device’s monotonic clock that the core maps to wall-clock via a Ping offset round, and acknowledged by CaptureAck (§ 10.4 Input operations); it survives a reconnect without duplicated frames via the CaptureOpen correlation_ref (§ 10.4 Input operations). Per-stream encoding (PCM_S16LE — signed 16-bit little-endian linear PCM — or OPUS [RFC6716]) is self-describing on the open.

Concurrency is bounded per direction, mirroring max_frame_bytes. Each peer advertises the most streams the other may hold open against it at once —​Hello.max_concurrent_streams for the core’s render streams toward the satellite (MediaRenderOpen, TextRenderOpen, ImageRenderOpen), Accepted.max_concurrent_streams for the satellite’s captures toward the core — so a kilobytes-of-RAM device is never forced to hold more reassembly state than it declared, and an arbitrary flood of opens cannot exhaust it. A value of 0 means the protocol default of 8 applies. A stream counts against the limit from its Open until its StreamEnd or StreamCancel — the window in which chunks can still arrive and cost buffer state, not the longer wait for a terminal. Only chunked streams count: a one-shot Render, an Invoke, or a Ping is request-shaped, not a stream. Opening one past the peer’s limit draws Error{code = TOO_MANY_STREAMS} on the new id, and the opener waits for an open stream to end before minting another.

StreamCancel is authoritative and unilateral: it needs no acknowledgement and has no terminal — it is the terminal. The sender treats the named id as dead the instant it emits the cancel, the receiver treats it dead on receipt, and neither waits for the other — that is what keeps barge-in instant. Any frame that arrives on a cancelled (or never-opened) id — StreamChunk, StreamEnd, MediaControl — is silently discarded by the receiver; a peer that keeps emitting on a cancelled id is non-conformant, and the receiver MAY close the connection if it persists. There is no "refuse" on the wire: a peer cannot decline a StreamCancel. A device that physically cannot interrupt an in-progress effect degrades in quality but sends nothing back — the canceller has already moved on. Finally, StreamCancel ends one stream instance, not a streaming mode: an always-streaming device whose ambient CaptureOpen is cancelled MAY immediately open a fresh ambient CaptureOpen on a new id to resume. Cancel rotates the id; it does not revoke always-streaming.

10.4. Input operations

Tier 2. CaptureOpen, TextInput, and Event are satellite-initiated and not declared as capabilities — the core reacts to the frame that arrives. CaptureOpen opens an input stream and its envelope id is the correlation handle for the whole input: a present trigger makes it a discrete input the core processes and replies to, an absent trigger makes it ambient. A device MAY open a single ambient CaptureOpen at connect and stream continuously, letting the core run wakeword detection server-side rather than on the device. The id’s terminal is a Response or Error, and it outlives the audio: the microphone stream ends at StreamEnd while the core keeps processing (§ 6.3 Identifiers). TextInput is a discrete text input from a chat satellite, optionally carrying MediaBlock images alongside the text — a chat satellite forwarding a photo attaches it there rather than opening a capture. An image that fits the core’s max_frame_bytes rides inline in the block’s data; a larger one streams ahead over UploadOpen (below) and the block carries only the upload’s id in MediaBlock.upload. TextInput carries a from (the sender) and an opaque lane key, both owned by the satellite. from identifies the person; the core resolves it to a user. lane identifies the room and is set ONLY when the message is not a private 1:1 (a group chat): the core then keys conversation history on the lane rather than merging every chat on that connection into one thread, and treats the turn as shared context — no private per-speaker memory recall, since a shared room must not leak one person’s private facts. A private DM leaves lane empty and keys on from. The same from/lane pair rides CaptureOpen for voice turns. Event is a device-emitted signal whose meaning the user configures (§ 8 Surface declaration).

UploadOpen is the input-direction mirror of ImageRenderOpen (§ 10.5 Render operations): a satellite→core binary upload over the stream substrate (§ 10.3 Streaming operations), for any discrete op’s media that does not fit one frame — a forwarded photo on a TextInput, a camera still on a Response, a tool result’s media on a ToolResult. The satellite opens it with the payload’s mime and exact total_bytes, the encoded bytes ride the chunk tails, and the stream ends with StreamEnd{COMPLETE}; the discrete op that follows then references the completed upload by its envelope id via MediaBlock.upload instead of carrying inline data. The transport is ordered, so by the time the referencing op arrives the core holds the whole payload; the core resolves the reference back to inline bytes before anything downstream sees the block, and the upload is consumed by the first op that references it. UploadOpen is fire-and-forward like ImageRenderOpen — no positive terminal; an Error on its id (a declared size over the core’s budget, a stream that ends short of or runs past total_bytes) is the only answer and voids the id, and a later op referencing an unknown, incomplete, or voided upload draws Error{INVALID_REQUEST} on the referencing op’s id. An upload never outlives its connection, and it counts against Accepted.max_concurrent_streams from its open to its StreamEnd like any satellite-opened stream. As a bulk transfer it obeys the interleaving discipline (§ 10.1 Delivery classes): control frames go between its chunks, never behind the whole transfer. The core MUST NOT send UploadOpen or reference MediaBlock.upload itself — core→satellite media streams over ImageRenderOpen / MediaRenderOpen instead. RequestCapture is the one core-initiated input verb, gated by the MIC capability (or IMAGE_CAPTURE for a still-frame grab, below): the core sends it on an id R carrying a correlation_ref token (enrollment, a follow-up question, a cross-device trigger), and it is request-shaped — the device MUST answer, never leaving R unanswered. The answer is exactly one of two frames on that correlation. Accept is a CaptureOpen echoing correlation_ref equal to the request’s token — the input-side echo the LIVE path otherwise lacks, and for a DEFERRED upload the same field doubles as the resumable handle; that correlated CaptureOpen is R’s positive terminal, and the capture then streams and ends like any input (§ 10.4.1 What the satellite observes). Decline is an Error on R with code = UNAVAILABLE — the capability is advertised but transiently unavailable, i.e. the device is busy or has no capture capacity right now. Because the decline arrives as a frame rather than a timeout, the core learns of the refusal immediately and falls back on its own policy (reuse an already-running ambient stream, retry, or surface the failure) instead of waiting one out.

When RequestCapture sets image_source (gated by the IMAGE_CAPTURE capability, not MIC), it is a still-frame grab rather than an audio stream: the device captures one frame of the named source kind — webcam or screen, one the device listed in ImageCaptureParams.kinds — and answers R directly with a Response carrying the frame as a single MediaBlock content block: inline when it fits the core’s max_frame_bytes, streamed ahead over UploadOpen and referenced via MediaBlock.upload when it does not. This is a discrete request/response: no CaptureOpen, no capture stream. A device that cannot produce the frame declines with an Error on R. This is how the core’s image orchestrator pulls a picture from a device that advertised IMAGE_CAPTURE.

Concurrency for a requested capture is bounded by Accepted.max_concurrent_streams (§ 10.3 Streaming operations). A device with spare capacity opens the requested CaptureOpen concurrently with any capture it is already running (e.g. an ambient stream). A device at capacity — including a single-slot, always-streaming device — SHOULD supersede rather than decline: it ends or pauses the ambient capture, serves the requested one, and resumes the ambient capture on the requested capture’s close, because the core’s explicit request outranks the device’s default ambient mode. Only a device that genuinely cannot serve MUST decline with Error{UNAVAILABLE}.

CaptureAck is the cumulative acknowledgement for a DEFERRED upload — it belongs to the deferred-capture path, not the universal stream substrate, so a LIVE capture never sees one.

A DEFERRED upload survives a dropped connection and resumes without duplicating frames the core already stored. The envelope id cannot carry the resume — it is per-connection and resets on reconnect — so CaptureOpen carries an optional correlation_ref, a stable handle the device reuses across reconnects (the input-side counterpart to RequestCapture.correlation_ref). To resume, the device reopens the DEFERRED CaptureOpen with the same correlation_ref; the core maps it to the partial state it holds and re-issues its high-water-mark CaptureAck on the new id, so the device drops the frames already stored and continues from there — closing the duplication window even when the last pre-disconnect CaptureAck was lost. correlation_ref is meaningful only for DEFERRED: a LIVE capture is real-time and stale on reconnect, so it carries none and dies with its connection (a drop draws Error on its id, and the user re-triggers).

10.4.1. What the satellite observes

The satellite needs no notion of a "turn" — the core pushes its pipeline phase explicitly with PipelineState, a spontaneous notification (id = 0, § 10.10 Per-operation rules) sent per-connection, never broadcast. phase is one of IDLE, LISTENING, PROCESSING, SPEAKING, or ERROR (error_code set only on ERROR); the satellite renders it — LEDs, a face, a status pill — and no longer infers phase from the frame sequence. This covers turns a satellite cannot otherwise see coming, such as a captureless announcement, a text/chat exchange, or a render routed to a different sink, and keeps a multi-segment turn from flickering between segments.

PipelineState is advisory and UI-only, decoupled from mic control: a satellite MUST NOT drive its microphone from it. The mic lifecycle stays owned by RequestCapture, EndCapture (StreamEnd{END_DETECTED}), and StreamCancel (§ 10.4 Input operations).

The capture stream’s own terminal is a separate concern from the UI phase: a CaptureOpen always ends in exactly one Response (done) or Error (aborted) on its id, and the satellite returns to idle from that input from this terminal alone (§ 10.8 Results and terminals). How the core detects end of input, decides, and synthesizes stays server-internal and off the wire —​PipelineState narrates it for display, it does not expose it.

10.5. Render operations

Tier 2 — server-produced, each gated by the playback or display capability it targets. MediaRenderOpen opens a finite TTS render on a SPEAKER or a long-lived A/V stream on a MEDIA_SINK; TextRenderOpen opens a text stream into a TEXT_DISPLAY (transcript, assistant, or karaoke); ImageRenderOpen streams a still image into an IMAGE_DISPLAY; Render is a one-shot push to a display surface (IMAGE_DISPLAY, HTML_DISPLAY, or TEXT_DISPLAY); MediaControl steers a MEDIA_SINK’s playout buffer (pause, resume, flush) and sets its playout volume (set-volume, which doubles as the duck level while the assistant speaks over a running stream).

Every render open (MediaRenderOpen, TextRenderOpen, ImageRenderOpen, Render) carries an optional to (the person, the echo of the driving input’s from) and lane (the room, set for a group). A satellite multiplexing many conversations over one connection delivers the stream to coalesce(lane, to) and routes every later frame of the stream by its envelope id; a single-conversation device leaves both empty. This is what lets one bridge run many chats concurrently without a global in-flight lock.

An image reaches an IMAGE_DISPLAY by one of two paths, chosen by size. An image that fits the negotiated frame rides a single one-shot Render with the bytes inline. An image too large for one frame streams over ImageRenderOpen: the encoded bytes ride the chunk tails, the device sizes a buffer from the open’s total_bytes, reassembles, and paints at StreamEnd — the same stream substrate (§ 10.3 Streaming operations) audio and video already use, so the per-frame working set stays bounded by max_frame_bytes while the image itself may be far larger. This frees the image size from the frame size: a RAM-tight panel can advertise a small max_frame_bytes and still receive a large picture, holding only one chunk in transit plus the reassembly buffer. ImageRenderOpen is fire-and-forward like TextRenderOpen — no terminal, just a StreamEnd, and an Error on the id if the device cannot allocate the buffer or display the surface. The core MUST NOT leave a capable display blank for want of a path: it downscales or recompresses the image until it fits — into one frame for an inline Render, or under the device’s buffer for the stream — so a panel always receives a viewable image rather than nothing.

The HTML_DISPLAY surface renders sandboxed markup, and sandboxed is normative: the markup the core pushes is often assembled from model or tool output and is treated as untrusted, so a satellite advertising HTML_DISPLAY MUST render it in an isolated context with no ambient authority over the host — no access to the device filesystem, native or device-capability APIs, persistent storage, or the local network, and no navigation away from the pushed document. The content is presentational only. How a satellite isolates it — an OS webview’s sandbox, an iframe sandbox attribute ([HTML], which removes script, form, same-origin and top-navigation authority unless re-granted by token), a restricted renderer — is the satellite’s concern and not on the wire.

The isolation is a trust assumption, not a property the core can attest. The core pushes untrusted, often model-generated markup to a surface it cannot inspect: it knows only that the device advertised HTML_DISPLAY, not that the device’s renderer actually sandboxes. For a published wire whose whole point is third-party satellites, the sandbox is therefore required by this spec and honoured by convention — every implementer’s good faith — rather than enforced by the protocol. A conformant satellite MUST isolate as above; a core MUST treat the guarantee as contingent. To bind that unverifiable surface to the one thing it can verify — device identity — a core SHOULD push HTML_DISPLAY only to an elevated, paired device (§ 7.3 Admission), never to a guest, so untrusted markup never reaches an unattested renderer admitted on first contact alone. HtmlDisplayParams carries only the viewport the core authors against (§ 8 Surface declaration).

Now-playing metadata — the title, artist, and artwork of a running MEDIA_SINK stream — is not a media-plane concern and has no dedicated operation. It is display content: the core pushes it as a one-shot Render to a TEXT_DISPLAY or HTML_DISPLAY, the structured fields riding as JSON in the ContentBlock text and any artwork as an inline MediaBlock. Playout position is not streamed — the device is the one playing the audio, so it owns its own scrubber; the core sends the static metadata once when the stream opens and again as a cleared state when it stops. A Render carries no role, so a device that wants to distinguish a now-playing card from other display content relies on the surface it targets; a future revision MAY add an optional role to Render if a device needs to tell them apart on one surface.

10.6. Tool operations

Tier 2. Invoke calls a satellite-hosted tool by name — gated by the tools the device declares in its Hello — and the device replies with a ToolResult on the id. With observe = true the pair is an informational, server-redacted copy of the active exchange, delivered only to satellites that advertise the TOOL_ACTIVITY capability. Source, trust, namespacing, and routing are server-internal (§ 12 Tool federation).

10.7. Device operations

Tier 2. SettingsWrite tells a device to apply one of its declared settings and the device replies with a SettingState on the id; a SettingState sent without an id is a spontaneous report the core observes. Presence is optional multi-user ambient context — who is in the channel — reusing the from ref type (§ 11 Speaker identity); it is not an event and opens no input.

10.8. Results and terminals

Every request-shaped operation has exactly one terminal correlated on its own envelope id: a Response terminates a CaptureOpen, a PlaybackDone terminates a render stream (MediaRenderOpen), and a ToolResult (above) terminates a tool call (Invoke). RequestCapture is request-shaped too: its terminal on R is the accepting CaptureOpen that echoes its correlation_ref (§ 10.4 Input operations), and the failure counterpart is a decline Error{UNAVAILABLE} on R — the device MUST send one or the other, never leaving R unanswered. An Error (above) on that id is the failure terminal for any of them. A CaptureOpen therefore always ends in exactly one Response (done) or Error (aborted) on its id — including a voice exchange whose audio left over a separate MediaRenderOpen stream, where the Response carries no content and is a pure completion signal. A satellite returns to idle from this terminal alone; it never infers completion from a render stream.

Render, TextRenderOpen, and ImageRenderOpen are fire-and-forward and carry no terminal: a one-shot Render is delivered reliably, so the absence of an Error is the success signal, and TextRenderOpen and ImageRenderOpen each end at their StreamEnd. Each still fails loudly — an Error on its id reports a surface that could not display.

10.9. Capability mapping

What each declared part of the surface (§ 8 Surface declaration) unlocks. A satellite that does not advertise a row neither sends nor receives its operations; an older device that advertises fewer rows simply never sees the newer arms.

The stream substrate (§ 10.3 Streaming operations) and control operations (§ 10.2 Control operations) are not in this table — they are universal, not gated.

10.10. Per-operation rules

The rules every operation obeys, beyond the field tables (Appendix A: Full wire schema):

Correlation

A request-shaped operation gets exactly one terminal on its own id — a Response, PlaybackDone, or ToolResult, or an Error in its place (§ 10.8 Results and terminals). The originator mints the id at its parity (§ 6.3 Identifiers); the responder MUST answer on that same id and MUST NOT mint a new one for the reply. A frame with id = 0 is a notification and MUST NOT be answered.

Streams

A StreamChunk’s seq starts at 0 and increments by one. On a LOSSY_OK stream (§ 10.1 Delivery classes) the receiver MUST tolerate gaps; on a RELIABLE stream there are none. After a StreamEnd or StreamCancel on an id, the sender MUST NOT send further chunks on it; a chunk for an unknown or closed stream draws Error{code = UNKNOWN_STREAM} or STREAM_CLOSED.

Concurrency

A peer MUST NOT hold more concurrently open streams against the other than its advertised max_concurrent_streams (§ 10.3 Streaming operations, default 8); an Open past the limit draws Error{code = TOO_MANY_STREAMS} on the new id.

Capability gating

A peer MUST NOT send a Tier 2 operation the other side has not unlocked (§ 10.9 Capability mapping). A core that receives an operation for an undeclared capability answers Error{code = CAPABILITY_MISSING}; a capability that is declared but momentarily unusable (screen asleep, speaker muted) draws UNAVAILABLE on the operation’s id, never a connection drop.

Frame size

A sender MUST NOT emit a frame whose total_len exceeds the peer’s advertised max_frame_bytes (§ 6.1 Frame size limits), chunking a tail, streaming a large image over ImageRenderOpen, streaming large input media over UploadOpen, or shrinking an inline block to stay under. A receiver MAY reject an over-limit frame with Error{code = FRAME_TOO_LARGE}.

Liveness

Either peer MAY send a Ping at any time; the responder SHOULD answer promptly with a PingResponse on the same id, stamping t2 on receipt and t3 on send (§ 10.2 Control operations).

Forward compatibility

A peer MUST ignore envelope fields and message fields it does not recognise, and MUST tolerate an unrecognised oneof arm rather than drop the connection — the wire is append-only (§ 2.2 Extensibility), so an unknown operation from a newer peer is simply not acted on.

11. Speaker identity

Who produced an input travels per-input on one optional field, from: a single opaque string passed through verbatim by the satellite. The core matches it against its registry by exact string comparison; it never parses it. It rides on CaptureOpen, TextInput, and Event, and Presence reuses the same ref type. Resolution to a user — binding, voiceprint, channel credential —​is entirely server-internal and out of scope for this wire.

The hint carries no authority. A satellite — including a guest or a compromised peer — can place any string in from, so a core MUST NOT derive a speaker’s permissions from it: authority follows from the authenticated device key (§ 7.3 Admission), a voiceprint, or a channel credential, never from the hint. What a paired device represents sets how far this reaches — a device paired to one person carries that identity in its device key and from adds nothing, while a device paired to a shared space fronts many speakers behind one key and needs the separate signal above. The advisory Hello.multi_speaker flag only defaults the pairing UI toward one binding or the other; it informs that choice, never makes it, and a core MUST NOT derive authority from it either.

12. Tool federation

The wire carries only two things for tools: the ToolDescriptor in the Hello (§ 8 Surface declaration) and the Invoke/ToolResult pair (§ 10.6 Tool operations). Everything else — source, trust tier, qualified naming, collision resolution, routing across satellite, builtin, and MCP backends — is derived inside the server and never travels on the wire. Tool observation is gated by the TOOL_ACTIVITY capability: Invoke/ToolResult with observe = true are server-redacted copies of the active exchange, delivered only to satellites that advertise it.

13. Version gate and compatibility

Satellites and the core are updated independently — a phone app, an OTA firmware push, and the user’s own server each move on their own schedule. So version skew is the normal state, not a coordinated event: updating one side never breaks the other, and there is no flag day.

Compatibility is structural, not negotiated by comparing version numbers. The wire is append-only (§ 2.2 Extensibility), so any two versions interoperate on their common subset, and capabilities (§ 8 Surface declaration) negotiate which features a given device actually has. A newer satellite talking to an older core — or the reverse — simply uses what both understand.

Hello.wire_version is field 1, frozen forever: the one anchor every epoch can parse. It is a floor, not an exact gate — the core accepts any peer it can speak, which under the append-only rule is every normally-evolved one. It exists so that the single situation the wire cannot absorb — a peer below the baseline the core still supports — fails legibly: a typed VersionMismatch on HelloResponse (§ 10.2 Control operations) carrying the server version, the management UI flagging the device, the device signalling locally. In normal operation it never rejects.

Dead arms are not garbage-collected by rejecting old devices — that breaks end users — so they are left in place; the field-number space is vast and an unused arm costs nothing.

A genuinely breaking change — a framing or handshake flaw that cannot be fixed additively — should be vanishingly rare. When one is unavoidable, the core absorbs it: it speaks both the old and the new framing across a long deprecation window, and satellites migrate at the user’s pace. The compatibility burden sits on the core — the central, routinely-updated side — never on the user as a forced simultaneous update.

14. Prior art and alternatives

Alabama is deliberately a coupled protocol — an application-agnostic substrate (§ 2.1 Tiers) plus the aleph voice-assistant operation surface over it —​not a generic transport. It exists because the closest alternatives each settle one layer well, but not the whole.

14.1. Wyoming

Wyoming [WYOMING] is the direct inspiration, and the name is a nod to it — one US state for another. Aleph began on Wyoming and outgrew it: Wyoming is a deliberately simple voice protocol — a newline-delimited JSON header with an optional binary payload — and pushing aleph’s richer surface onto it ran into JSON parse overhead, the bandwidth of string keys on every audio frame, and mounting complexity. Length-prefixed binary protobuf removed all three. Alabama is Wyoming’s successor in spirit: voice-coupled like Wyoming, but typed, encrypted, with a device identity and a far richer operation surface.

14.2. ESPHome native API

ESPHome’s native API [ESPHOME-API] is the closest serious prior art at the substrate level, and it independently made the same low-level choices: length-prefixed protobuf over TCP, optional Noise encryption, and a Hello-handshake opening. That convergence is evidence the substrate is not gratuitous reinvention. It does not fit as-is because it is coupled to Home Assistant’s entity/service model and to ESPHome’s YAML-generated firmware, and its security is a single pre-shared key (Noise_NNpsk0) with no public-key device identity. Aleph’s satellites are heterogeneous — phone, desktop, chat bridge, microcontroller — and this wire is published so third parties can build their own into the same feature set, not only the satellites aleph ships. They need the capability, tool, event, and speaker-identity surface the entity API does not model, over a public-key identity with trust-on-first-use admission (§ 7 Identity and admission).

14.3. WebRTC and LiveKit

WebRTC [WEBRTC] is the real-time media stack — jitter buffering, packet-loss concealment, congestion control, NAT traversal, and mandatory encryption (the W3C API mandates DTLS-SRTP; the jitter/PLC/congestion machinery lives in the underlying IETF RTP layer) — and LiveKit [LIVEKIT] productises it with a self-hostable SFU and a voice-agent framework; its ESP32 SDK [LIVEKIT-ESP32] now reaches microcontrollers — both the ESP32-S3 this protocol targets and the more capable ESP32-P4 [ESP32-P4] (only hardware H.264 video is P4-exclusive; the S3 carries bidirectional Opus audio) — though it remains a heavyweight stack, and an explicit Developer Preview, on the S3. It is not adopted because it is a media and room transport, not a semantic protocol: aleph’s capability, tool, event, and operation surface would still have to be built on top of a data channel. A self-hosted SFU with TURN and signalling is also heavy infrastructure against the project’s "start one binary and it works" goal, and most of its value — traversal and loss resilience — would be spent on what is usually a LAN. Where it genuinely helps — robust real-time media on a capable device over a lossy link — plain WebRTC is a candidate media transport for a future binding, a sibling to the deferred QUIC option (§ 5 Transport), not a replacement for the protocol.

14.4. gRPC and HTTP/2

gRPC [GRPC] is the closest match to Alabama’s framing layer: bidirectional-streaming protobuf with multiplexing and flow control is, in the abstract, exactly what the substrate (§ 2.1 Tiers) provides, and the resemblance is not accidental. The envelope id parity convention (§ 6.3 Identifiers) is the HTTP/2 stream-identifier rule ([RFC9113] §5.1.1, which obsoletes [RFC7540]) on which gRPC is built [GRPC-HTTP2] — a peer-allocated stream identifier (31-bit unsigned in HTTP/2, the field’s high bit being reserved; Alabama widens it to the full 32 bits) whose low bit encodes the originator so the two sides never collide. The satellite, as the connection initiator, takes the odd values and the core the even — the same client-odd / server-even split as HTTP/2. The substrate is, deliberately, a re-derived slice of HTTP/2.

gRPC itself is not adopted, for three concrete reasons, each rooted in the ESP32-S3 target:

HTTP/2 weight

A full HTTP/2 stack — HPACK header compression [RFC7541], the SETTINGS/WINDOW_UPDATE machinery, the connection preface ([RFC9113] §3.4) — is heavy for a microcontroller already spending its internal RAM on the AEC and wakeword stack (§ 5 Transport). The substrate keeps only the one HTTP/2 idea it needs (a multiplexed, peer-parity id) and drops the rest.

Message framing versus the raw tail

gRPC frames every message as a length-delimited protobuf, which fights Alabama’s zero-copy design: here the Envelope header is protobuf but the audio or video payload rides the frame tail as raw bytes a device passes straight to its I2S audio bus [I2S-ESP-IDF] without decoding (§ 6 Wire encoding and framing). Wrapping each chunk as a gRPC message would force a copy on the hottest path.

Identity

Alabama’s device identity is the Noise static key authenticated in the transport handshake (§ 7 Identity and admission), with trust-on-first-use admission. gRPC offers no native equivalent; its security is TLS plus an application-level auth token, which does not give a public-key device identity for free.

None of these rules out gRPC as an idea source — the substrate borrows its best one — but they rule it out as the wire.

Appendix A: Full wire schema

Every message and enum, as field tables. The **canonical source of truth is the .proto** under docs/protocol/proto/ in the repository — these tables are generated from it at build time, so they cannot drift from it, and an implementer wanting the exact protobuf reads it there.

envelope.proto

surface.proto

content.proto

operations.proto