Build a streaming assistant client

Connect to the running sidecar

A streaming assistant client connects to the same local sidecar as the desktop. Obtain its assigned loopback URL and current application bearer token from the connection you are integrating with. A port saved from yesterday's launch is not a durable address: the sidecar requests an ephemeral port each time it starts. An account session token, companion pairing token, and scoped action-bridge token have different purposes and cannot replace the application bearer on this chat route.

Before submitting a turn, read GET /api/ai/status, /api/ai/providers, and optionally /api/ai/models?configuredOnly=true. A configured credential means that a key is present; it does not prove that the provider will accept the request or that an endpoint supports the selected model. The default host is embedded nikcli. Direct mode can stream a model answer but does not supply the embedded host's persistent tool loop or specialist scheduler.

This guide uses CADCAMFUN_URL and CADCAMFUN_TOKEN as client-shell variables. Supply them through your existing connection handling. See the AI API reference for the exact schemas and Sidecar for launch behavior. The examples do not install credentials or choose a model on your behalf.

Choose project, drawing, and session identity

The client can supply a sessionId on its first turn and reuse it on later turns. The service maps that client identifier to the actual host session and checks the binding before each prompt. Retain the event/session identity as conversation state, rather than creating a different identifier every time the UI reconnects.

A real projectId gives the turn access to that project's permitted resources. An optional drawingId must belong to that project; it binds the turn's tools to that drawing. A drawing cannot be named without its project. A session already owned by another account or project is refused. These are admission checks before the model can execute project tools, not hints in a system prompt.

For a general explanatory conversation, omit project and drawing. The application uses its default context. For geometry work, carry the actual selected project/drawing IDs through chat, state recovery, queue, abort, and control requests. An account switch should not silently adopt the previous account's active session.

Submit a streaming turn

POST /api/ai/chat/stream returns an SSE body. It requires a non-blank message even though the transport schema makes several fields optional so handlers can return readable validation messages. The following general question disables stored context injection, making it suitable for checking the transport independently of project geometry.

curl --no-buffer --fail-with-body \
  -H "Authorization: Bearer $CADCAMFUN_TOKEN" \
  -H "Content-Type: application/json" \
  --data '{"message":"Explain the difference between G0 and G1.","sessionId":"inspection-demo","includeContext":false,"mode":"gcode"}' \
  "$CADCAMFUN_URL/api/ai/chat/stream"

Add projectId and drawingId to work on a specific drawing. mode accepts general, cad, cam, gcode, or toolpath. A model selector can contain provider, model, tier, and variant; tiers are fast, balanced, or best. Discover provider/model IDs from the running catalogue instead of hard-coding an assumption about which vendor is available.

permissionMode accepts approve, ask, or auto. Destructive tool groups retain human approval in every mode. The agent field selects a configured assistant role; read GET /api/ai/agents rather than guessing role IDs.

Read complete SSE data frames

The route uses Effect's SSE data mode: each SSE data payload is a JSON event record with a type discriminator. A network chunk is not an event boundary. JSON, line endings, and UTF-8 characters can be split between reads. Accumulate decoded text and parse only complete frames, joining multiple data lines when present.

async function* events(response: Response): AsyncGenerator<Record<string, unknown>> {
  if (!response.ok) throw new Error(await response.text())
  if (!response.body) throw new Error("The response has no stream")
  const reader = response.body.getReader()
  const decoder = new TextDecoder()
  let pending = ""
  try {
    while (true) {
      const next = await reader.read()
      pending += decoder.decode(next.value, { stream: !next.done })
      let boundary: RegExpExecArray | null
      while ((boundary = /\r?\n\r?\n/.exec(pending)) !== null) {
        const frame = pending.slice(0, boundary.index)
        pending = pending.slice(boundary.index + boundary[0].length)
        const data = frame.split(/\r?\n/)
          .filter((line) => line.startsWith("data:"))
          .map((line) => line.slice(5).replace(/^ /, "")).join("\n")
        if (!data) continue
        const event: unknown = JSON.parse(data)
        if (!event || typeof event !== "object" || !("type" in event)) {
          throw new Error("Invalid assistant event")
        }
        yield event as Record<string, unknown>
      }
      if (next.done) break
    }
  } finally {
    await reader.cancel().catch(() => {})
    reader.releaseLock()
  }
}

This reader demonstrates framing; a production client should also validate each event against the shared contract or use the typed generated streaming client. It intentionally ignores SSE comment/control fields because this endpoint's application state lives in JSON data records. Releasing a client stream is transport cleanup, not an explicit request to abort server-side work.

Render progress without duplicating content

Start the fetch with the same bearer and JSON body shown above, then iterate the reader. Append delta.text to visible answer text and route reasoning.text to its own view. A done event carries the final response text; replace or reconcile the accumulated answer with that final value instead of appending the entire answer again.

Correlate tool calls and results by callId, not their array position. A failed tool can return useful diagnostics while the assistant continues. A provider error after a successful editing tool does not roll back that geometry change.

Answer permissions and questions

A permission_request includes sessionId, requestId, permission, and patterns. Present the request to the person and send the selected decision to POST /api/ai/permission/reply. The equivalent session-control action is reply_permission. The server verifies that the request is still pending in the conversation or an admitted child session.

{
  "sessionId": "inspection-demo",
  "requestId": "request-id-from-the-event",
  "decision": "once"
}

Decision is once, always, or reject. It is a server permission decision, distinct from an action argument such as confirm. A pending question_request contains an ordered questions array. Reply through /api/ai/sessions/control with action answer_question, the request ID, and one answer array per question; reject it with reject_question.

{
  "sessionId": "inspection-demo",
  "action": "answer_question",
  "requestId": "question-id-from-the-event",
  "answers": [["The selected option label"]]
}

Every answer must be non-empty. A single-choice question rejects multiple answers, and custom=false permits only listed labels. Preserve the exact labels and question order. If the server says the request is no longer pending, refresh controls instead of repeatedly posting the old reply.

Queue, steer, and explicitly stop work

Use POST /api/ai/chat/queue for a message submitted while the turn works. It takes sessionId, message, optional projectId, and delivery. queue is the default and waits for the next safe step; steer asks the host to incorporate it promptly. A user_message event signals that the message has entered the conversation.

Use POST /api/ai/chat/abort with the conversation identity when the person presses Stop. This stops the turn and joins its associated workers. Do not implement Stop solely by cancelling fetch: losing the HTTP stream is not the same operation. A session can then accept another turn without creating a new project identity.

POST /api/ai/sessions/remove removes the host session after cancelling and joining its children. It returns a removed flag. Removing a conversation is not a rollback of project edits made by its tools; review or restore those through the project/history APIs when needed.

Recover after losing the stream

When the reader ends without a final event, treat completion as unknown. Call POST /api/ai/sessions/state with the same sessionId and projectId. The response contains busy and text, with optional recovered tool/todo/diagnostic events, usage, and tasks. Reconstruct the UI from this snapshot; replaying recovered tool events does not execute them again.

Then request /api/ai/sessions/control with action status to inspect pending permissions/questions, worker tasks, goal, and context policy. A busy turn waiting for an unanswered question needs a reply, not a second copy of the original prompt. A session reported idle with accumulated text can be displayed as recovered output even though the original TCP connection vanished.

Keep a distinction between network failure, a typed provider/config error event, explicit abort, and session removal. Retry the model request only after checking what the first turn actually did. In particular, read the current drawing after a partial mutation diagnostic or interrupted editing worker.

Inside the server, a dropped model connection is resumed up to five times with a 2-second backoff that doubles up to 30 seconds. A client reconnecting to the stream should still read the session state, because those resumes can finish work that the client did not see stream.

Context, attachments, and compaction

includeContext: false disables acquisition of stored context; explicit context text can still be supplied for that turn. Normal assembly orders standing requirements, current observations, UI snapshots, memory, and supporting references, deduplicates them, and applies a 24,000-character aggregate budget. Omitted/truncated blocks are reported; this budget is not the provider's token window.

Attachments include name, optional MIME, data, and optional text/base64 encoding. There are at most five per request. Images are limited to 5 MiB each, PDFs to 10 MiB, and text to 2,000,000 characters; short text up to 6,000 characters also appears inline. Text retained for tools is project-scoped in memory, so reconnecting is different from restarting the service. Unsupported files are rejected with reasons.

Control action context requests a detailed token breakdown; ordinary status polling avoids that tokenizer work. Action compact asks the host to summarize older conversation material. Memory consolidation is a separate host operation. Persist the source drawing and manufacturing requests through their own APIs rather than relying on a compacted conversation to reproduce exact geometry.

Separate transport errors from domain outcomes

HTTP 401 means the request did not pass bearer authorization. A 400 before streaming commonly means an empty message, inaccessible session identity, invalid project/drawing combination, or malformed body. Once a stream is established, the handler can emit an error event with config, provider, aborted, or internal classification; handle that event even when the response status was 200.

A tool result with ok=false is a tool outcome, not necessarily a failed stream. Show its output and diagnostic so the assistant or person can recover. A worker queue can also be full: the scheduler permits four active and sixteen waiting tasks. An interrupted reader may be recovered within bounded attempts; editing work is treated as uncertain rather than automatically replayed.

Read the executable contracts in packages/api/src/groups/ai.ts and ai-events.ts; admission and wire behavior in apps/sidecar/src/http/ai.ts; session mapping in packages/ai/src/service.ts; question validation in nikcli/session-controls.ts; heartbeat timing in heartbeat.ts. These sources define which outcome the client must present, independently of the assistant's prose.

Event reference

Every SSE data: frame is one JSON object with a type and a sessionId. Parse complete frames only; a frame may arrive split across network chunks.

A complete streaming client

const base = process.env.CADCAMFUN_URL!       // e.g. http://127.0.0.1:53211
const token = process.env.CADCAMFUN_TOKEN!

const response = await fetch(`${base}/api/ai/chat/stream`, {
  method: "POST",
  headers: { "content-type": "application/json", authorization: `Bearer ${token}` },
  body: JSON.stringify({ projectId: "p1", message: "Summarise this project", mode: "general" }),
})
if (!response.ok || !response.body) throw new Error(`HTTP ${response.status}`)

const reader = response.body.pipeThrough(new TextDecoderStream()).getReader()
let buffer = "", answer = ""
for (;;) {
  const { value, done } = await reader.read()
  if (done) break
  buffer += value
  let end
  while ((end = buffer.indexOf("\n\n")) >= 0) {        // one SSE frame
    const frame = buffer.slice(0, end); buffer = buffer.slice(end + 2)
    const data = frame.split("\n").filter((l) => l.startsWith("data:"))
                      .map((l) => l.slice(5).trimStart()).join("\n")
    if (!data) continue
    const event = JSON.parse(data)
    switch (event.type) {
      case "delta": answer += event.text; break
      case "permission_request": /* ask, then POST /api/ai/permission/reply */ break
      case "done": answer = event.text; console.log(answer); break
      case "error": throw new Error(`${event.code}: ${event.message}`)
    }
  }
}

If the connection drops, do not send the message again. Read the session with POST /api/ai/sessions/state, because the server may have resumed the model connection and finished the turn without you. Use /api/ai/chat/abort to stop a turn deliberately, and /api/ai/chat/queue to add a message while the session is busy.

What a stream looks like without a provider

With no AI provider configured, the stream still opens and answers with a typed error, followed by a heartbeat. Captured from a fresh sidecar:

data: {"type":"error","sessionId":"","code":"config","message":"No AI provider is configured. Add an API key in the AI settings."}

data: {"type":"heartbeat"}

• The HTTP status is still 200 and the content type text/event-stream: configuration problems are domain outcomes, delivered as events.
• sessionId is empty because no session was created.
• heartbeat frames keep the connection alive; ignore them in rendering and do not count them as progress.
• Each frame ends with a blank line. Parse by splitting on blank lines and joining the data: lines of a frame, as the SDK iterator does.

In 3D