Documentation
AI assistant
Project-scoped conversations, typed CAD/CAM actions, explicit permissions, and provider credentials managed by the application.
Assistant services
@cadcamfun/ai provides model access, conversation events, the action catalogue, prompts, skills, context assembly, and host integration. The default conversation engine is embedded nikcli; AI SDK v7 supplies direct text, streaming, and structured-output calls. The sidecar injects handlers for project mutations, libraries, history, and application storage.
Chat can inspect and change a project through registered actions. Stateless title generation, design analysis, parameter recommendations, G-code optimization, and design-suggestion summaries return advice or structured output; those service methods do not save project edits. Review generated machining advice through deterministic CAM and verification before relying on it.
Conversations and session controls
POST /api/ai/chat/stream emits typed events for text, reasoning, turn/step state, tool calls and results, diagnostics, questions, permission requests, retries, context changes, goals, completion, and errors. Client session identifiers are bound to host sessions and checked against their project. Retain the session ID to continue the same conversation.
POST /api/ai/sessions/state recovers busy state, accumulated text, and task information after a disconnected stream. /api/ai/chat/queue admits a follow-up to an active conversation; /api/ai/chat/abort stops the turn and its associated workers. Removing a session cancels and joins its children before removing the host session.
/api/ai/sessions/control supports status, detailed context inspection, compaction, memory consolidation, task cancellation, permission decisions, and answering or rejecting pending questions. Requests are checked against the parent conversation and its admitted child sessions. A question that is no longer pending cannot be answered by reusing its old identifier.
Direct model calls retry transient provider failures with the same curve as nikcli: the first delay is 2 seconds and doubles on each attempt, a provider's own retry-after header is honoured, and without such a header the delay never exceeds 30 seconds. By default a request makes at most five calls, including the first; the retryAttempts setting can override this (packages/ai/src/retry.ts, direct-model.ts). When the model connection drops mid-turn, the chat loop resumes up to five times with the same 2–30 second backoff and asks the model to continue without repeating completed steps.
Providers and credentials
GET /api/ai/providers lists provider metadata and configured state; GET /api/ai/models exposes the model picker catalogue with optional configuredOnly=true. Embedded-host catalogue reads are cached for one minute. Configuration and policy affect which providers are listed and usable.
Set credentials with PUT /api/ai/credentials and remove them with DELETE /api/ai/credentials/:provider. Keys must be non-empty, and a supplied base URL must be HTTP or HTTPS. The direct model factory supports Anthropic, OpenAI, MiniMax, and the AI gateway. Other provider IDs require an OpenAI-compatible base URL for direct calls. The embedded host uses its own provider catalogue and routing.
The sidecar persists credentials at <dataDir>/ai/credentials.json using a temporary file, owner-only mode 0600, and rename. A missing or damaged file reads as empty. A failed persistence write leaves the key available for the current process but does not guarantee it survives restart. Status responses expose configuration state rather than raw keys.
Credentials are represented with Redacted inside services and propagated to the host through a scoped credential sink. Closing the scope clears the host's copy. Credential changes update a revision used to distinguish cached stateless results.
The default model tier is balanced, and MiniMax requests go to https://api.minimax.io/v1 unless CADCAMFUN_AI_MINIMAX_BASE_URL overrides it (packages/ai/src/models.ts). The model catalogue is seeded from the public https://models.dev/api.json endpoint only when the cached file is missing and network access is allowed; no credentials are sent, responses are capped at 8 MiB, and a body listing fewer than 20 providers is rejected as truncated.
Actions and permissions
The action catalogue declares each action identifier, parameter schema, effect, and execution path. Tools group related actions behind a required action field. The host validates tool input, and sidecar handlers perform domain validation and resource access checks before persisting changes.
Permission modes are approve, ask, and auto. They affect how ordinary tool calls are admitted; tools containing destructive actions retain a human approval rule in every mode. An input flag such as confirm: true does not replace that permission request. The host does not expose unrestricted shell or file-editing tools as a substitute for application actions.
Policies in <dataDir>/ai/policy.json restrict provider use, tool use, and bridge access. They are loaded by the service configuration; do not assume editing the file updates an already-running service. Project-scoped scratch directories and isolated nikcli configuration keep the embedded host separate from a user's standalone agent state.
Skills provide CAD/CAM playbooks and user-authored instructions. They guide use of existing tools; adding a skill does not add a new mutation API. The skills HTTP endpoints list, save, and delete supported user skill content.
Registered tool groups
The built-in groups are cad, cad_edit, cam, cam_edit, library, parts, library_edit, memory, memory_edit, research, vcs, vcs_edit, and agent. cam_verify exposes the generation and verification subset needed by the verifier role without granting the full CAM editing tool.
Tool grouping preserves effect boundaries. Read-only geometry inspection, library lookup, and memory recall are separate from changes to those resources. The injected registry can also admit extension tools; duplicate identifiers are rejected.
Discover the current catalogue
GET /api/ai/actions returns tool descriptions, action names and kinds, and JSON parameter schemas for outside callers. Use it to discover the running application's actions instead of copying a static list into a client. POST /api/ai/actions/invoke takes a project ID, tool ID, and input containing the action discriminator.
Built-in catalogue entries are maintained under packages/ai/src/actions. Examples include CAD inspection and parametric checks, CAD edits, CAM planning and manufacture, library lookup and updates, project memory, research, and VCS proposals. A contribution requiring an application service returns an unavailable result when that host did not supply its handler.
Public integration APIs
ACTION_CATALOGUE, ACTION_TOOL_IDS, EXPOSED_TOOL_IDS, actionById, toolKind, and runAction support catalogue consumers. ToolHandlers is the sidecar integration contract; ToolRegistry composes built-in and extension tools. AiService supplies conversations, model metadata, one-shot calls, and session operations.
assembleContext composes turn context with a default aggregate budget of 24,000 characters. It orders standing requirements, current observations, UI snapshots, memory, and supporting references, deduplicates by identity and content, and records omitted or truncated sources. This context budget is separate from the model's full token window and host compaction threshold.
Background work and evidence
The runtime scheduler allows four active background tasks and sixteen queued tasks. Tasks are owned by a project and parent session. Cancellation retains active ownership until a worker stops; stopping or deleting the parent also stops its associated work. Read-worker recovery is bounded to two attempts, while interrupted editing work is marked uncertain instead of blindly replayed.
Assistant tool success is a result to inspect. CAD operations can return partial geometry diagnostics; CAM artifacts carry their own claims and release levels; history services enforce protected branches. Conversation text, a task marked complete, and a geometric verification result describe different evidence.
Project context sent to the model is budgeted: a stored project context document may hold up to 8,000 characters, of which 2,000 reach the prompt, with 1,500 characters each for project memory and user habits. Project notes older than 90 days that the project context does not reference are pruned when a session settles (packages/ai/src/project-context.ts).
Runtime selection
CADCAMFUN_AI_HOST selects nikcli by default or direct. The direct path uses AI SDK calls without the embedded host's persistent agent tool loop and specialist scheduler. Stateless operations use the direct model service even when the embedded host is enabled.
The host's XDG directories and optional home override are rooted under <dataDir>/ai/nikcli. Project configuration loading and unrelated inherited agent settings are disabled or scrubbed before the SDK import. Its workspace is a per-project application scratch directory, rather than the user's source directory.
Standalone nikcli access
@cadcamfunv2/cadcamfun-nikcli-plugin connects a standalone nikcli to a running application. Configure its url, token, and default projectId, or use CADCAMFUN_URL, CADCAMFUN_TOKEN, and CADCAMFUN_PROJECT. Supply a scoped bridge token issued by the application for outside access.
Bridge tokens are restricted to one project and read-only or read/write actions. They default to 30 days, allow 1–365 days, can be listed and revoked, and reveal their secret only when issued. They admit only the two action bridge routes. Policy can narrow the visible tools and actions further.
The plugin fetches the catalogue from the application and converts its schemas into nikcli tool arguments. Destructive actions are refused even with a confirmation flag, and the in-process agent delegation tool is excluded. Write-capable plugin extension tools are not exposed through this outside bridge. HTTP 401 reports a rejected token; action refusals return readable failure results.
Implementation map
service.ts owns application-facing AI operations; direct-model.ts owns AI SDK calls; credentials.ts owns key state; nikcli/ owns isolation, sessions, policies, context, and workers; actions/ and tools/ own capability definitions and adapters. Sidecar services/ai.ts wires project handlers and storage. packages/nikcli-plugin owns the outside HTTP client.
Start and recover a conversation
With a running sidecar and a configured model, this request opens a conversation without injecting stored workspace context. Set CADCAMFUN_URL to the discovered loopback base URL and CADCAMFUN_TOKEN to its current application token. The session identifier below is client-selected; keep it for follow-ups.
curl --no-buffer --fail-with-body \
-H "Authorization: Bearer $CADCAMFUN_TOKEN" \
-H "Content-Type: application/json" \
--data '{"message":"Explain the difference between G0 and G1.","sessionId":"inspection-demo","includeContext":false,"mode":"gcode"}' \
"$CADCAMFUN_URL/api/ai/chat/stream"For a project turn, also supply its real projectId, and optionally a drawingId belonging to that project. The drawing binds the turn's tools to that drawing. Reusing a session under another project or account is refused.
curl --fail-with-body \
-H "Authorization: Bearer $CADCAMFUN_TOKEN" \
-H "Content-Type: application/json" \
--data '{"sessionId":"inspection-demo"}' \
"$CADCAMFUN_URL/api/ai/sessions/state"The returned busy, text, optional events, and tasks reconstruct the view; replaying those stored events does not execute tools again. To stop work, send an explicit abort. Closing the stream alone does not join the conversation's workers.
Source: packages/api/src/groups/ai.ts, apps/sidecar/src/http/ai.ts, and packages/ai/src/service.ts.
Handle events without losing state
| Event | Consumer behavior |
|---|---|
delta / reasoning | Append text to the matching session's text or reasoning channel. |
tool_call / tool_result | Correlate by callId; inspect ok and retain diagnostics independently of assistant text. |
permission_request | Show the request and reply with its request/session IDs and once, always, or reject. |
question_request | Keep its request ID and questions; send one answer array per question through session controls. |
done | Use final text and optional usage as the completed response; do not append the entire final text to already accumulated deltas. |
error | Display message with config, provider, aborted, or internal classification. |
heartbeat | Keep the connection alive without adding content or declaring the turn complete. |
Session controls accept only pending permission/question IDs in that conversation or its child sessions. A question requires non-empty answers; a single-choice question rejects multiple selected values, and a question that disallows custom answers accepts only its listed option labels.
Source: packages/api/src/ai-events.ts, packages/ai/src/nikcli/session-controls.ts, and packages/ai/src/heartbeat.ts.
Attachments and user skills
Each chat request accepts up to five attachments. Supported images are PNG, JPEG, WebP, and GIF, with a 5 MiB limit each; PDFs have a 10 MiB limit. Text attachments accept up to 2,000,000 characters, with texts up to 6,000 characters also included inline. Unsupported files produce a rejection reason rather than silently disappearing.
Text attachments are kept in a project-scoped in-memory store for import tools: at most 32 names and 8,000,000 characters per project, and 32,000,000 characters across at most 64 stored projects. Reusing a name replaces its previous text. Restarting the service does not retain this attachment store. A PDF-capability mismatch is reported to the model by the provider transform.
User skill names are lowercase letter/digit words joined by hyphens, at most 64 characters. Description and body limits are 300 and 20,000 characters respectively; at most 50 user skills can be loaded. Built-in skills cannot be removed through the user skill API. A saved skill's live result indicates whether host reload took effect.
Source: packages/ai/src/attachments.ts, packages/ai/src/skills/user.ts, and apps/sidecar/src/http/ai.ts.
Diagnose assistant failures
- No configured model: inspect
/api/ai/statusand/api/ai/providers, then set a key for a listed provider. A configured flag proves a stored key exists, not that the vendor accepted it. - Unknown provider or invalid URL: use an advertised provider ID and an HTTP(S) base URL. A non-built-in direct provider needs a compatible endpoint in addition to its key.
- Session inaccessible: recover using the original project/account/session identity. Do not invent a replacement project ID for an existing host session.
- A turn waits indefinitely: inspect session control state for pending permissions or questions before retrying a duplicate prompt. Use explicit abort to cancel the turn and join its workers.
- A tool partly changed the model: inspect its
tool_diagnostic, includingmutationApplied, then read the current drawing. Retrying an editing action without inspecting its result can duplicate geometry.
Tool results and typed failures preserve the stage that failed. A provider error after successful tools does not roll those project mutations back. Deterministic CAM and history checks remain available separately from the model answer.
Assistant endpoint reference
All assistant routes live under /api/ai and require the application bearer token. A bridge token for a standalone nikcli opens only the action listing and invocation routes, and only for its project.
| Route | Purpose |
|---|---|
POST /api/ai/chat/stream | Start a streaming turn; the response is an SSE stream of assistant events. |
POST /api/ai/chat/queue | Queue a follow-up message for a busy session. |
POST /api/ai/chat/abort | Abort the running turn of a session. |
POST /api/ai/permission/reply | Answer a permission request: once, always, or reject. |
POST /api/ai/sessions/state, /sessions/control, /sessions/remove | Read session state, control a session, remove one. |
GET /api/ai/status, /models, /providers, /agents, /prompts | Discovery: readiness, model catalogue, providers with their configuration state, specialist agents, prompt suggestions. |
PUT /api/ai/credentials, DELETE /api/ai/credentials/:provider | Store or clear provider credentials. |
GET/PUT /api/ai/skills, DELETE /api/ai/skills/:name | List, save, and delete user skills. |
GET /api/ai/actions, POST /api/ai/actions/invoke | Action catalogue and direct invocation (also used by the nikcli bridge). |
GET/POST /api/ai/bridge/tokens, DELETE /api/ai/bridge/tokens/:id | Issue, list, and revoke scoped bridge tokens. |
POST /api/ai/completion | Single completion: prompt, systemPrompt, model, temperature; returns text and model. |
POST /api/ai/title | Title for a conversation excerpt. |
POST /api/ai/design-analysis | Suggestions with confidence, priority (low/medium/high), and kind (optimization/warning/critical). |
POST /api/ai/machining-parameters | Cutting speed (m/min), feed (mm/min), depth of cut (mm), stepover (%), coolant (flood/mist/none), engagement (conventional/climb), and warnings. |
POST /api/ai/gcode-optimization | Optimised G-code with per-line changes and reasons, estimated time and tool-life savings, and notes. |
POST /api/ai/design-suggestions | Next-step suggestions with title, rationale, and next step. |
POST /api/ai/analyze-gcode | Issues (error/warning/info with code, line, and message) and statistics: lines, rapid and cut moves, tool changes, and bounds. |
POST /api/ai/gcode, /toolpath-analysis, /feedback, /proxy | G-code actions, toolpath insights, feedback, and the provider proxy. |
Model-backed analysis replies are model output in a typed shape. Values such as machining parameters are suggestions to review against tooling data and machine limits, not verified cutting data.
Streaming request fields
| Field | Values | Effect |
|---|---|---|
message | string | The user's message for this turn. |
sessionId | string | Continue an existing session; omit to start a new one. |
projectId | string | Project whose context and tools the turn uses. |
drawingId | string | The drawing this turn may act on. It is checked against the project before the turn is admitted, and the turn's tools act on this drawing and nothing else. |
mode | general, cad, cam, gcode, toolpath | Focus of the turn. |
permissionMode | auto, ask, approve | How tool permissions are requested. |
agent | string | Specialist agent name from /api/ai/agents. |
attachments | [{ name, mime?, data, encoding?: "base64" | "text" }] | Up to 5 attachments: images up to 5 MB, PDFs up to 10 MB, and text up to 2,000,000 characters (text files up to 6,000 characters are also written into the message). |
context, includeContext | string, boolean | Extra context text and whether to include assembled project context. |
model | model selector | Override the configured model for this turn. |
Limits and defaults
| Limit | Value |
|---|---|
| Assembled context budget | 24,000 characters; a block that does not fit is truncated with a marker, and a block that would be cut below 400 characters is left out instead |
| Project context document | Up to 8,000 stored characters, 2,000 sent to the model |
| Project memory / user habits | 1,500 characters each |
| Project notes | Pruned after 90 days when not referenced by the project context |
| Tool output sent to the model | 24,000 characters |
| Elements created per call | 40 |
| Output tokens | 32,000 maximum |
| Heartbeat | Every 5 seconds while the stream is active |
| Exact-geometry actions | 60-second budget; exact reports 1.5 seconds |
| Kernel jobs | 120-second deadline, worker start included |
| Background tasks | 4 active, 16 queued, 2 attempts for read workers, settled tasks kept 7 days |
| Automatic continuations | 50 per request; 5 connection resumes with 2–30 s backoff |
| Direct-call retries | 5 calls including the first; 2 s doubling to 30 s, honouring retry-after |
| User skills | 50 skills; name up to 64 characters (lowercase words joined by hyphens), description up to 300, body up to 20,000 |
| Web search fetch | 256 KB per response |
| Bridge tokens | 30 days by default, 365 days maximum |
Read the result, not the prose
A tool's outcome is in tool_result and tool_diagnostic events. Check whether a mutation was applied before trusting the assistant's summary of what it did.
The full action catalogue
The assistant acts through 83 declared actions in 13 families. Every action declares its result schema and its effects: what it changes or reaches beyond reading the open document. Read families (cad, cam, library, memory, research, vcs) mostly have no effect; their _edit counterparts change persistent state. Effects are listed in parentheses.
| Family | Count | Actions (effects) |
|---|---|---|
cad | 20 | describe, check, relations, grep, vision (interface), inspect_elements, inspect_layers, inspect_search, inspect_property, drawing, parametric_validate, parametric_inspect, parametric_test, parametric_dfm, parametric_exact, exact, assembly, parametric_fastener, parametric_recipe, parametric_from_element |
cad_edit | 12 | add (drawing), add_entities (drawing), modify (drawing), transform (drawing), group (drawing), delete (drawing), editor (interface+drawing), import (drawing), parametric_create (drawing), parametric_update (drawing), parametric_recipe_create (drawing), parametric_convert (drawing) |
cam | 14 | plan_check, review, machine, operations, suggest, compare_setups, analyze, stock, vision (interface), chain_suggest, materials, cutting_data, manufacture, chain_status |
cam_edit | 6 | manufacture (operation+program), create (operation), update (operation), generate (program), verify (report), stock_set (stock) |
library | 5 | component_list, lookup_component, lookup_material, lookup_tool, lookup_machine_config |
parts | 3 | save (component), insert (drawing), delete (component) |
library_edit | 5 | upsert_material (library), upsert_tool (library), upsert_machine_config (library), set_default (library), delete (library) |
memory | 2 | recall, context_read |
memory_edit | 2 | remember (memory), context_update (context) |
research | 2 | search (external), read_attachment |
vcs | 3 | status, diff, list_pull_requests |
vcs_edit | 5 | start_change (history), commit (history), restore (history+drawing), open_pull_request (history), run_workflows (workflow) |
agent | 4 | start (task), list, read, cancel (task) |
| Effect | Meaning |
|---|---|
drawing, component, operation, program, stock, report | Persistent design or manufacturing data |
library, memory, context | Library records, project memory, workspace context |
history, workflow | Version control and workflow runs |
task | The delegation scheduler (background agents) |
interface | The editor or viewport on screen; not persistent |
external | The network (web research) |
A read action may only have interface or external; every write names at least one persistent effect. Effects are declared per action and never inferred from the tool that implements it.
How an action call ends
| Outcome | When |
|---|---|
succeeded | The action did what was asked. Includes reads with a negative finding, such as a failing check. |
partial | Part of the answer or batch: complete: false, omitted > 0, non-empty skipped, or a nextOffset to page on |
refused | Nothing done: ok: false with an error or tag (invalid input, denied, unavailable, nothing to do) |
conflict | The document moved on: ActionConflict or status stale/conflict. Inspect again before retrying. |
cancelled | ActionInterrupted |
uncertain | ActionUncertain: a write may have committed but its answer could not be read. Reconcile by inspecting; never retry blindly. |
Writes are bound to a drawing revision. A committed write returns an ActionCommitted receipt with previousRevision and revision; a write against a stale expectedRevision returns ActionConflict with both expectedRevision and actualRevision. The same classification (actionOutcome) is shared by the host, Code Mode, the desktop and the bridge.
{
"ok": false,
"_tag": "ActionConflict",
"actionId": "cad_edit.modify",
"projectId": "…", "drawingId": "…",
"expectedRevision": 41, "actualRevision": 42,
"message": "…"
}