AI Conversation

Three things every realtime AI surface needs:

Streamed tokens — render the assistant's reply word by word, without re-rendering the entire history per delta.
Presets — a single send with a typed preset field that the server routes (summarize, translate, explain, generate code).
Artifacts — structured outputs (code, docs, tables) that arrive alongside the stream and render in their own card.

One useSocketEventBatch for tokens, two useSocketEvents for the lifecycle frames, one zustand store split into history / streaming / artifacts.

Live demo

Pick a preset, ask a question. The Code preset emits an artifact card at stream-end.

AI Conversation

idle

Pick a preset and ask. Tokens stream in; code preset emits an artifact at the end.

Protocol

protocol.ts

type TPreset = "summarize" | "translate" | "explain" | "code";

type TClientMsg = {
  type: "ask";
  id: string;
  prompt: string;
  preset?: TPreset;
};

type TServerMsg =
  | { type: "stream-start"; id: string; role: "assistant" }
  | { type: "stream-delta"; id: string; delta: string }
  | { type: "artifact"; id: string; kind: "code" | "doc" | "table"; title: string; body: string }
  | { type: "stream-end"; id: string };

Token batching

Tokens arrive at 30–100ms cadence. A setState per delta is fine for one stream — death for many. Coalesce inside the bridge:

StreamBridge.tsx

function StreamBridge() {
  useSocketEvent(manager, "stream-start", (msg) => {
    useStore.getState().startStream(msg.id);
  });
  useSocketEventBatch(
    manager,
    "stream-delta",
    (msgs) => {
      // Group by stream id (concurrent streams share the channel)
      const grouped = new Map<string, string[]>();
      for (const m of msgs) {
        const list = grouped.get(m.id) ?? [];
        list.push(m.delta);
        grouped.set(m.id, list);
      }
      for (const [id, deltas] of grouped) {
        useStore.getState().appendDeltas(id, deltas);
      }
    },
    { flushMs: 16, idleMs: 8 },
  );
  useSocketEvent(manager, "artifact", (msg) => {
    useStore.getState().addArtifact(msg.id, msg);
  });
  useSocketEvent(manager, "stream-end", (msg) => {
    useStore.getState().endStream(msg.id);
  });
  return null;
}

Why split history / streaming / artifacts

Each slice has a different change frequency. History only changes when a turn finishes, so it can be memoised and skipped during a stream. Streaming mutates per token but only ever for one component. Artifacts only land at stream-end. Keeping them in separate fields is the simplest thing that makes per-token renders cheap.

store.ts

type TStore = {
  history: TTurn[];                          // append once per finished turn
  streaming: Record<string, string>;         // mutates per delta, keyed by stream id
  artifacts: Record<string, TArtifact>;      // lands at stream-end, keyed by stream id
};