Chat system map (Frontend)

Scope (MUST stay accurate):

• src/app/chat/**

Goal of this document:

• Give a complete mental model of the chat feature.
• Map every file in-scope, what it does, what depends on it, and whether it’s active/legacy/partial.
• Provide visual diagrams (Mermaid) for runtime flows + state machines.
• Identify likely dead code, duplicate code paths, and unintegrated features.

Non-goals:

• This file does not implement refactors.
• It can propose consolidation targets, but it should not prescribe UI/UX.

---

0) Glossary

• ChatMessage: UI message shape (role + parts[]) used in the browser and stored by the controller.
• Run stream: Controller-owned SSE stream from POST /chats/:sessionId/turn emitting run_start, message_*, tool_execution_*, plan_updated, and run_end.
• Pi agent runtime: Controller-owned agent loop built on @mariozechner/pi-agent-core + @mariozechner/pi-ai that executes tools server-side.
• Controller: vLLM Studio backend that stores sessions/messages, MCP config, runs tools, etc. Accessed from browser via /api/proxy/... (not in scope, but critical dependency).
• Inference backend: OpenAI-compatible /v1 endpoint (vLLM/LiteLLM/etc) called by the controller, not by Next.js.
• Artifacts: Code/UI snippets extracted from assistant text (```artifact-* ``` or <artifact ...>), shown in preview UI.
• Agent Plan: Synthetic tools create_plan + update_plan that let the model maintain a checklist in agent mode.

---

1) High-level architecture (3 “planes”)

1.1 Runtime system diagram

graph TD
  subgraph Browser["Browser / Next.js Client"]
    ChatRoutePage["page.tsx - dynamic, ssr:false"]
    ChatPage["chat-page.tsx - GOD component"]
    Zustand["Zustand store - chat-slice.ts"]
  end

  subgraph NextServer["Next.js Server"]
    ApiProxy["/api/proxy/* - controller proxy"]
    ApiTitle["/api/title"]
    ApiTranscribe["/api/voice/transcribe"]
  end

  subgraph Controller["Controller Backend"]
    Runs["POST /chats/:id/turn (SSE run stream)"]
    Sessions[("Sessions + Messages DB")]
    MCP[("MCP servers + tools")]
    Compaction[("Compaction endpoint")]
  end

  subgraph Inference["Inference Backend"]
    OpenAICompat["/v1/chat/completions"]
  end

  ChatRoutePage --> ChatPage

  ChatPage -->|"api.* via /api/proxy"| ApiProxy
  ApiProxy --> Runs
  ApiProxy --> Sessions
  ApiProxy --> MCP
  ApiProxy --> Compaction
  Runs --> OpenAICompat

  ChatPage -->|"title generation"| ApiTitle
  ChatPage -->|"voice transcription"| ApiTranscribe
  ChatPage <--> Zustand

1.2 Single controller-owned chat backend (important)

There is now one network path for chat:

1. Controller run stream

• Client sends messages to POST /chats/:sessionId/turn (via /api/proxy).
• Controller persists messages, runs the agent loop, executes tools, and streams SSE events.

2. Inference backend

• Only the controller talks to /v1 during the run.

This removes the previous split between streaming and persistence. Tool execution is server-side.

---

2) Core runtime loops (state machines)

2.1 Chat streaming + tool loop (controller-run)

stateDiagram-v2
  [*] --> Ready

  Ready --> Submitted: sendMessage()
  Submitted --> Streaming: open SSE run

  Streaming --> ToolExec: tool_execution_*\n(server-side)
  ToolExec --> Streaming: tool result events

  Streaming --> Ready: run_end

  Ready --> [*]: user leaves page

Notes:

• Tool calls are executed on the controller by the Pi agent runtime.
• The client only renders message_*, tool_execution_*, and plan_updated events from the run stream.

2.2 End-to-end sequence diagram (user message)

sequenceDiagram
  participant User
  participant ToolBelt as ToolBelt UI
  participant ChatPage as ChatPage (client)
  participant Controller as Controller (/api/proxy)
  participant Agent as Pi agent runtime
  participant LLM as Inference (/v1)

  User->>ToolBelt: Type message / attach / record
  ToolBelt->>ChatPage: onSubmit(text, attachments)

  ChatPage->>Controller: create session if needed\nPOST /chats
  ChatPage->>Controller: POST /chats/:id/turn (SSE run)

  Controller->>Agent: start run (Pi agent core)
  Agent->>LLM: chat/completions

  Agent-->>Controller: message_*\n tool_execution_*\n plan_updated\n run_end
  Controller-->>ChatPage: SSE events (stream)

  ChatPage->>Controller: abort run (optional)

2.3 Agent Plan state machine

stateDiagram-v2
  [*] --> NoPlan

  NoPlan --> Planned: create_plan(tasks)
  Planned --> Executing: update_plan(step, running)
  Executing --> Planned: update_plan(step, done)\n(next step becomes current)

  Executing --> Blocked: update_plan(step, blocked)
  Blocked --> Planned: update_plan(next step, running)

  Planned --> Done: all steps done
  Done --> NoPlan: clearPlan()

Where it lives:

• Tool definitions + execution: controller services/agent-runtime/tool-registry.ts (out of scope).
• UI: src/app/chat/_components/agent/agent-plan-drawer.tsx + chat-page.tsx event handling.

2.4 Session lifecycle state machine

stateDiagram-v2
  [*] --> NoSession

  NoSession --> SessionSelected: URL has ?session=...\n(loadSession)
  NoSession --> PendingCreate: first user send\n(createSessionWithMessage)

  PendingCreate --> ActiveSession: controller returns session id

  SessionSelected --> LoadingMessages: controller returns stored messages
  LoadingMessages --> ActiveSession: ChatPage.setMessages(mapped)

  ActiveSession --> Forking: forkChatSession(messageId)
  Forking --> ActiveSession: navigate to /chat?session=newId

  ActiveSession --> NewSession: ?new=1 OR user starts new
  NewSession --> NoSession

  ActiveSession --> Deleted: deleteSession
  Deleted --> NoSession

2.5 Attachments lifecycle (UI)

stateDiagram-v2
  [*] --> Empty

  Empty --> Picking: user opens file/image picker
  Picking --> Selected: file(s) chosen\n(attachments[] in Zustand)

  Selected --> Selected: add more
  Selected --> Empty: remove last attachment

  Selected --> Submitting: user hits Send
  Submitting --> Empty: ToolBelt clears attachments

  note right of Selected
    Current send pipeline
    does NOT pass binary files
    to the model. It only adds
    text placeholders.
  end note

2.6 Voice recording + transcription lifecycle

stateDiagram-v2
  [*] --> Idle

  Idle --> Recording: startRecording()\n(MediaRecorder.start)
  Recording --> Stopping: stopRecording()
  Stopping --> Transcribing: POST /api/voice/transcribe

  Transcribing --> Idle: success\nappend transcript to input
  Transcribing --> Error: failure\nshow transcriptionError
  Error --> Idle: dismiss or timeout

2.7 Artifact lifecycle (extraction → preview → modal)

graph TD
  A["Assistant ChatMessage parts - type:text"] --> B["ChatPage: extractArtifacts"]
  B --> C["sessionArtifacts + artifactsByMessage Map"]

  C --> D["UnifiedSidebar: ArtifactPreviewPanel"]
  C --> E["ChatMessageItem: MiniArtifactCard chips"]

  E -->|"click"| F["Zustand: activeArtifactId"]
  F --> G["ArtifactModal"]
  G --> H["ArtifactViewer - iframe runner"]

2.8 Context compaction lifecycle (auto)

stateDiagram-v2
  [*] --> Normal

  Normal --> High: utilization >= threshold
  High --> Compacting: runAutoCompaction

  Compacting --> Normal: success - setMessages
  Compacting --> Normal: failure - compactionError

2.9 Intra-chat component graph (in-scope)

graph TD
  ChatRoutePage["page.tsx"] --> ChatPage["chat-page.tsx"]

  ChatPage --> useChatSessions["use-chat-sessions"]
  ChatPage --> useChatTools["use-chat-tools"]
  ChatPage --> useChatDerived["use-chat-derived"]
  ChatPage --> useChatUsage["use-chat-usage"]
  ChatPage --> useAgentFiles["use-agent-files"]
  ChatPage --> useAgentState["use-agent-state"]

  ChatPage --> UnifiedSidebar["unified-sidebar"]
  UnifiedSidebar --> ChatConversation["chat-conversation"]
  ChatConversation --> ChatMessageList["chat-message-list"]
  ChatMessageList --> ChatMessageItem["chat-message-item"]
  ChatMessageItem --> MessageRenderer["message-renderer"]
  MessageRenderer --> EnhancedCodeBlock["enhanced-code-block"]

  ChatPage --> ChatToolbeltDock["chat-toolbelt-dock"]
  ChatToolbeltDock --> ToolBelt["tool-belt"]
  ToolBelt --> ToolBeltToolbar["tool-belt-toolbar"]
  ToolBeltToolbar --> ToolDropdown["tool-dropdown"]
  ToolBeltToolbar --> AgentModeToggle["agent-mode-toggle"]

  ChatPage --> ChatModals["chat-modals"]
  ChatModals --> ChatSettingsModal["chat-settings-modal"]
  ChatModals --> MCPSettingsModal["mcp-settings-modal"]
  ChatModals --> UsageModal["usage-modal"]
  ChatModals --> ExportModal["export-modal"]

  ChatPage --> ArtifactModal["artifact-modal"]
  ArtifactModal --> ArtifactViewer["artifact-viewer"]

  UnifiedSidebar --> ArtifactPreviewPanel["artifact-preview-panel"]
  UnifiedSidebar --> ActivityPanel["ActivityPanel - from chat-side-panel"]
  UnifiedSidebar --> ContextPanel["ContextPanel - from chat-side-panel"]
  UnifiedSidebar --> AgentFilesPanel["agent-files-panel"]
  ToolBelt --> AgentPlanDrawer["agent-plan-drawer"]

---

3) Feature map (what’s implemented vs partial vs missing)

Legend:

• ✅ Implemented + integrated
• 🟡 Implemented but partial / missing wiring
• 🔴 Missing (UI stub or config only)

---

4) State management model (Zustand)

Important: messages are NOT stored in Zustand.

• Messages live inside chat-page.tsx local state.
• Zustand stores UI state, sessions list/metadata, tool execution status, attachments, artifacts UI state, etc.

4.1 Zustand keys used directly by chat UI

Primary store file (external dependency): src/store/chat-slice.ts

Core keys used by ChatPage:

• Input: input, setInput
• Model: selectedModel, availableModels
• System prompt: systemPrompt
• Feature toggles: mcpEnabled, artifactsEnabled, deepResearch, agentMode
• Timing: streamingStartTime, elapsedSeconds
• Context queue: queuedContext
• Artifacts: activeArtifactId
• Session metadata: currentSessionId, currentSessionTitle, sessions[]
• Tool execution: executingTools, toolResultsMap
• Agent plan: agentPlan

4.2 Legacy state cleanup (done)

Legacy panel + artifact renderer keys were removed (toolPanelOpen, activePanel, artifactPanelSelectedId, artifactRendererState, codeSandboxState) to align with UnifiedSidebar.

---

5) Directory inventory (every file)

5.1 In-scope file tree

src/app/chat/
  page.tsx
  types.ts
  utils/index.ts
  hooks/
    use-agent-files.ts
    use-agent-state.ts
    use-chat-derived.ts
    use-chat-sessions.ts
    use-chat-tools.ts
    use-chat-usage.ts
  _components/
    agent/
      agent-files-panel.tsx
      agent-mode-toggle.tsx
      agent-plan-drawer.tsx
      agent-types.ts
      index.ts
    artifacts/
      artifact-modal.tsx
      artifact-preview-panel.tsx
      artifact-renderer.tsx
      artifact-viewer.tsx
      mini-artifact-card.tsx
    code/
      enhanced-code-block.tsx
    input/
      attachments-preview.tsx
      recording-indicator.tsx
      tool-belt-toolbar.tsx
      tool-belt.tsx
      tool-dropdown.tsx
      transcription-status.tsx
    layout/
      chat-action-buttons.tsx
      chat-conversation.tsx
      chat-modals.tsx
      chat-page.tsx
      chat-side-panel.tsx
      chat-splash-canvas.tsx
      chat-toolbelt-dock.tsx
      chat-top-controls.tsx
      unified-sidebar.tsx
    messages/
      chat-message-item.tsx
      chat-message-list.tsx
      message-renderer.tsx
      typing-indicator.tsx

---

6) File-by-file documentation (with “status”)

src/app/chat/page.tsx (✅ ACTIVE)

Role: /chat page entry. Uses dynamic import:

• dynamic(() => import("./_components/layout/chat-page"), { ssr:false })

Why: avoids SSR/hydration problems for browser-only APIs (media recorder, localStorage, SSE, etc).

---

src/app/chat/types.ts (✅ ACTIVE)

Role: Shared UI-level types.

Key exports:

• Attachment (UI attachment object)
• ModelOption (for model dropdown)
• ActivityItem, ActivityGroup, ThinkingState (for activity panel)

---

src/app/chat/utils/index.ts (✅ ACTIVE)

Active exports used in chat:

• stripThinkingForModelContext(text) → used in chat-page.tsx when building model context.
• tryParseNestedJsonString(raw) → used when loading stored tool calls in chat-page.tsx.

6.1 Hooks

hooks/use-chat-sessions.ts (✅ ACTIVE)

Role: Session list + current session metadata manager.

• Uses controller API: api.getChatSessions(), api.getChatSession(id).
• Stores sessions + selection in Zustand.
• Does not load messages into UI; chat-page.tsx calls setMessages() after fetching.

---

hooks/use-chat-tools.ts (✅ ACTIVE)

Role: MCP tool discovery + execution.

• loadMCPServers() → controller /mcp/servers
• loadMCPTools() → controller /mcp/tools (or per-server tools)
• getToolDefinitions() → formats tool names as ${server}__${tool}
• executeTool({ toolCallId, toolName, args }) → controller /mcp/tools/{server}/{tool}

Tracks ephemeral UI state:

• executingTools: Set<toolCallId>
• toolResultsMap: Map<toolCallId, ToolResult>

---

hooks/use-chat-derived.ts (✅ ACTIVE)

Role: Derived UI data:

• Thinking extraction (reasoning parts + <think> tags)
• Activity timeline groups (per assistant message)

Depends on:

• thinkingParser from src/lib/services/message-parsing (external)

---

hooks/use-chat-usage.ts (✅ ACTIVE)

Role: Fetch + store session usage.

Calls controller:

• api.getChatUsage(sessionId)

---

6.2 Layout

_components/layout/chat-page.tsx (✅ ACTIVE, very large)

Role: The orchestrator (“god component”).

Major responsibilities:

• Owns controller run stream lifecycle:
  • api.streamChatRun() for SSE
  • handleRunEvent() for message_*, tool_execution_*, plan_updated, run_end
• Loads sessions and restores messages (setMessages(mapStoredMessages(...))).
• Loads models list and stores in Zustand.
• Builds effectiveSystemPrompt (system + agent-mode block).
• Builds context stats + auto-compaction.
• Extracts artifacts from assistant messages.
• Renders full UI via UnifiedSidebar + ChatConversation + ToolBelt + modals.

Known architectural smells:

• Mixes UI rendering, persistence, model IO, compaction, artifact parsing, and sidebar behavior.