Claire on-device intelligence specification

Product outcome and boundary#

The user experiences one Claire. Whether an answer was generated on the phone or by managed AI is an execution detail, except where it changes privacy, availability, cost, or quality enough that the user should know.

On-device Claire means message retrieval, ranking, prompt construction, generation, and citation selection can complete without sending message text to a model provider. It does not mean the entire Claire product is serverless. WhatsApp, Telegram, Instagram, and other bridges still need network services; cross-device synchronization needs a canonical source; and external actions still call the relevant API.

Why the current architecture can support it#

Claire already has most of the boundaries a local lane needs. Mobile uses encrypted SQLite, caches recent timelines, can optionally backfill full history, stores contacts and conversation settings, and preserves stable message IDs. Ask Claire v2 already separates deterministic query planning, retrieval, model generation, citations, persistence, and stream transport. Loops already separate their free gate, model extraction, deterministic relevance, reconciliation, and storage.

Target architecture#

The policy router makes the choice before retrieval. It must never begin locally, discover after generation that required evidence was missing, and silently produce a weaker answer. Index readiness and scope coverage are explicit inputs to routing.

type AIExecutionLane = 'on_device' | 'managed' | 'deterministic_only';

interface DeviceAICapabilities {
  generation: 'ready' | 'downloadable' | 'unsupported' | 'busy';
  modelId: string | null;
  modelRevision: string | null;
  supportedLanguages: string[];
  contextTokens: number | null;
  indexState: 'empty' | 'building' | 'partial' | 'ready' | 'stale';
  indexedThroughCursor: number;
  fullHistory: boolean;
}

interface ExecutionDecision {
  lane: AIExecutionLane;
  reason:
    | 'local_ready'
    | 'model_unavailable'
    | 'index_incomplete'
    | 'language_unsupported'
    | 'context_too_large'
    | 'local_busy'
    | 'quality_escalation'
    | 'offline_search_only';
}

On-device model runtime#

Use Expo AI Kit behind a Claire-owned adapter. Its built-in route uses Apple Foundation Models on eligible iOS devices and ML Kit on eligible Android devices. It can also run downloadable LiteRT-LM models and exposes an AI SDK provider. Claire must not import the package throughout feature code or couple persisted records to its native types.

Plain-text answers may stream token by token. Structured output and tool calls can be buffered until validation succeeds. Only one text generation may run at a time on a device, so Ask Claire, Loop extraction, summaries, and drafts share a single priority queue. A user-visible request outranks background Loop processing; embeddings may proceed independently when the native backend supports that safely.

Private local search index#

Local generation is only useful when retrieval is trustworthy. Claire must search the local replica rather than placing an entire message archive in the model context. The index combines exact search, semantic search, time and conversation filters, and deterministic personal-memory tables.

The existing encrypted SQLite key protects these tables at rest. Local indexes are derived data with the same sensitivity as the underlying messages: clearing local data, signing out, deleting a message, or disabling full history must remove or trim matching index rows.

Ask Claire on-device flow#

A cloud fallback is a new request lane, not a hidden continuation. Before message evidence leaves the device for managed inference, apply the user's AI privacy setting and show a compact disclosure when required. Do not upload the full local index; send only the bounded evidence package selected for that answer.

The existing Ask Claire UI, stream parts, answer cards, citations, Stop behavior, action proposals, and thread history remain unchanged. The result adds execution metadata: lane, model identity, index coverage, fallback reason, and measured latency. That metadata is diagnostic and privacy-visible but does not clutter the normal conversation.

Can Loop generation run locally?#

“Loop generation” is three distinct workloads. They should not move as one switch.

interface LoopDetectionLease {
  leaseId: string;
  userId: string;
  chatId: string;
  fromCursor: string | null;
  throughCursor: string;
  owner: 'device' | 'server';
  expiresAt: string;
}

interface ProposedLoopBatch {
  requestId: string;       // idempotency key
  leaseId: string;
  operations: LoopOp[];    // create, update, close
  evidenceMessageIds: string[];
  model: { id: string; revision: string | null };
}

Local intelligence and the future action system#

Local models fit the action system when they are planners, not authorities. Ask Claire or a Loop may propose a typed action. Claire then validates the proposal, displays the exact destination and fields, obtains approval, and hands it to a trusted native or plugin executor.

Present only the tools relevant to the current task. Small on-device models should never choose from the entire installed plugin catalog. A deterministic capability router selects a small read/propose tool set; write credentials and execution functions stay outside the model runtime. See the Plugin system specification.

Privacy, security, and data lifecycle#

Performance, battery, storage, and cost budgets#

On-device inference removes per-token model charges; it does not make computation free. Claire pays with application complexity, test coverage, support, and synchronization, while the user pays with storage, battery, memory, and heat. Rollout gates must measure all four.

These are candidate launch targets, not measurements. The reference-device matrix and current managed baseline must be captured before they become release gates.

Evaluation and release gates#

A local model ships by task, device class, OS version, model revision, and language—not because a single demo looked good. Apple may update the built-in model with the OS, so the same eval suite runs again for every supported revision.

Start with anonymized fixtures and the existing synthetic Loop corpus, then run the Lucas two-device flow after the action is explicitly approved under its testing specification. Local and managed candidates receive identical evidence packages so generation quality can be compared independently from retrieval quality.

Delivery plan#

1. Spike 0 · Reference benchmark

   Prove the model before changing product architecture

   Integrate Expo AI Kit in a development-only screen, run saved Ask Claire evidence and Loop windows, and capture quality, latency, memory, heat, and cancellation on real devices.

2. PR 1 · Runtime boundary

   Add the on-device provider and capability router

   Create a Claire-owned client runtime, model availability UI, one-generation queue, typed errors, and managed fallback without changing answers yet.

3. PR 2 · Local generation pilot

   Generate from existing server-selected evidence

   Keep retrieval unchanged, send a bounded evidence package to the phone, stream locally, validate citations, and compare against the managed answer path.

4. PR 3 · Lexical index

   Make exact and time-bounded search local

   Add encrypted index state, content hashes, deletion propagation, local query planning, rank fusion, and partial-history disclosures.

5. PR 4 · Semantic and memory index

   Add local embeddings and deterministic personal memory

   Persist model-versioned vectors, multilingual namespaces, relationship metrics, plan facts, incremental indexing, and resumable rebuilds.

6. PR 5 · Ask Claire local-first

   Route eligible golden queries entirely on-device

   Enable local retrieval plus generation by cohort, retain explicit escalation, and expose trustworthy privacy and freshness states.

7. PR 6 · Loop assistant local

   Move user-initiated Loop questions and drafts

   Use a small read/propose tool set over the local Loop replica with no execution authority.

8. PR 7 · Leased Loop detection

   Pilot local extraction with server guarantees

   Add cursor-range leases, shared deterministic guards, idempotent operation batches, server revalidation, and expiry takeover.

Decisions to resolve in the benchmark#

External implementation references#

• Expo AI Kit platform support and fallback requirements
• Built-in and downloadable model lifecycle
• Local embeddings, RAG, model identity, and language namespaces
• Expo AI Kit provider for Vercel AI SDK
• Tool calling, bounded repair, and human approval
• Apple Foundation Models availability and fallback guidance