AI platform and self-hosting specification

Licensing prerequisite#

The repository currently has no explicit LICENSE file. Claire must not describe the downloadable edition as “open source” until a license is selected and added. Until then, use “self-hostable” or “source available.” License selection is a business decision outside this technical specification.

Claire-managed AI#

Claire owns the provider account and pays the upstream bill. A plan includes a monthly AI allowance.

Data flow:

Client → Claire API → Claire context builder → Claire-selected model provider
                                      ↘ usage ledger + redacted operational telemetry

Properties:

• No API-key setup for the customer.
• Claire selects task-appropriate models behind stable product tiers such as fast, balanced, and best.
• Claire can change the underlying model without changing user-visible settings, but only after evals show equal or better quality.
• Usage, rate limits, fallbacks, and provider outages are Claire’s responsibility.
• The UI identifies that selected conversation content is sent to Claire’s configured AI provider.

Cloud BYOK#

Claire hosts the product, but the customer supplies a provider key and pays that provider directly.

Data flow:

Client → Claire API → Claire context builder → provider using customer credential
                          ↘ metadata-only usage record

Properties:

• Message content still passes through Claire Cloud because the cloud server builds the context.
• The provider credential is stored in an encrypted secret store, never an ordinary application table.
• Claire does not add model usage to the customer’s managed-AI allowance.
• Claire still charges for managed infrastructure.
• Provider usage and retention terms belong to the customer’s provider account.
• Disconnecting a provider deletes its secret and disables dependent task profiles.

“Use my own key” must never be described as “local” or “Claire cannot see the content.”

Self-hosted BYOK#

The user’s Claire server calls the selected provider directly using environment-backed or secret-store credentials on the user’s host.

Properties:

• Claire Cloud receives no AI request or credential.
• The external model provider still receives the prompt and selected message context.
• Product telemetry is off by default in Community deployments.
• The setup wizard validates the provider without transmitting the key to Claire.

Local model#

The Claire server calls a model runtime on the same host or trusted LAN, normally through an OpenAI-compatible endpoint.

Initial supported runtimes:

• Ollama
• LM Studio
• Any explicitly configured OpenAI-compatible endpoint in a self-hosted deployment

Later supported runtimes may include vLLM, llama.cpp servers, and managed private endpoints.

Important constraints:

• A Claire Cloud process cannot call localhost on a customer’s computer.
• Cloud accounts that want a desktop-resident model require a supervised desktop execution broker. That broker is a separate project and is not needed for Community self-hosting.
• Local model quality and feature support vary. Claire must capability-test structured output, context length, embeddings, tool calling, and streaming instead of assuming compatibility.
• The desktop or self-hosted machine must remain online for AI features to work.

Settings structure#

Add Settings → AI & models with three primary choices:

1. Claire AI — recommended
2. “Included with your Claire Cloud plan.”
3. Shows allowance remaining, renewal date, and current quality tier.
4. Use my provider account
5. OpenAI, Anthropic, Amazon Bedrock, Google, xAI, Groq, Mistral, and advanced OpenAI-compatible setup.
6. Shows who bills the usage and where the key is stored.
7. Run models locally
8. Ollama, LM Studio, or custom compatible endpoint.
9. Shows which device executes AI and whether that device is currently reachable.

Each provider setup screen contains:

• Provider name and link to create a key.
• Secret field that can be submitted but never read back.
• Optional organization, project, region, and base URL fields when applicable.
• “Test connection” action.
• Models discovered from the provider when supported, plus an advanced manual model-ID field.
• A plain-language data-flow disclosure specific to the selected mode.
• Key fingerprint, creation time, last successful check, last error category, and revoke action.

Simple and advanced model selection#

Most users should not configure five model dropdowns. The default UI offers:

• Fast — prioritizes latency and low usage.
• Balanced — default for reply suggestions and Ask Claire.
• Best quality — higher cost and latency, with an explicit allowance impact.

An advanced panel maps individual AI jobs:

A provider connection can be valid for some roles and invalid for others. For example, an Anthropic connection can generate text but does not by itself provide an embedding model; the user must choose a separate embedding provider or use local lexical search.

Usage UI#

Managed AI displays:

• Allowance used and remaining.
• Usage grouped by feature, not just raw tokens.
• Optional daily warning and monthly hard cap.
• Whether a request used fast, balanced, or best.

BYOK displays:

• Request and token counts reported by the provider.
• A clear statement that charges appear in the provider account.
• Cost estimates only when Claire has a current price entry; otherwise show usage without invented currency estimates.

Local mode displays:

• Requests, latency, and model runtime health.
• No dollar estimate unless the user adds an optional local-cost profile.

Boundary#

Only files under server/src/services/ai/ may import Vercel AI SDK or provider packages. Product services depend on a Claire-owned interface so a future SDK change does not touch every feature.

Reply / Ask Claire / Voice profile / Memory / Loop detector
                              │
                       ClaireAIService
                              │
             ┌────────────────┼────────────────┐
             │                │                │
       Task router      Policy middleware   Usage meter
             │                │                │
             └──────── Provider registry ─────┘
                              │
     OpenAI | Anthropic | Bedrock | Google | OpenAI-compatible | Local

Proposed interface:

type AITaskRole = 'reply' | 'assistant' | 'analysis' | 'classification' | 'embedding';

interface AIExecutionContext {
  userId: string;
  task: AITaskRole;
  hostingMode: 'claire_cloud' | 'self_hosted' | 'desktop_local';
  providerConnectionId?: string;
  modelProfileId: string;
  requestId: string;
  abortSignal?: AbortSignal;
}

interface ClaireAIService {
  generateObject<T>(context: AIExecutionContext, input: StructuredAIInput<T>): Promise<AIResult<T>>;
  streamText(context: AIExecutionContext, input: TextAIInput): Promise<AITextStream>;
  embedMany(context: AIExecutionContext, values: string[]): Promise<AIEmbeddingResult>;
}

Every result normalizes:

• Provider and resolved model ID.
• Provider request ID when available.
• Input, cached-input, and output token usage.
• Finish reason and warnings.
• Latency and retry count.
• Whether a fallback was used.
• Estimated managed cost and price-book version.

Prompt text, message content, and model output are not included in operational logs or usage events.

Provider registry#

Use AI SDK createProviderRegistry with stable Claire connection IDs rather than global singleton providers. Provider instances are created per encrypted credential reference and cached briefly in memory.

Initial provider packages:

• ai
• @ai-sdk/openai
• @ai-sdk/anthropic
• @ai-sdk/amazon-bedrock
• @ai-sdk/openai-compatible

Add Google and other dedicated provider packages only when a product setup screen and contract tests exist. Kimi can initially use the OpenAI-compatible provider. Prefer Anthropic’s dedicated provider over Anthropic’s OpenAI compatibility endpoint because Anthropic documents that compatibility as a testing path with feature limitations.

Provider-neutral structured output#

Reply suggestions and analysis features currently depend on hand-parsed JSON. Migrate them to a schema-backed structured-output call and validate the result again at the Claire boundary.

Rules:

• Every structured task owns a Zod schema.
• A model must pass a setup-time structured-output smoke test before being selectable for that role.
• No provider-specific response_format appears in product services.
• Invalid output receives at most one repair attempt; repeated invalid output marks the model-role pair unhealthy.
• Fallbacks are allowed only between explicitly enabled model profiles with equivalent privacy and billing modes.

Never fall back from BYOK or local execution to Claire-managed AI without explicit user consent.

Middleware#

Wrap every language model with Claire middleware for:

1. Policy and task-role validation.
2. Context-size and output-token budgets.
3. Secret and PII-safe error normalization.
4. Retry and provider circuit breaking.
5. Usage measurement and allowance settlement.
6. Content-free tracing.

If AI SDK OpenTelemetry is enabled, set recordInputs: false and recordOutputs: false. Message content, embeddings, tool arguments, and outputs must not be copied into traces.

ai_provider_connections#

• id
• user_id
• workspace_id nullable
• provider_type
• execution_mode: managed | cloud_byok | self_hosted | desktop_local
• display_name
• secret_reference nullable
• base_url nullable
• provider_account_metadata JSONB containing non-secret organization/project/region fields
• credential_fingerprint nullable
• status: pending | active | invalid | rate_limited | revoked | unreachable
• capability_snapshot JSONB
• last_checked_at, last_error_code
• timestamps

ai_model_profiles#

• id, user_id, workspace_id
• provider_connection_id
• task_role
• model_id
• quality_tier
• capabilities JSONB
• context_window nullable
• max_output_tokens nullable
• embedding_dimensions nullable
• is_default, status
• timestamps

ai_usage_events#

• id, request_id, user_id, workspace_id
• provider_connection_id, model_profile_id
• task_role, feature
• billing_source: managed_allowance | managed_overage | byok | local
• input_tokens, cached_input_tokens, output_tokens
• provider_cost_micro_usd nullable
• billable_cost_micro_usd nullable
• price_book_version nullable
• latency_ms, retry_count, status, error_code
• provider_request_id nullable
• timestamp

No prompt, response, message excerpt, embedding, API key, or raw provider error body belongs in this table.

ai_allowance_accounts and ai_allowance_entries#

Use an append-only credit ledger with reservations, settlements, top-ups, refunds, and expirations. Do not maintain a mutable balance without the ledger that explains it.

embedding_profiles#

Store the immutable profile described above and add embedding_profile_id to each message embedding.

Cloud#

• Put provider keys in a managed secret store with envelope encryption and KMS-backed rotation.
• Ordinary database rows store only an opaque secret reference and fingerprint.
• Decrypt immediately before provider-client construction; do not persist plaintext in caches.
• Never return a stored key through an API.
• Redact authorization headers and known secret patterns before logging.
• Revoke deletes the secret first, then marks the connection revoked.
• Admin tooling may report that a key exists but may never reveal it.

Desktop and self-hosted#

• Desktop-held keys use macOS Keychain or Windows Credential Manager.
• Self-hosted server keys use environment variables, Docker secrets, or a user-configured secret store.
• Keys never enter React state, AsyncStorage, analytics, crash reports, or ordinary Supabase tables.

Custom endpoint rules#

Arbitrary base URLs create server-side request forgery risk.

• Claire Cloud v1 supports only curated provider endpoints.
• Advanced custom URLs are permitted in self-hosted mode because the operator controls the network.
• A later Cloud enterprise feature must require HTTPS, resolve and validate DNS on every connection, block loopback/private/link-local/metadata ranges, use an egress proxy, cap response sizes, and prevent redirect-based bypasses.

Migration rule#

No product service may construct a provider SDK client after the migration. All provider calls go through ClaireAIService.

Proposed file structure#

server/src/services/ai/
  index.ts
  types.ts
  definitions.ts
  registry.ts
  execution-service.ts
  task-router.ts
  policy-middleware.ts
  usage-meter.ts
  price-book.ts
  secret-store.ts
  errors.ts
  providers/
    openai.ts
    anthropic.ts
    bedrock.ts
    openai-compatible.ts
  schemas/
    reply.ts
    assistant-answer.ts
    conversation-analysis.ts
    contact-memory.ts

Phase 0 — product and security decisions#

• Select and add the repository license.
• Define Community versus Cloud entitlements without exact model names in marketing.
• Select the managed cloud secret store and KMS.
• Decide whether the initial billing UI shows dollars or branded AI credits backed by dollars.
• Define provider subprocessors and update the privacy policy.

Phase 1 — provider-neutral core#

• Add AI SDK Core and the OpenAI, Anthropic, Bedrock, and OpenAI-compatible providers.
• Implement ClaireAIService, task roles, schemas, normalized errors, and mock-model tests.
• Migrate ai-processor.ts, conversation assistant answers, voice profiles, smart cards, and contact insight extraction.
• Preserve the existing process-wide environment configuration as the default compatibility path.
• Add usage events without billing.

Phase 2 — Community BYOK and local models#

• Add configuration-file and environment-based provider definitions.
• Support Ollama and LM Studio through OpenAI-compatible endpoints.
• Add provider/model health checks and capability tests.
• Add embedding profiles and resumable reindexing.
• Document outbound data paths in the self-hosting guide.

Phase 3 — Claire Cloud managed AI#

• Add managed provider connections, allowance ledger, price book, budgets, and per-feature usage.
• Add subscription entitlements and opt-in overage.
• Add provider circuit breakers, quality eval gates, and equivalent-mode fallback.
• Ship the simple fast / balanced / best settings UI.

Phase 4 — Cloud BYOK#

• Add encrypted secret storage, credential rotation, provider setup UI, and deletion audit.
• Support OpenAI and Anthropic first; Bedrock follows after region and IAM UX is designed.
• Add model discovery where reliable and manual model IDs in advanced settings.
• Run penetration tests focused on secrets, logs, SSRF, and cross-tenant isolation.

Phase 5 — supervised desktop-local execution#

• Define an authenticated, replay-resistant AI job protocol for a companion device.
• Keep credentials and inference on the selected desktop.
• Make device availability and data flow visible on every client.
• Complete egress auditing before making desktop-only privacy guarantees.

Provider behavior#

• The same reply, explanation, assistant-answer, and memory schemas pass against every supported generation provider.
• Unsupported capabilities prevent model selection before a real user request.
• Provider fallback never crosses managed, BYOK, or local billing/privacy boundaries silently.
• Cancellation, timeouts, rate limits, invalid keys, quota exhaustion, and malformed output map to stable Claire error codes.

Security#

• No credential appears in React state, AsyncStorage, ordinary tables, logs, analytics, traces, support exports, or crash reports.
• Cloud secrets can be rotated and deleted without deployment.
• Cross-user connection IDs return 404 and cannot influence routing.
• Custom endpoint SSRF tests cover DNS rebinding, redirects, IPv4/IPv6 private ranges, and cloud metadata addresses before custom URLs are allowed in Claire Cloud.

Billing#

• Managed usage reserves and settles atomically and is idempotent by request_id.
• BYOK and local requests never consume managed allowance.
• Exceeding an AI cap disables only AI features, not messaging.
• Provider invoice totals can be reconciled to ledger totals by provider, model, and day.

Search and embeddings#

• Vectors from different embedding profiles are never queried together.
• A model change performs a resumable parallel reindex with rollback.
• Keyword search remains available when semantic indexing is disabled or incomplete.
• Deleting messages deletes their embeddings and derived index rows.

Privacy#

• Data-flow disclosures match observed network traffic for each execution mode.
• AI SDK telemetry records no prompts, outputs, embeddings, or tool payloads.
• Managed, BYOK, self-hosted, and local modes pass automated egress-policy tests.