LLM Data Kit

LLM Data Kit: Architecture Overview

LLM Data Kit: Architecture Overview

This is the primary onboarding doc. It is written to let a new agent make safe refactors without needing to scan the whole codebase.

Stack (Source Of Truth)

  • Backend: Django (backend/) with Django Ninja OpenAPI (backend/config/api.py)
  • Auth: django-allauth + allauth.headless (backend/config/urls.py, frontend/hooks/use-auth.ts)
  • Frontend: Next.js App Router (frontend/app/)
  • API client: Orval-generated React Query hooks (frontend/generated/) using Axios mutator (frontend/lib/api-client.ts)
  • DB: PostgreSQL + pgvector (compose.yml, memory models under backend/apps/ai/models/)
  • Queue: Celery + Redis (backend/config/celery.py, compose.yml)
  • Payments: Stripe + dj-stripe (backend/apps/payments/, backend/config/settings.py)

How Requests Flow (End To End)

sequenceDiagram
  participant UI as Next.js UI (browser)
  participant Auth as allauth headless (/_allauth)
  participant API as Ninja API (/api/v1)
  participant MW as Tenant middleware
  participant DB as Postgres
  participant Stripe as Stripe

  UI->>Auth: POST /_allauth/browser/v1/auth/login (email/password)
  Auth-->>UI: session cookie + X-Session-Token (stored in localStorage)

  UI->>API: GET /api/v1/users/profile/ (cookies + X-Session-Token + CSRF)
  API->>MW: Resolve active tenant (session -> thread-local)
  MW->>DB: Tenant-scoped queries via `.objects`
  DB-->>API: Data
  API-->>UI: JSON (camelCase)

  UI->>API: POST /api/v1/billing/checkout-session/ (create embedded checkout)
  API->>Stripe: Create checkout session
  Stripe-->>API: session id + client_secret
  API-->>UI: Checkout session details

URL Layout (Two API Surfaces)

There are two API surfaces the frontend talks to:

  1. Business API (OpenAPI, Orval): /api/v1/...
    • Implemented by Django Ninja in backend/config/api.py.
    • OpenAPI schema at /api/v1/openapi.json (used to generate frontend/generated/).
  2. Authentication API (allauth headless): /_allauth/browser/v1/...
    • Implemented by allauth.headless (mounted in backend/config/urls.py).
    • Used manually by frontend/hooks/use-auth.ts via authClient (frontend/lib/api-client.ts).

There is also a small public Ninja endpoint at /api/v1/auth/enabled-providers/ (backend/apps/accounts/api/__init__.py) which the login UI uses to decide which social buttons to show.

Backend Mental Model

Start points:

  • Settings: backend/config/settings.py
  • URL routing: backend/config/urls.py
  • Ninja API (routers + auth): backend/config/api.py
  • Tenant context:
    • thread-local store: backend/apps/tenants/utils.py
    • request enforcement: backend/apps/tenants/middleware.py
    • tenant-aware managers: backend/apps/tenants/models/mixins.py
  • Billing/Stripe:
    • endpoints: backend/apps/payments/api/billing.py
    • plan definitions: backend/apps/payments/constants.py
    • entitlement logic: backend/apps/payments/models/mixins.py

What to change:

  • Add a new API surface: add a router and mount it in backend/config/api.py.
  • Change JSON casing behavior: backend/config/api.py (parser/renderer) and schema configs.
  • Change tenant enforcement rules: backend/apps/tenants/middleware.py.

Frontend Mental Model

Start points:

  • Next.js routes: frontend/app/
  • API client glue:
    • Orval hooks: frontend/generated/ (generated)
    • Axios instances and interceptors: frontend/lib/api-client.ts
  • Auth state machine: frontend/hooks/use-auth.ts
  • Billing UI: frontend/app/(app)/billing/ and frontend/generated/api/billing/

What to change:

  • Change how the app authenticates: frontend/hooks/use-auth.ts and frontend/lib/api-client.ts.
  • Add a new backend endpoint usage: regenerate Orval (make create_spec) then use the new hook from frontend/generated/.

Docs Map

Backend:

  • docs/backend/index.md
  • docs/backend/settings.md
  • docs/backend/api.md
  • docs/backend/auth.md
  • docs/backend/multitenancy.md
  • docs/backend/payments.md
  • docs/backend/tasks.md
  • docs/backend/database.md

Frontend:

  • docs/frontend/index.md
  • docs/frontend/overview.md
  • docs/frontend/api.md
  • docs/frontend/auth.md
  • docs/frontend/payments.md
  • docs/frontend/testing.md

Ops:

  • docs/development.md
  • DEPLOYMENT.md
  • SEED.md
  • docs/memory-system.md

Last updated June 21, 2026