← Back to home

Developer guide

LiteMindUI is a local-first AI workspace supporting chat, retrieval-augmented generation (RAG), web search, and realtime voice. It is composed of a FastAPI backend and a Next.js frontend that talk to each other exclusively over HTTP. They share no code imports.

Directory layout

The backend lives in app/; the UI in nextjs-frontend/src/.

Key design patterns

LiteLLM Gateway (app/services/llm_gateway.py)

A unified transport for ollama, openrouter, and nvidia_nim. For Ollama it calls the native ollama Python client directly (bypassing LiteLLM streaming) to avoid a known upstream bug. resolve_backend_config() normalises provider names, API bases, and keys from request params or environment variables.

Pluggable Skill Layer (app/skills/)

Chat and RAG requests route through ChatSkillRegistry / RAGSkillRegistry. Each skill implements supports(), validate(), and stream(), so you can add new capabilities without touching the API routes.

RAG system (app/services/rag_service.py)

ChromaDB vector store plus BM25 keyword retrieval for hybrid search. Configurable embedding providers (sentence-transformers, Ollama, OpenRouter, Nvidia NIM). Pipeline: format detection → extraction → chunking → embedding → indexing.

Conversation memory (app/services/conversation_memory.py)

Session-based multi-turn context. Older messages are auto-summarised once token usage passes 75% of the 24 K context limit. Persisted in SQLite (conversation_db.py).

Generative UI (app/backend/api/chat.py)

When enable_generative_ui is set, the LLM emits ```ui:component_name fenced blocks. The Next.js frontend renders these as charts, tables, metrics, progress bars, and iframe apps.

Realtime voice mode (voice.py, voice_pipeline.py)

Browser and server establish a WebRTC peer connection; the browser POSTs an SDP offer to POST /api/voice/offer. A Pipecat pipeline runs in the background, with Whisper STT + Kokoro TTS + LLM. Transcripts and control events flow back over the WebRTC data channel. Voice is a separate pipeline, not a Skill. Do not route it through the skill layer.

Authentication system (app/backend/api/auth*)

LiteMindUI uses a hybrid authentication system powered by GoTrue (Supabase Auth). On login/register, the backend sets an HTTP-only access_token cookie for browsers and returns the JWT in the response body for CLI/desktop clients. Protected routes validate the token via cookie or Authorization header. Every chat session, conversation, and RAG context is namespaced by the authenticated user's ID, ensuring per-user data isolation.

How a request travels

A chat message is a good example of the moving parts. The browser POSTs to POST /api/chat, and chat.py picks a skill from the ChatSkillRegistry based on the request. The skill calls the llm_gateway to reach a model, streams tokens back to the route, and the route forwards them to the browser over Server-Sent Events. RAG and web search follow the same shape, with an extra retrieval step before the model is called.

Local development tips

• Hot reload. The backend runs with uvicorn --reload and the frontend with npm run dev; both rebuild on save.
• Start with a small local model. gemma3:1b through Ollama keeps everything on your machine while you build.
• Read the logs. Set LOG_LEVEL=DEBUG to see request routing, skill selection, and RAG retrieval scores.
• Containers are optional. make dev gives you the same stack with hot-reload if you would rather not manage Python and Node versions yourself.

Extending LiteMindUI

The skill layer is the main extension point for chat and RAG functionality. To add a capability, write a class that implements supports(), validate(), and stream(), then register it with the relevant registry. The API routes never change, so new features stay isolated and testable. Voice and authentication are intentional exceptions: voice runs as its own Pipecat pipeline, while authentication is implemented via dedicated API routes with dependency-based route protection.

LLM provider backends

Configuration & environment

Copy .env.example → .env and fill in secrets. Critical variables:

Document ingestion

Supported formats: PDF (PyMuPDF + pdfplumber + Camelot tables), DOCX, PPTX, XLSX, EPUB, RTF, ODF, HTML, CSV, images (EasyOCR fallback), and plain text.

Useful commands

# Python / backend
uv sync --group all              # install all dependency groups
uv run uvicorn main:app --reload # start backend on :8000
uv run pytest                    # run tests
uv run ruff check .              # lint
uv run ty check app              # type-check
Next.js frontend
cd nextjs-frontend
npm install
npm run dev                      # dev server on :3000
npm run build                    # production build
npm run lint                     # eslint
Docker (primary workflow)
make up                          # build & run the stack
make dev / make prod             # dev (hot-reload) / production
make logs / make health          # tail logs / health check

Quality & CI

Pull requests run pr-checks.yml (Python compile, ruff, ty); Docker images build via docker-publish.yml; releases bump version.json and tag via release.yml. PRs are labelled patch (default), minor, or major.

← Back to home