Skip to content
Start free

The private RAG backend for your product

DocsGPT is an open-source (MIT) RAG and agent backend you run in your own cloud behind your product: an OpenAI-compatible API, streaming, webhooks, white-label widgets, any model, per-domain benchmarks. Arc53 offers it as a subscription plus an engineering retainer; LexEU, a legal-tech platform, has run on it since 2023. The same team builds the platform and does the engineering, so where configuration is not enough, we change the platform.

Book a demoSelf-host in 5 minutes

Last updated:

Product architecture
  1. 01Your app / widgetReact · HTML · search bar
  2. 02DocsGPT API/v1/chat/completions · SSE · webhooks
  3. 03Ingestion & retrievalyour corpus · hybrid · GraphRAG
  4. 04Model of your choiceany provider · BYOM · fallback
  5. 05 · humanReview & benchmarksyour feedback loop · docsgpt-cli bench
Text version

Five stages: your application or an embedded widget calls the DocsGPT API (OpenAI-compatible, server-sent events, webhooks); DocsGPT ingests and retrieves over your corpus; the model you registered generates the answer; your team's review loop and the benchmark suite keep quality measurable on every release.

The situation

  • Where the value is

    A vertical AI product's value is in domain quality — citations, multilingual retrieval, evals — not in re-implementing ingestion, chunking, retrieval and authentication.

  • Customer demands

    Customers in legal, medical and government demand private cloud or on-premises deployment and no lock-in to a single model vendor.

  • Team cost

    A two-person retrieval team is expensive to run and slow to replace; the backend they built leaves with them.

What DocsGPT does here

Workflow 01

API surface

Each agent exposes OpenAI-compatible /v1/chat/completions and /v1/models under its own API key — a drop-in for OpenAI SDKs with streaming, JSON-Schema structured outputs, files and images, client-side tool calling and Idempotency-Key. The native /api/answer endpoint returns the answer and its sources as one JSON response; /stream emits answer, source, tool_calls and thought events over SSE and accepts attachments. Both pass variables into prompt templates and can run any agent by id with a personal access token. The MCP server at /mcp lets assistants such as Claude or Cursor search an agent's knowledge, and every instance serves its own Swagger reference.

/v1/chat/completions · /v1/models · /api/answer · /stream (SSE) · JSON Schema · Idempotency-Key · /mcp · Swagger reference

API, MCP & Widgets →

Workflow 02

Async and events

Per-agent webhooks with task polling and idempotency for long-running jobs; a realtime SSE channel for live updates; interrupted streams resume where they stopped.

Webhooks · task polling · idempotency · realtime SSE · stream resume

API, MCP & Widgets →

Workflow 03

White-label front ends

A React or plain-HTML chat widget on npm, themable to your product, with file attachments and voice input; a search-bar widget with voice search; a Nextra wrapper for documentation sites. All open source.

npm docsgpt 0.8 · React / HTML chat widget · attachments · voice · search widget · Nextra wrapper

API, MCP & Widgets →

Workflow 04

Any model, your evals

Connect any provider or OpenAI-compatible endpoint, choose the models each agent may use, and set fallbacks — the agent's other models, then an instance-wide fallback — with every response reporting which model answered. Override the model per workflow node. The default embeddings are multilingual; another local or remote embedding model can be swapped in. Run docsgpt-cli bench with YAML suites, a model matrix, JUnit output and baseline diffs in CI.

Any provider · BYOM · fallback (reported per response) · per-node model · multilingual embeddings · docsgpt-cli bench · --matrix · JUnit

Built for the client · fine-tuned legal embeddings, as for LexEU.

Models & BYOM →Benchmarks & Evals →CLI →

Workflow 05

Tenancy and control

Teams and roles; an API key per agent; personal access tokens with scopes, resource limits and an expiry; usage quotas per user, set directly or through a team allowance, to keep each tenant inside its budget; guardrails under an instance floor; execution traces exported as OpenTelemetry GenAI spans. Agents are code too: docsgpt-cli exports them as YAML and applies them from CI, so configurations travel between environments.

Teams & roles · agent API keys · access tokens · usage quotas · guardrail floor · OTel GenAI traces · agents as code (YAML, CI)

Admin, Security & Analytics →CLI →Guardrails & Human-in-the-loop →

Drop-in for OpenAI SDKs

Each agent has its own API key and answers on /v1/chat/completions. Existing client code changes a base URL.

# Your agent, OpenAI-compatible; point any OpenAI SDK at this base URL
curl -N https://your-docsgpt.example/v1/chat/completions \
  -H "Authorization: Bearer $AGENT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"docsgpt-agent","stream":true,
       "messages":[{"role":"user","content":"Which clause governs termination?"}]}'
# → SSE stream: answer tokens, then sources with title and page
# The agent behind the key sets the model; the response reports which one answered

Build vs buy, by layer. Your product keeps the domain corpus, prompts, fine-tuned embeddings, UI and the customer relationship; the rest comes from an MIT-licensed backend you can fork. Cost comparison: vs Building in-house.

Build vs buy: what a product team maintains in-house against what DocsGPT provides
 You maintain in-houseDocsGPT provides
IngestionParsers for 30+ formats, OCR for scans, connectors (Drive, SharePoint, Confluence, GitHub, S3)Ingestion pipeline, parser + OCR, connectors with scheduled sync
RetrievalChunking, embeddings, vector store, hybrid/graph retrievalFive chunking strategies, hybrid BM25 + vector, GraphRAG, 6 vector stores
Answers and APICitation plumbing, streaming, structured outputsCited answers, SSE streaming, OpenAI-compatible API with JSON Schema, MCP server, Swagger reference
AgentsAgent runtime, tools, scheduling, workflowsFour agent types, connectors and MCP/API/DB tools, schedules, visual workflows
Auth and tenancyAuth, SSO, tenancy, audit, quotasOIDC, SCIM, roles, teams, agent API keys, access tokens, exportable audit log, usage quotas
EvalsEval harness and regression testsdocsgpt-cli bench (assertions, LLM-judge, JUnit, baseline diffs)
OwnershipEverything above, forever, with a team that can leaveMIT code you can fork; Arc53 engineering retainer optional

Open source on GitHub: 18,317 stars, 239 contributors, MIT licence.

your-product.example
Ask the docsdocsgpt

How do I rotate an API key?

Open Settings › Agents, pick the agent and choose Regenerate key. The previous key stops working immediately and the new one is shown once, so update clients first 1.

→ Admin guide · API keys · p. 12

Models available to agents
models3 models · pick per chat or agent

  • Qwen3.8-Flash-Next

    Ollama · http://ollama:11434

    defaultlocal

  • GLM-5.3-Flash

    vLLM · http://gpu-01:8000/v1

    local

  • GPT-5.6 Sol

    DocsGPT · api key · via Arc53

    cloud

  • + Add model bring your own · openai-compatible

    base url https://…/v1api key ••••••••model id my-model
Issue triage: a GitHub webhook to a cited draft reply
  1. startNew issueGitHub → agent webhook
  2. ai agentSearch the docsagent's sources · citations
  3. toolRead the repoGitHub connector · read only
  4. endDraft replycited answer for a maintainer
Text version

1. New issue (GitHub calls the agent's webhook) → 2. Search the docs (the agent's sources, with citations) → 3. Read the repo (GitHub connector, read only) → 4. Draft reply (a cited answer a maintainer reviews before posting).

Evidence

Results from named deployments, with the source.

2.5M+ pages · 6 languages

Multilingual EU-law research over 2.5M+ pages, answers in 6 EU languages with full source citations, 100–200 ms retrieval.

Source: LexEU

6+ h → under 5 min

IP application drafting: 6+ hours to under 5 minutes per application, with lawyer review as the final step.

Source: Balt Alnoor

How it is deployed

Typical posture: Self-hosted or On-premises in your own cloud, with your own operators — or a Managed instance if you'd rather not operate it.

Product teams run DocsGPT themselves in their own cloud — Kubernetes manifests + kustomize or Docker Compose, PostgreSQL with pgvector, S3-compatible storage — which is the Self-hosted or On-premises posture with your own operators. Teams that would rather not operate it take a dedicated managed instance from Arc53.

Documentation

What a first project looks like, and the commercial model

Scope, measure, then decide.

  1. 01

    One-week integration spike

    Run DocsGPT in your cloud, point your client at the OpenAI-compatible endpoint, load one corpus.

    base_url → /v1/chat/completions

  2. 02

    Write the domain benchmark suite together

    Source and content assertions, LLM-judge rubrics, in YAML.

    docsgpt-cli bench

  3. 03

    Retrieval tuning and embedding choice

    Fine-tuning only if the benchmark says so.

  4. 04

    Production

    SSO, usage quotas per tenant, webhooks, widgets, traces into your observability stack, and agents as code in CI.

  5. 05

    Retainer

    Monthly engineering plus upgrades; the benchmark suite is re-run on every release.

Subscription plus an engineering retainer. LexEU, a legal-tech platform, has run on one since 2023. Self-hosting is free under MIT. See pricing and build vs buy.

Related

Frequently asked questions

Should I build or buy a RAG pipeline?

Build the parts that differentiate your product (corpus, prompts, domain evals, UI) and take the rest from an open-source backend you can fork. DocsGPT covers ingestion, retrieval, agents, auth and evals under the MIT licence, so “buy” here does not mean lock-in.

Is the API OpenAI-compatible?

Yes. Each agent exposes /v1/chat/completions and /v1/models with its own API key, including streaming, structured outputs and tool calling, so existing OpenAI SDK code points at DocsGPT with a base-URL change.

Can we use our own models and embeddings?

Yes. Connect any provider, or a local engine (vLLM, Ollama, llama.cpp and others) as an OpenAI-compatible server, override the model per workflow node, and swap the multilingual default embeddings for another local or remote model, including fine-tuned ones.

Does it work with MCP?

Yes. Each agent has an MCP server at /mcp, authorised with the agent's API key, that gives assistants such as Claude or Cursor a search tool over that agent's sources; the assistant's own model writes the reply.

How do we know retrieval quality holds after an upgrade?

Run your benchmark suite with docsgpt-cli bench in CI: assertions on cited sources, content and latency, JUnit output, and a diff against the last baseline before you deploy.

What does the commercial model look like?

Self-hosting is free under MIT. Arc53 offers a subscription plus an engineering retainer for teams that want the platform team on call; LexEU, a legal-tech platform, has run this way since 2023.

See DocsGPT on your documents

A 30-minute demo on your use case, or a free on-premises review with an engineer.

Book a demoSelf-host in 5 minutes