Available for HireFull-time · Contract · Remote
Live demo Self-hosted · Private · Production ready

Give your documents a memory.

A privacy-first RAG system that answers questions from your own documents, with citations back to the exact passage. Self-hosted Llama 3 via Ollama — no document, no chunk and no embedding ever leaves the server.

Try RAG live View architecture
  • Your data, your control
  • Open-source stack
  • Deploys with one compose file
Self-hosted

Summarise the key points from our product requirements.

Here are the key points from the document:

  1. Target users and core use cases
  2. Key features and requirements
  3. Technical architecture overview

Product-Requirements.pdf · p.4 · similarity 0.86

How it works

From your documents to an answer you can check.

  1. 1DocumentsPDF, DOCX or plain text, uploaded by you
  2. 2ProcessingExtraction, chunking, embedding
  3. 3RetrievalVector search for the passages that matter
  4. 4ContextThe prompt is built from what was retrieved
  5. 5AnswerGrounded response with its sources attached

The problem

Search finds documents. It does not answer questions. And the fastest way to get answers — paste the document into a public chatbot — is exactly what a regulated business cannot do.

  • Sensitive documents cannot be pasted into a public chatbot
  • Keyword search returns files, not answers
  • Answers with no source are answers you cannot act on
  • Generic models know nothing about your domain
Live demo

Try it in under two minutes

  1. 1 Open the demo account — one click, no form
  2. 2 Ask questions against the indexed sample corpus
  3. 3 Register to upload documents of your own

The demo account is read-only against pre-indexed documents, so a single visitor cannot exhaust the inference queue for everyone else. Uploading needs a verified account — email, one code, done.

Open the RAG demo

The solution

Retrieval-augmented generation running entirely on infrastructure you control, answering from your corpus and showing its sources.

  • Runs entirely on your own infrastructure
  • Answers grounded in your documents, with citations
  • Per-session data isolation and automatic purge
  • One docker compose file from clone to running

What I built

Four parts, each with a decision behind it.

Document ingestion

PDF and DOCX extraction, then ~220-word chunks with a 40-word overlap so a fact split across a boundary is still retrievable whole.

Retrieval

nomic-embed-text embeddings in Qdrant, top-K vector search, and the per-passage similarity score surfaced to the user rather than hidden.

Generation

Llama 3 through Ollama, prompted strictly from retrieved context, with a bounded inference queue so the VPS degrades gracefully instead of falling over.

User experience

Upload, ask, read the answer, click the citation. A non-technical visitor reaches a grounded answer without being taught what RAG means.

System architecture

Modular, production-shaped, and deployable as one compose file.

Client

Angular web app

API

NestJS
Auth guard
Inference queue

Retrieval

Chunking
Embeddings
Top-K search

Storage

Qdrant vectors
PostgreSQL metadata

Inference

Ollama
Llama 3

Response

Answer + citations

Key engineering decisions

Why self-hosted rather than a hosted vector database and a cloud LLM?
Because the use cases worth building this for — regulated finance, healthcare, government — are exactly the ones where "your data leaves our network" ends the conversation. Self-hosting is not a cost optimisation here; it is the product requirement, and designing for it changes the architecture rather than just the invoice.
Why 220-word chunks with a 40-word overlap?
A chunk this size is roughly 300 tokens, which keeps each embedding about one idea. Larger chunks dilute the vector across topics the question was not about and make similarity scores meaningless; smaller ones lose the surrounding sentence a citation needs to be readable out of context. The overlap is a fifth of a chunk — enough that a sentence straddling a cut point still appears whole somewhere.
How is one visitor’s data isolated from another’s?
Every session gets its own subject identifier, and every chunk, vector and document row is scoped to it. Two people using the demo at the same moment retrieve only their own passages. Demo-session data is purged on a TTL rather than accumulating on the disk of a single VPS.
How does it fail?
Inference concurrency is capped and the queue has a maximum depth. Past that depth requests are rejected immediately with "at capacity" rather than queued indefinitely — a fast honest failure beats a request that hangs for ninety seconds and then times out anyway. Qdrant being slow to start does not crash-loop the API; it retries lazily on first use.

What this demonstrates about me

Not “I know RAG”. That I can take an AI requirement from a user-facing question, through retrieval design and backend architecture, to a deployed system that is still running and still answering — and explain every decision along the way.

Build something similar with meNext system: MCP Agent →
Available for new opportunities

Have a system that needs an architect?

I'm open to full-time roles, contract engagements and product/architecture work. Everything on this site is a system I designed, built, deployed and still operate — the next one could be yours.

“Give me a problem. I'll architect it, build it, and ship it.”

— Deepak Kumar Jha