{"post":{"slug":"how-to-build-a-rag-app-2026","title":"How to build a RAG app in 2026: the complete practical guide","excerpt":"Retrieval-augmented generation is the most shipped AI pattern in 2026. This guide walks through every component with working code and honest trade-offs.","category":"code","date":"April 30, 2026","author":"ByteLedger Team","content":"<span class=\"dropcap\">R</span>etrieval-augmented generation (RAG) is the most commonly shipped AI pattern in 2026. It is how you give an LLM access to your data without fine-tuning, and how you ground answers in sources you can verify. The pipeline is straightforward, but the quality lives in the details. This guide covers every step with working Python code and honest guidance on where to invest your effort.\n\n## What changed in 2026\n\n- **Hybrid search is now the standard** — vector-only RAG produces worse results than hybrid dense+sparse search. All production stacks use both.\n- **Reranking became table stakes** — a cross-encoder reranker on the top-20 retrieved chunks before generation significantly improves precision. Worth the ~100ms latency.\n- **Chunking research matured** — the community ran large-scale benchmarks. 500–800 token overlapping chunks with 10–15% overlap are the current best-practice starting point.\n- **Structured metadata filtering is standard** — storing source, date, section, and tenant ID as vector metadata and filtering at query time produces far better precision.\n\n## The RAG pipeline\n\n```\nDocuments → Chunk → Embed → Store (vector DB)\n                              ↓\nQuery → Embed query → Retrieve top-k → Rerank → Generate\n```\n\nEach step has quality decisions that compound.\n\n## Step 1: Chunking\n\n```python\nfrom langchain.text_splitter import RecursiveCharacterTextSplitter\n\nsplitter = RecursiveCharacterTextSplitter(\n    chunk_size=700,        # tokens\n    chunk_overlap=70,      # ~10% overlap preserves context at boundaries\n    length_function=len,\n    separators=[\"\\n\\n\", \"\\n\", \". \", \" \", \"\"]\n)\n\nchunks = splitter.split_text(document_text)\n```\n\n**Rules of thumb:**\n- 500–800 tokens per chunk for most use cases\n- 10–15% overlap to avoid cutting sentences at boundaries\n- Chunk on semantic boundaries (paragraph > sentence > word)\n- Store the source document ID and section with each chunk as metadata\n\n## Step 2: Embedding\n\n```python\nfrom anthropic import Anthropic  # or openai, cohere, etc.\nimport openai\n\nclient = openai.OpenAI()\n\ndef embed_chunks(chunks: list[str]) -> list[list[float]]:\n    response = client.embeddings.create(\n        model=\"text-embedding-3-small\",  # 1536 dims, good price/quality\n        input=chunks,\n    )\n    return [item.embedding for item in response.data]\n```\n\nUse the same model for indexing and querying. Mixing models produces garbage similarity scores.\n\n## Step 3: Storing in pgvector\n\n```python\nimport psycopg2\nimport json\n\nconn = psycopg2.connect(DATABASE_URL)\ncur = conn.cursor()\n\ncur.execute(\"\"\"\nCREATE TABLE IF NOT EXISTS chunks (\n  id        BIGSERIAL PRIMARY KEY,\n  doc_id    TEXT NOT NULL,\n  content   TEXT NOT NULL,\n  metadata  JSONB,\n  embedding vector(1536)\n)\n\"\"\")\n\ncur.execute(\"CREATE INDEX IF NOT EXISTS chunks_embedding_idx ON chunks USING hnsw (embedding vector_cosine_ops)\")\n\nfor chunk, embedding, meta in zip(chunks, embeddings, metadatas):\n    cur.execute(\n        \"INSERT INTO chunks (doc_id, content, metadata, embedding) VALUES (%s, %s, %s, %s)\",\n        (meta[\"doc_id\"], chunk, json.dumps(meta), embedding)\n    )\nconn.commit()\n```\n\n## Step 4: Retrieval with hybrid search\n\n```python\n# Dense vector retrieval\ndef dense_retrieve(query_embedding, top_k=20):\n    cur.execute(\"\"\"\n        SELECT id, content, metadata,\n               1 - (embedding <=> %s::vector) AS score\n        FROM   chunks\n        ORDER  BY embedding <=> %s::vector\n        LIMIT  %s\n    \"\"\", (query_embedding, query_embedding, top_k))\n    return cur.fetchall()\n\n# Add BM25 keyword search for hybrid\n# Use pg_trgm for simple hybrid, or a dedicated search layer\n```\n\n## Step 5: Reranking\n\n```python\nfrom sentence_transformers import CrossEncoder\n\nreranker = CrossEncoder(\"cross-encoder/ms-marco-MiniLM-L-6-v2\")\n\ndef rerank(query: str, candidates: list[dict], top_k: int = 5) -> list[dict]:\n    pairs = [(query, c[\"content\"]) for c in candidates]\n    scores = reranker.predict(pairs)\n    ranked = sorted(zip(scores, candidates), reverse=True)\n    return [c for _, c in ranked[:top_k]]\n```\n\n## Step 6: Generation\n\n```python\nimport anthropic\n\nclient = anthropic.Anthropic()\n\ndef generate(query: str, context_chunks: list[str]) -> str:\n    context = \"\\n\\n---\\n\\n\".join(context_chunks)\n    message = client.messages.create(\n        model=\"claude-sonnet-4-5\",\n        max_tokens=1024,\n        system=\"Answer the question using only the provided context. If the answer is not in the context, say so.\",\n        messages=[{\n            \"role\": \"user\",\n            \"content\": f\"Context:\\n{context}\\n\\nQuestion: {query}\"\n        }]\n    )\n    return message.content[0].text\n```\n\n## How to pick the stack\n\n1. **Under 10M vectors, running Postgres?** pgvector — no extra service needed.\n2. **Need managed fully hosted?** Supabase (Postgres + pgvector) or Pinecone.\n3. **100M+ vectors or complex filtering?** Qdrant managed.\n4. **Python team?** FastAPI + pgvector + Anthropic SDK.\n5. **TypeScript team?** Hono/Next.js + Drizzle + pgvector + Vercel AI SDK.\n\n## Common mistakes\n\n**Not chunking.** Embedding whole documents produces poor retrieval — the signal is diluted across too much text.\n\n**Skipping the reranker.** Vector similarity alone returns many false positives. A reranker cuts precision errors significantly at low latency cost.\n\n**Same prompt for all queries.** The generation system prompt should vary by use case. Customer support RAG and internal knowledge base RAG need different instructions.\n\n**Not logging retrieval results.** Log what was retrieved for every query. This is your primary tool for debugging poor answers.\n\n## What to skip\n\n- **Fine-tuning instead of RAG** for knowledge-intensive tasks — RAG is cheaper, updatable, and auditable.\n- **Very small chunks** (< 100 tokens) — lose too much context; similarity scores become noisy.\n- **Multi-query fusion** without measuring — it adds latency and cost; only add it if you have retrieval benchmarks showing it helps.\n\n## FAQ\n\n**What is the right chunk size?**\nStart at 700 tokens with 70-token overlap. Measure recall@5 on a test set. Adjust from there.\n\n**Do I need to reembed when I update a document?**\nYes — update the chunks and re-embed them. Most systems delete old chunks by doc_id and reinsert.\n\n**How do I evaluate RAG quality?**\nBuild a test set of (question, expected_answer, source_document) triples. Measure retrieval recall@k and answer faithfulness (does the answer follow from the retrieved context?).\n\n**Can RAG replace fine-tuning?**\nFor knowledge and facts, yes. For behavior, tone, and format — use fine-tuning or system prompts. They solve different problems.\n\n## Where to go next\n\n[Vector databases in 2026](/blog/vector-databases-2026) covers the full storage layer options in depth. [Best backend for AI apps in 2026](/blog/best-backend-for-ai-apps-2026) covers the full server-side stack that surrounds the RAG pipeline. [Build an app with AI in 2026](/blog/build-an-app-with-ai-2026) shows how to ship an AI app end-to-end.","hue":314,"metaTitle":"How to Build a RAG App in 2026: Complete Guide","metaDescription":"Complete guide to building a RAG app in 2026. Covers chunking, embedding, vector storage, hybrid search, reranking, and generation with working code.","keywords":["how to build a rag app 2026","rag tutorial 2026","retrieval augmented generation tutorial","rag pipeline python","rag with pgvector","build rag application","rag chunking strategy","hybrid search rag","rag reranking 2026","rag app architecture 2026"],"tldr":["**RAG = chunk + embed + store + retrieve + generate** — the pipeline is simple; quality lives in the details of each step.","**Chunking strategy is the highest-leverage decision** — 500–800 token overlapping chunks beat both too-small and too-large.","**Hybrid search (dense + sparse) consistently outperforms vector-only** — add BM25 before tuning embedding models.","**Reranking improves precision significantly** — a cross-encoder reranker on the top-20 results before generating is worth the latency.","**Skip:** using whole-document embeddings — chunk first, always."],"readTime":"4 min read","wordCount":932},"related":[{"slug":"retrieval-augmented-generation-architecture-2026","title":"Retrieval-Augmented Generation Architecture in 2026","excerpt":"A practitioner walkthrough of retrieval-augmented generation architecture in 2026 — chunking, retrieval, reranking, and generation, plus where the pattern breaks down.","category":"ai","date":"July 7, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":280},{"slug":"build-an-ai-agent-from-scratch-2026","title":"Build an AI agent from scratch in 2026: a practical guide","excerpt":"Building an AI agent from scratch teaches you what every framework hides. This guide walks you through the loop, tool calling, and memory in 2026.","category":"code","date":"April 22, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":315},{"slug":"rag-vector-store-comparison-2026","title":"RAG Vector Store Comparison for 2026","excerpt":"A practical comparison of vector store options for retrieval-augmented generation in 2026 — pgvector, dedicated vector databases, and managed services, and how to pick between them.","category":"ai","date":"July 7, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":310}],"sameCategory":[{"slug":"bazel-build-explained-2026","title":"Bazel Explained: What Hermetic Builds Actually Cost","excerpt":"Bazel delivers reproducible, precisely cached builds and demands that you describe everything. Here is what you get, what it costs, and who should say no.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":145},{"slug":"graphql-federation-2026","title":"GraphQL Federation: One Graph, Many Teams","excerpt":"Federation composes independently-owned subgraphs into a single API. Here is what it solves, the operational cost it adds, and when a simpler gateway is enough.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":300},{"slug":"hermetic-builds-explained-2026","title":"Hermetic Builds Explained: Same Inputs, Same Output","excerpt":"A hermetic build depends only on declared inputs. Here is what that property gives you, the common leaks that break it, and how to move toward it incrementally.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":130},{"slug":"monorepo-tooling-compared-2026","title":"Monorepo Tooling Compared: What You Actually Need","excerpt":"Monorepo tools solve task orchestration and caching, not repository layout. Here is what each tier provides and how to tell when you have outgrown the simple option.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":150}],"newer":{"slug":"how-much-down-payment-for-a-house-2026","title":"How Much Down Payment for a House in 2026?","excerpt":"The 20 percent rule was never a law. Here is how much down payment a house really needs in 2026, by loan type, plus what to skip.","category":"finance","date":"April 30, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":38},"older":{"slug":"how-to-install-python-2026","title":"How to Install Python: Step-by-Step Setup Guide 2026","excerpt":"A plain-language, no-nonsense walkthrough of how to install Python on Windows, macOS, and Linux in 2026, plus the pitfalls that quietly break your setup.","category":"code","date":"April 30, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":128}}