# You're doing RAG wrong By **akshay-pachaar** · Artículos A technical argument for replacing text chunks with structured QA packets in RAG pipelines, for ML engineers and data architects building production retrieval systems. - Source: https://x.com/akshay_pachaar/status/2052743644411765230 - Tags: rag, retrieval, llm, vector-search, embeddings, open-source, data-architecture, production-ai - Upvotes: 0 --- ## The big picture Most RAG systems fail at ingestion, not retrieval. The chunk, a fixed-size window of prose cut wherever the token count runs out, is the default unit of knowledge in nearly every RAG pipeline, and it is the root cause of the most common retrieval failures. Manideep Patibandla's dev.to article (2024) argues that fixing this upstream makes downstream tuning mostly unnecessary. ML engineers and data architects who own an ingestion pipeline are the primary audience here. If you manage a vector store used by an LLM application, this argument is directly actionable. ## Why it matters A chunk has no idea boundaries, no version state, and no access-control metadata. When a splitter cuts at token count, the vector store ends up holding half a table, a conclusion without its argument, or the same paragraph from twelve near-identical document versions across SharePoint, Confluence, and Git. Top-K retrieval returns five copies of the same stale paragraph, and the LLM blends current and deprecated content into a confident, wrong answer. The version problem and the access-control problem compound together. Because a chunk carries no typed metadata, role filters and clearance logic get bolted onto the orchestrator, disconnected from the content they govern. ## How it works The proposed fix is structural: replace the chunk with a question-answer (QA) packet. One unit holds one question, its validated answer, and typed governance fields such as version state, clearance level, and source, all on the same object. User queries are already questions, so embedding a question against a question-indexed store produces a direct structural match rather than a loose semantic float. Cosine distance between a query embedding and a QA-packet embedding is tighter than cosine distance between the same query and a prose window that happens to contain the answer. Per the dev.to article, this shift produces measurable gains: 40x corpus size reduction, 3x fewer tokens per query, and 2.3x improvement in vector search relevance, without changing the reranker, embedding model, or retrieval algorithm. Blockify, a preprocessing layer from Iternal Technologies, implements this pattern as a structure it calls an IdeaBlock. To bridge the gap that LangChain, LlamaIndex, and Haystack leave open (none of them transform documents into QA packets by default), the most direct approach is a custom document-transformation stage before the vector store write: extract claims, generate question-answer pairs, attach metadata, then embed. Blockify's documented integration path follows this pattern. ## Yes, but A second practitioner thread, published on agntai.net (updated May 2026), diagnoses a parallel failure: embedding model mismatch. Swapping a generic sentence-transformers model for OpenAI's text-embedding-ada-002 lifted relevance scores by 35% in one reported case, because domain-specific text needs a context-aware embedding. The agntai.net author also flags database over-engineering: teams with 10,000 documents reaching for distributed vector databases when FAISS would suffice, and teams with a million documents trying to make SQLite work. The QA-packet argument and the embedding-model argument are not mutually exclusive. Both point to the same underlying principle: retrieval quality is determined before the query runs, not during it. ## The bottom line Audit your ingestion pipeline for chunk boundary logic before evaluating a new reranker or embedding model. If your chunks cross idea boundaries or mix document versions, no retrieval algorithm will compensate fully. The concrete next step: pick one document type in your corpus, extract QA pairs manually from ten documents, embed them, and compare top-K relevance against your current chunks using the same queries. The structural difference in cosine distance will tell you whether the approach is worth automating at scale. ## FAQ ### What is the central argument of 'You're doing RAG wrong'? The article argues that the chunk is the wrong unit of knowledge for RAG, and that this ingestion-layer mistake causes most of the retrieval failures developers try to fix with better rerankers or embedding models. Per the dev.to source by Manideep Patibandla, replacing prose chunks with structured question-answer packets fixes the problem upstream and produces reported gains of 40x corpus reduction, 3x fewer tokens per query, and 2.3x better vector search relevance. The argument is that no retrieval algorithm can fully compensate for structurally broken input. ### What is a QA packet and how does it differ from a standard chunk? A QA packet is a structured unit holding one question, its validated answer, and typed governance metadata such as version state, clearance level, and source, all on the same object. A standard chunk is a fixed-size prose window cut at token boundaries with no idea boundaries, no version state, and no embedded access-control metadata. Because user queries are already questions, embedding a QA packet produces a direct structural match with incoming query embeddings, tightening cosine distance compared to a prose window that only incidentally contains the answer. ### Is there a tool that implements QA packets, and is it free to use? Blockify, a preprocessing layer from Iternal Technologies, implements the QA-packet pattern using a structure it calls an IdeaBlock. Each IdeaBlock holds a question, its validated answer, and typed governance fields. The sources do not provide pricing or licensing details for Blockify, but the QA-packet pattern itself is an architectural approach any team can implement as a custom document-transformation stage before writing to a vector store, without requiring a specific commercial product. ### What other root causes do the articles identify beyond chunk boundaries? The agntai.net practitioner article (updated May 2026) identifies embedding model mismatch as a major, independent failure mode: teams use generic sentence-transformers models without evaluating domain fit, and one reported case showed a 35% relevance lift from switching to OpenAI's text-embedding-ada-002 for domain-specific text. The same source flags database over-engineering, teams with 10,000 documents using distributed vector systems that belong at a million-document scale, and the inverse problem of under-powered stores for large corpora. The Towards AI article also highlights context blindness, where retrieved chunks lack enough surrounding information to be interpreted correctly, and ### How does fixing the ingestion layer compare to tuning the retrieval layer? Switching the knowledge unit outperforms retrieval tuning on the metrics reported in the dev.to article: QA packets yield 40x corpus reduction and 2.3x relevance improvement without touching the reranker, embedding model, or retrieval algorithm. Retrieval-layer fixes, such as reranking or hybrid search, operate on whatever the vector store contains and cannot recover information destroyed by bad chunk boundaries or deduplicate mixed-version paragraphs that were never tagged at ingestion. The articles position ingestion-layer fixes as higher leverage because they address the structural cause rather than compensating for it. ### What are the main limitations or risks of this approach? The sources report benchmark numbers (40x, 3x, 2.3x, 35%) without naming the specific datasets, model versions, or experimental conditions used, so these figures should be treated as directional rather than universally reproducible. Generating QA packets at scale requires an additional extraction step, either manual curation or an LLM-assisted pipeline, which adds latency and cost to ingestion. The sources also note that LangChain, LlamaIndex, and Haystack do not implement QA-packet transformation by default, so teams adopting this approach need to build or integrate a custom preprocessing stage before the vector store write. --- [View on Analog](https://analoghq.ai/es/akshay-pachaar/articles/youre-doing-rag-wrong)