Six RAG Chunking Strategies Compared
Fixed, recursive, semantic, structural, sentence-window and parent-document chunking measured on retrieval recall across three corpora.
By Renjith ·
Hypothesis
Chunking strategy affects retrieval recall more than chunk size does, and structure-aware strategies outperform size-based ones on documents that have real structure.
Methodology
Six strategies evaluated on three corpora — technical documentation, a policy handbook, and meeting transcripts — with 50 annotated questions per corpus.
Embedding model, retrieval depth and reranker held constant across all strategies so the chunking variable is isolated. Recall@5 is the primary metric; MRR reported as a secondary.
Each strategy tested at three size settings where size applies, to separate the strategy effect from the size effect.
Dataset
150 questions across three corpora, each annotated with the specific passage that answers it. Annotation was done once and reviewed a second time; disagreements were resolved by rewriting the question.
Experiment setup
pgvector for retrieval, a single embedding model throughout, cross-encoder reranking applied uniformly. The full harness is the RAG Evaluation Lab project.
Discussion
Why chunk size gets the attention#
It is the easiest parameter to change. It is a number in a config file, and changing it feels like tuning. Strategy changes require rewriting the ingestion path, so they get deferred — and then the discussion settles on the knob that is easy to turn rather than the one that matters.
The result in one line
Within a sensible range, chunk size is noise. Chunking strategy is a 0.20 recall swing. Spend your effort accordingly.
What to do with this#
Run the same comparison on your own corpus. Three strategies and one afternoon will tell you more than any published table, including this one — which is exactly the point of publishing the harness alongside the numbers.
Results
Recall@5, averaged across corpora:
| Strategy | Recall@5 | Best corpus | Worst corpus |
|---|---|---|---|
| Parent-document | 0.81 | Documentation (0.88) | Transcripts (0.71) |
| Structural (heading-aware) | 0.78 | Documentation (0.91) | Transcripts (0.58) |
| Sentence-window | 0.74 | Transcripts (0.79) | Handbook (0.69) |
| Semantic | 0.71 | Handbook (0.77) | Documentation (0.66) |
| Recursive character | 0.68 | Handbook (0.71) | Transcripts (0.63) |
| Fixed size | 0.61 | Handbook (0.65) | Transcripts (0.55) |
Varying chunk size within 256–768 tokens moved recall by less than 0.03 for every strategy.
Interpretation
Strategy dominates size, decisively. The spread between the best and worst strategy is 0.20 recall; the spread across sensible sizes within a strategy is under 0.03. The parameter that gets the most discussion is close to the least important one measured here.
Structure-aware strategies win where structure exists and lose where it does not. Heading-aware chunking is the best strategy on documentation, at 0.91, and among the worst on transcripts, at 0.58 — because transcripts have no headings and the strategy degrades to arbitrary splits.
Parent-document is the most robust default. It is not the best on any single corpus, and it is never bad on any of them. Retrieving small chunks for precision and then expanding to the parent for context hedges against the corpus not matching your assumptions — which is the situation most people are actually in.
The practical advice: if your corpus has reliable structure, use it. If you do not know, use parent-document and spend the saved time on reranking, which was worth more than any chunking choice in the companion evaluation.
Limitations
Fifty questions per corpus. Differences under about 0.05 recall are not distinguishable from noise at this sample size.
English text only, and three corpora is not a broad sample of document types. Tables, code and mixed-media documents behave differently and are not covered.
One annotator wrote and reviewed the questions, which introduces a consistent phrasing style that may favour some strategies.
One embedding model. Strategy interactions with embedding model are plausible and untested.
Reproducibility
Corpora, annotations and the harness are in the repository. A full run costs roughly $4 in embedding and reranking API calls and takes about 25 minutes. Question annotations are versioned so a rerun after an annotation fix is comparable.
Found something different when you reran this? That is the point — the notebook and data are linked above. Corrections are published as updates rather than quietly edited.
Related work
RAG Evaluation Lab
Most RAG systems are tuned by vibes. This is a reproducible harness that measures retrieval and generation separately, so you can see which half is failing.
RAG Chunking Calculator
Work out how many chunks a corpus produces, what retrieval depth fits your context window, and what indexing will cost.
Why Your RAG Evaluation Is Wrong
Most RAG evaluations measure the wrong thing, on the wrong data, with a judge that rewards the wrong behaviour. Here is what to measure instead.