ResearchEvaluation

Six RAG Chunking Strategies Compared

Fixed, recursive, semantic, structural, sentence-window and parent-document chunking measured on retrieval recall across three corpora.

By Renjith ·

Hypothesis

Chunking strategy affects retrieval recall more than chunk size does, and structure-aware strategies outperform size-based ones on documents that have real structure.

Methodology

Six strategies evaluated on three corpora — technical documentation, a policy handbook, and meeting transcripts — with 50 annotated questions per corpus.

Embedding model, retrieval depth and reranker held constant across all strategies so the chunking variable is isolated. Recall@5 is the primary metric; MRR reported as a secondary.

Each strategy tested at three size settings where size applies, to separate the strategy effect from the size effect.

Dataset

150 questions across three corpora, each annotated with the specific passage that answers it. Annotation was done once and reviewed a second time; disagreements were resolved by rewriting the question.

Experiment setup

pgvector for retrieval, a single embedding model throughout, cross-encoder reranking applied uniformly. The full harness is the RAG Evaluation Lab project.

Discussion

Why chunk size gets the attention#

It is the easiest parameter to change. It is a number in a config file, and changing it feels like tuning. Strategy changes require rewriting the ingestion path, so they get deferred — and then the discussion settles on the knob that is easy to turn rather than the one that matters.

The result in one line

Within a sensible range, chunk size is noise. Chunking strategy is a 0.20 recall swing. Spend your effort accordingly.

What to do with this#

Run the same comparison on your own corpus. Three strategies and one afternoon will tell you more than any published table, including this one — which is exactly the point of publishing the harness alongside the numbers.

Results

Recall@5, averaged across corpora:

Strategy Recall@5 Best corpus Worst corpus
Parent-document 0.81 Documentation (0.88) Transcripts (0.71)
Structural (heading-aware) 0.78 Documentation (0.91) Transcripts (0.58)
Sentence-window 0.74 Transcripts (0.79) Handbook (0.69)
Semantic 0.71 Handbook (0.77) Documentation (0.66)
Recursive character 0.68 Handbook (0.71) Transcripts (0.63)
Fixed size 0.61 Handbook (0.65) Transcripts (0.55)

Varying chunk size within 256–768 tokens moved recall by less than 0.03 for every strategy.

Interpretation

Strategy dominates size, decisively. The spread between the best and worst strategy is 0.20 recall; the spread across sensible sizes within a strategy is under 0.03. The parameter that gets the most discussion is close to the least important one measured here.

Structure-aware strategies win where structure exists and lose where it does not. Heading-aware chunking is the best strategy on documentation, at 0.91, and among the worst on transcripts, at 0.58 — because transcripts have no headings and the strategy degrades to arbitrary splits.

Parent-document is the most robust default. It is not the best on any single corpus, and it is never bad on any of them. Retrieving small chunks for precision and then expanding to the parent for context hedges against the corpus not matching your assumptions — which is the situation most people are actually in.

The practical advice: if your corpus has reliable structure, use it. If you do not know, use parent-document and spend the saved time on reranking, which was worth more than any chunking choice in the companion evaluation.

Limitations

Fifty questions per corpus. Differences under about 0.05 recall are not distinguishable from noise at this sample size.

English text only, and three corpora is not a broad sample of document types. Tables, code and mixed-media documents behave differently and are not covered.

One annotator wrote and reviewed the questions, which introduces a consistent phrasing style that may favour some strategies.

One embedding model. Strategy interactions with embedding model are plausible and untested.

Reproducibility

Corpora, annotations and the harness are in the repository. A full run costs roughly $4 in embedding and reranking API calls and takes about 25 minutes. Question annotations are versioned so a rerun after an annotation fix is comparable.

Found something different when you reran this? That is the point — the notebook and data are linked above. Corrections are published as updates rather than quietly edited.

Related work

AI Systems

RAG Evaluation Lab

Most RAG systems are tuned by vibes. This is a reproducible harness that measures retrieval and generation separately, so you can see which half is failing.

IntermediateDifficulty: Intermediate~12h buildPython · Ragas · DuckDB
Code
AI & LLMFree

RAG Chunking Calculator

Work out how many chunks a corpus produces, what retrieval depth fits your context window, and what indexing will cost.

840 runs
AI Engineer Lab

Why Your RAG Evaluation Is Wrong

Most RAG evaluations measure the wrong thing, on the wrong data, with a judge that rewards the wrong behaviour. Here is what to measure instead.

IntermediateDifficulty: Intermediate3 min readPython · Ragas · OpenAI API
Video