Chunking strategies for heterogeneous web documents

The most useful thing we did last quarter was read our own failures. In DBR-2026-01 we hand-inspected 200 retrieval misses and found that most of them were not the model failing to understand the query. They were the relevant sentence sitting on the wrong side of a boundary we drew ourselves.

That is a much cheaper problem than a representation problem. Better embeddings cost storage, memory, and indexing time, all of which are scarce on a laptop. Better chunking costs some parsing logic and runs once per document. So we held the embedding model fixed at 384 dimensions with product quantisation and changed only how documents are split.

Six strategies

• Fixed 512 tokens. The baseline from DBR-2026-01. Split on token count, no overlap, no regard for structure.
• Fixed 512 with 15% overlap. The same, but each chunk repeats the last 15% of the previous one, so a sentence spanning a boundary appears whole somewhere.
• Sentence-aligned. Never split mid-sentence; pack sentences up to a token budget.
• Structure-aware. Split on heading boundaries where the document has them, falling back to sentence-aligned within long sections.
• Structure-aware + caption binding. As above, but tables, figures, and code blocks are kept with their captions and preceding paragraph regardless of length.
• Structure-aware + caption binding + overlap. All of the above together.

The headline is that structure-aware chunking with caption binding and overlap reaches 0.83, which is above what the 1024-dimension embedding manages with naive chunking (0.81) and costs a ninth of the storage. We did not expect the crossover. When we started this work our internal assumption was that chunking was worth two or three points and the rest would have to come from a bigger model.

Caption binding does most of the work

Decomposing the gain is more informative than the total. Overlap alone is worth about two points and is nearly free. Sentence alignment is worth another point and a half. Structure awareness adds three. Caption binding — refusing to separate a table, figure, or code block from the text that explains it — is worth two and a half points on its own, which is remarkable for a rule that fires on a minority of documents.

Structure-aware chunking produces fewer chunks than the baseline, because heading-bounded sections are often shorter than 512 tokens and the chunker stops rather than packing across a boundary. That is why index size falls slightly even as recall rises. Adding overlap puts the size back, and we think the trade is worth it, but a deployment tight on storage could reasonably stop at structure plus captions.

The cheapest gain in a retrieval system is usually the one you can get without touching the model. We keep relearning this.

The failures change character

After structure-aware chunking, boundary failures fall from 71% to 19% of a smaller total. What replaces them is vocabulary mismatch: the user asks about “the thing that broke the deploy” and the document says “pipeline failure in stage 3”. That is a representation problem, and it is the point at which a larger embedding model, or query expansion, starts to be the right investment. We are not there yet on storage grounds, but the analysis now points somewhere different than it did six months ago.

What structure-aware chunking costs

Indexing cost rises 38%, from about 240 ms to about 330 ms per document. Almost all of that is PDF parsing: an HTML page with real headings is cheap to segment, while an untagged PDF requires layout analysis to recover a structure the file never recorded. We cap the layout pass at 400 ms and fall back to sentence alignment when it exceeds that, which affects roughly 6% of PDFs in our corpus and costs about half a point of recall on those documents.

Method

Limitations

This study inherits every weakness of DBR-2026-01, most importantly that the corpus is synthetic and the queries were written by people who had seen it. It adds one of its own: caption binding and structure awareness are rules we wrote after inspecting failures in this corpus, so there is a real risk we have fitted the chunker to the evaluation set. We held out 200 documents and 120 queries that no one on the team inspected during development, and recall on that held-out slice was 0.81 rather than 0.83 — lower, which suggests some overfitting, but not enough to change the conclusion. We report both numbers.

The chunker is part of the open evaluation harness and can be run against your own documents locally. If you have a corpus where structure-aware chunking does not help, that is a result we want: research@dartbrowser.com.

Related work

Chunking is under-studied relative to its effect. Most retrieval literature holds the splitter fixed and varies the encoder, and the standard benchmarks [6] supply pre-chunked passages, which removes the variable entirely. The closest prior work is in the long-document QA literature, where passage segmentation is known to matter, but that setting assumes homogeneous documents. Personal browsing history is the opposite: an HTML article, a tagged PDF, and a Markdown README need different treatment, and the chunker cannot know in advance which it is holding.

Artefacts

Everything below is published or available on request. A number nobody can reproduce is an advertisement, not a result.

References

1. Karpukhin, V., Oğuz, B., Min, S., et al. (2020). Dense Passage Retrieval for Open-Domain Question Answering. EMNLP.
2. Robertson, S., & Zaragoza, H. (2009). The Probabilistic Relevance Framework: BM25 and Beyond. Foundations and Trends in IR.
3. Malkov, Y. A., & Yashunin, D. A. (2018). Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs. IEEE TPAMI.
4. Jégou, H., Douze, M., & Schmid, C. (2011). Product Quantization for Nearest Neighbor Search. IEEE TPAMI, 33(1).
5. Bhardwaj, M. (2026). Retrieval quality against index size for on-device browsing history. Dart Browser Research, DBR-2026-01.
6. Thakur, N., Reimers, N., Rücklé, A., Srivastava, A., & Gurevych, I. (2021). BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models. NeurIPS Datasets & Benchmarks.