Sources
Additional media







The concept of 'chunking' in the Retrieval-Augmented Generation (RAG) ecosystem is gaining attention for its role in improving information retrieval. Chunking involves splitting a large document into smaller segments to fit within the context window size of large language models (LLMs). However, this method faces challenges such as losing contextual information between chunks. An alternative approach, 'Late Chunking,' has been proposed to balance precision and cost in long-context retrieval applications. This technique, highlighted in the weaviate_io blog, suggests embedding the entire document first and then chunking the embeddings to maintain context while improving retrieval precision. Additionally, routing among domain-specific embeddings yields superior retrieval performance over single models. This method addresses the limitations of both full document embedding and traditional chunking, which either hurt retrieval precision or lose cross-chunk context.