Sources
Loading...
Additional media











































Jina AI has introduced a new chunking method called 'Late Chunking,' which enhances text embeddings for retrieval tasks by encoding entire documents before splitting them. This technique allows for better retention of context and improved retrieval performance, akin to ColBERT but with the same storage costs as regular methods. The method is available for use within Weaviate with minimal changes to existing code. The research team includes M Günther, I Mohr, B Wang, and H Xiao.