Jun 6, 06:18 PM

JinaAI's Jina-CLIP Outperforms Previous Models; TogetherCompute's Dragonfly Excels in Medical Imaging, Outperforms Med-Gemini

JinaAI has developed a new image-text contrastive pretraining technique that improves the text encoder and outperforms previous CLIP models on the MTEB benchmark. The new model, Jina-CLIP, is open-source and available via Transformers and Transformers.js. It supports text-text retrieval and MuRAG, offering a significant improvement over OpenAI's CLIP. Meanwhile, TogetherCompute has introduced Dragonfly, a set of vision-language models that leverage multi-resolution encoding and zoom-in patch selection to enhance fine-grained visual understanding. These models have shown superior performance in medical image understanding, even outperforming Med-Gemini. The Dragonfly models were developed in collaboration with James Y. Zou and include advanced features such as encoding images at multiple resolutions and selecting salient patches for language models, achieving state-of-the-art results in medical captioning. Additionally, a new approach called Cluster Masking enhances visual-language contrastive learning.

#JinaAI #Transformers #MuRAG #OpenAI #TogetherCompute #Dragonfly #Cluster Masking

Written with ChatGPT (GPT-4o).

Sources

Additional media

Image #1 for story jinaai-s-jina-clip-outperforms-previous-models-togethercompute-s-dragonfly-med

Image #2 for story jinaai-s-jina-clip-outperforms-previous-models-togethercompute-s-dragonfly-med

Image #3 for story jinaai-s-jina-clip-outperforms-previous-models-togethercompute-s-dragonfly-med