Nov 22, 07:41 PM

XGrammar Launches, Promising 3-10x Speedup and 14x Faster JSON-Schema Generation

The recent launch of XGrammar, a new structured generation engine, has generated significant excitement in the tech community. This engine promises accurate JSON and grammar generation with a latency speedup of 3 to 10 times. Additionally, it boasts a remarkable 14 times faster JSON-schema generation and up to 80 times faster context-free grammar (CFG)-guided generation. XGrammar is designed for seamless integration with existing language model engines, including MLC-LLM, SGLang, and WebLLM, with plans to incorporate vLLM and HuggingFace in the near future. Developers from various projects are expressing eagerness to collaborate on integrating XGrammar into their platforms, highlighting its potential to enhance structured generation capabilities.

#XGrammar #SGLang #WebLLM #vLLM #HuggingFace

Written with ChatGPT (GPT-4o mini).

Sources

Vaibhav (VB) Srivastav@reach_vb
2 years ago
LFG!! XGrammar: a lightning fast, flexible, and portable engine for structured generation! 🔥 > Accurate JSON/grammar generation > 3-10x speedup in latency > 14x faster JSON-schema generation and up to 80x CFG-guided generation > Now in MLC-LLM, SGLang, WebLLM; vLLM &… https://t.co/nRudqzA37i
Tianqi Chen@tqchenml
2 years ago
🚀Future LLM agents speak JSON, python, and other structures. Excited to announce XGrammar, an structured generation library that enables zero-overhead structure constraining. Bring 2x-10x speedup in grammar guided LLM serving. Checkout github repo, blog to learn more 👉 https://t.co/hJv7Pl5ZFW
vLLM@vllm_project
2 years ago
⬇️This is super cool and we look forward to work with the team and community to integrate XGrammar with vLLM. Let's bring fast JSON/grammar generation to everyone! https://t.co/iTBdu0eGEC

XGrammar Launches, Promising 3-10x Speedup and 14x Faster JSON-Schema Generation

Sources

Additional media