Aug 14, 12:06 PM

Tsinghua and Zhipu AI's LongWriter Model Generates Over 10,000 Words

Researchers from Tsinghua University and Zhipu AI have introduced a new model called LongWriter, specifically LongWriter-glm4-9b, designed to generate long-form text exceeding 10,000 words. This development addresses a limitation in current large language models (LLMs), which can process inputs up to 100,000 tokens but typically struggle to produce outputs longer than 2,000 words. The LongWriter model uses an agent-based pipeline and includes a dataset specifically created for this purpose, with some models fine-tuned for local usage. The paper, from THUDM, detailing this innovation has been assessed using GPT-4 judged metrics.

#Tsinghua University #Zhipu AI #LongWriter

Written with ChatGPT (GPT-4o).

Sources

Rohan Paul@rohanpaul_ai
2 years ago
Existing long context LLM already possesses the potential for a larger output window–all you need is data with extended output during model alignment to unlock this capability. Problem 😕: LLMs struggle to generate outputs exceeding 2,000 words, despite handling inputs over… https://t.co/rqahHFov6y
Gradio@Gradio
2 years ago
LongWriter-glm4-9b from @thukeg is capable of generating 10,000+ words at once!🚀 Paper identifies a problem with current long context LLMs -- they can process inputs up to 100,000 tokens, yet struggle to generate outputs exceeding lengths of 2,000 words. Paper proposes that an… https://t.co/2jfKyIpShK
AK@_akhaliq
2 years ago
New from @thukeg LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs author @realYushiBai is active in discussion section to answer your questions: https://t.co/UeebckjJjf https://t.co/FfOczYwz4G

Additional media

Image #1 for story tsinghua-zhipu-ai-s-longwriter-model-generates-over-10000-words

Image #2 for story tsinghua-zhipu-ai-s-longwriter-model-generates-over-10000-words