Jul 19, 07:03 PM

MMMU Benchmark and New Tools Enhance Llama3 70B RM Efficiency with 1 Million Token Capacity and EM-LLM Architecture

A series of advancements in large language models (LLMs) and multimodal AI technologies were announced, highlighting new benchmarks and tools aimed at enhancing model efficiency and performance. The MMMU benchmark was introduced to challenge AI models to understand and reason using text and images from college-level exams. Innovations such as XTuner promise to cut DPO training time in half and enable the training of Llama3 70B RM models with sequence lengths up to 1 million tokens on 64 A100 GPUs. Additionally, LightOn's research explored hardware-agnostic training of LLMs on AMD GPUs, while the EM-LLM architecture integrates aspects of human memory into transformer-based models. New tools like LlaMa-Factory and SmoLlm facilitate fine-tuning and deployment of LLMs, and various benchmarks, including MMLongBench-Doc, focus on evaluating long-context document understanding. Furthermore, the integration of WebAssembly as a portable runtime for LLMs addresses cross-platform challenges in cloud-native applications.

#XTuner #Llama #LightOn #AMD #SmoLlm #WebAssembly

Written with ChatGPT (GPT-4o mini).

Sources

Additional media

Image #1 for story mmmu-benchmark-new-tools-enhance-llama3-70b-rm-efficiency-1-million-token-em-llm

Image #2 for story mmmu-benchmark-new-tools-enhance-llama3-70b-rm-efficiency-1-million-token-em-llm

Image #3 for story mmmu-benchmark-new-tools-enhance-llama3-70b-rm-efficiency-1-million-token-em-llm

Image #4 for story mmmu-benchmark-new-tools-enhance-llama3-70b-rm-efficiency-1-million-token-em-llm

Image #5 for story mmmu-benchmark-new-tools-enhance-llama3-70b-rm-efficiency-1-million-token-em-llm

Image #6 for story mmmu-benchmark-new-tools-enhance-llama3-70b-rm-efficiency-1-million-token-em-llm

Image #7 for story mmmu-benchmark-new-tools-enhance-llama3-70b-rm-efficiency-1-million-token-em-llm

Image #8 for story mmmu-benchmark-new-tools-enhance-llama3-70b-rm-efficiency-1-million-token-em-llm

Image #9 for story mmmu-benchmark-new-tools-enhance-llama3-70b-rm-efficiency-1-million-token-em-llm