Sources
Loading...
Additional media
Loading...
DeepNewz, mobile.
People-sourced. AI-powered. Unbiased News.
Recent advancements in AI technology have enabled users to run large language models, such as Llama-3-70B and Llama-3-405b, on various personal devices. This can be achieved by combining the compute power of multiple devices, including iPhone 15 Pro Max, iPad Pro M4, Galaxy S24 Ultra, MacBook Pro M2 and M3 Pro, and MSI NVIDIA GeForce RTX 4090 SUPRIM GPUs. The process utilizes open-source code and MLX as a back-end, making it accessible for home use on both Mac and Windows. The open-source community is expected to optimize these models further, allowing them to run efficiently on limited compute resources.
Build a home cluster with Apple devices by @exolabs_ Code is open source https://t.co/DR8LYHLmEV ! Example of distributed LLM inference using MLX as a back-end: https://t.co/8EtNPqVhOc https://t.co/B5LCuT1Fel https://t.co/AaiQU0CXjT
You can run large AI models at home using any device you have (iPhone, Android, laptops, gaming GPUs, etc) Check the open-source code below 👇 Uses MLX, h/t @awnihannun https://t.co/Desh1sJlXw
Running Llama-3-70B at home with @exolabs_ Combines the compute of all these devices to make one big GPU: - iPhone 15 Pro Max - iPad Pro M4 - Galaxy S24 Ultra - MacBook Pro M2 and M3 Pro - 2 x MSI NVIDIA GeForce RTX 4090 SUPRIM Code is open source 👇 https://t.co/bFfwYIRCJI
