TechCrunch: “Meta’s latest open source AI model is its biggest yet. Today, Meta said it is releasing Llama 3.1 405B, a model containing 405 billion parameters. Parameters roughly correspond to a model’s problem-solving skills, and models with more parameters generally perform better than those with fewer parameters. At 405 billion parameters, Llama 3.1 405B isn’t the absolute largest open source model out there, but it’s the biggest in recent years. Trained using 16,000 Nvidia H100 GPUs, it also benefits from newer training and development techniques that Meta claims makes it competitive with leading proprietary models like OpenAI’s GPT-4o and Anthropic’s Claude 3.5 Sonnet (with a few caveats). As with Meta’s previous models, Llama 3.1 405B is available to download or use on cloud platforms like AWS, Azure and Google Cloud. It’s also being used on WhatsApp and Meta.ai, where it’s powering a chatbot experience for U.S.-based users…”

• See also Tech Republic – Llama 3 Cheat Sheet: A Complete Guide for 2024
• See also The Verge – Meta releases the biggest and best open-source AI model yet
• See also Ben’s Bytes – Meta has released a massive technical report outlining what are the challenges they faced building a model of this size and how they overcame them. The amount of research they have shared in it is insane. You can read the paper here and listen to the Latent Space podcast with Thomas Scialom (Llama 2 lead and Llama 3 post-training lead) for more technical details.