The week in AI: Elon Musk's xAI activates the world's most powerful AI training cluster

In partnership with

Welcome to The Dispatch! We are the newsletter that keeps you informed about AI. Each Thursday, we aggregate the major developments in artificial intelligence - we pass along the news, useful resources, tools and services; we highlight the top research in the field as well as exciting developments in open source. Even if you aren’t a machine learning engineer, we’ll keep you in touch with the most important developments in AI.

NEWS & OPINION

Elon Musk’s xAI supercomputer comes online in Memphis

-------------------------

Elon Musk's artificial intelligence company, xAI, has begun operations at what he claims is "the most powerful AI training cluster in the world" in Memphis, Tennessee. The new supercluster (built inside an abandoned 750,000-square-foot plant) boasts an incredible 100,000 liquid-cooled Nvidia H100 GPUs - estimated pricetag: $2.5 billion - linked on a single RDMA fabric that was developed in collaboration with Nvidia. To put the size/scale of xAI’s Memphis supercluster into perspective:

• OpenAI trained GPT-4 on an estimated 25,000 Nvidia A100 GPUs - the H100’s predecessor, less powerful and efficient for AI processes.

• Google Cloud ran what they claimed was the world’s largest distributed LLM training job in November 2023 across 50,000+ TPU v5e chips. These TPU’s are specialized for machine learning but have significantly lower peak performance per chip than the H100.

• Meta trained its state-of-the-art LLM, Llama 3 405B, on a cluster with 16,000 H100 GPUs.

Despite the grand claims, the project faces some significant challenges - particularly regarding power supply and backlash from local and environmental groups. The facility could require up to 150 megawatts of power at peak times, raising concerns about its impact on the local power grid. Notably, the Tennessee Valley Authority (TVA) has not signed off on the project yet, and discussions are ongoing regarding electricity demands. To sidestep the issue, Musk is currently renting a fleet of 14 VoltaGrid natural gas generators to provide supplementary power while negotiations with local utilities continue.

The project represents the largest capital investment by a new company in Memphis's history. Local utility company Memphis Light, Gas and Water (MLGW) plans to host public forums to address questions about the project's implications for the city. Meanwhile, Musk maintains that xAI will have "a significant advantage in training the world's most powerful AI by every metric by December this year." Given Musk’s propensity to offer grandiose or false predictions, there’s good cause for skepticism of that claim.

But not for a lack of hardware.

Microsoft’s mixed earnings report won’t stall the AI boom

-------------------------

A steep drop last Wednesday sent US stock indexes to their worst losses since 2022 after profit reports from Tesla and Alphabet helped suck some momentum from Wall Street’s frenzy around AI technology. Despite concerns over heavy AI spending across the tech sector, most analysts believe the long-term AI investment trend remains strong.

Microsoft reported strong overall financial results for its latest quarter, with revenue up 15% to $64.7 billion and profit rising 10% to $22 billion. However, growth in its crucial Azure cloud computing business surprisingly fell slightly short of expectations at 30%, causing the company's stock to drop in after-hours trading. Microsoft re-emphasized its commitment to heavy investments in artificial intelligence technology. While AI services are attracting new cloud customers, the company faces capacity constraints expected to persist through the end of 2024 - they signed a partnership with telecom giant Lumen last week to help address AI workload capacity.

Meta reported strong second-quarter earnings that exceeded analysts' expectations. While Meta continues to be one of the biggest investors in AI technology, its capital expenditures of $8.47 billion were much lower than anticipated. Meta also has a number of ongoing legal challenges, including a recent $1.4 billion settlement in a Texas privacy lawsuit, and they are pulling a number of their AI services from being released in Europe over regulatory concerns.

Apple and Amazon report later today. We’ll be covering them next week.

OpenAI releases ChatGPT’s hyperrealistic voice mode to some users

-------------------------

OpenAI is rolling out “Advanced Voice Mode” for ChatGPT, with full access gradually coming for all Plus users by fall of this year. We’re highlighting it here because while it might seem like just another product update, the ability to interact with a powerful AI model through voice rather than just text is a profound shift - at least when the voice isn’t jarringly robotic.

We don’t have access yet, but we do have Wharton Professor Ethan Mollick to share his experience with Advanced Voice Mode and explain why this matters. We wholeheartedly agree with his assessment here:

“This is all early, and based on first impressions, but I think that voice capabilities like GPT-4o’s are going to change how most people interact with AI systems.“

MORE IN AI THIS WEEK

• OpenAI CEO Sam Altman’s Washington Post op-ed: Who will control the future of AI?

• Taco Bell to roll out AI drive-thru ordering in hundreds of locations by end of year

• The Olympic AI Agenda

• Perplexity announces revenue sharing deal with publishers

• Google tweaks Search to help hide explicit deepfakes

• AMD is becoming an AI chip company, just like Nvidia

• A new White House report embraces open-source AI

• OpenAI is testing SearchGPT, a temporary prototype of new AI search features that give you fast and timely answers with clear and relevant sources

• Microsoft aims to boost AI workload capacity with Lumen partnership

• Video game performers will go on strike over artificial intelligence concerns

• Anthropic’s crawler is ignoring websites’ anti-AI scraping policies

FREE AI & ChatGPT Masterclass to automate 50% of your workflow

More than 300 Million people use AI across the globe, but just the top 1% know the right ones for the right use-cases.

Join this free masterclass on AI tools that will teach you the 25 most useful AI tools on the internet – that too for $0 (they have 100 free seats only!)

Get it now for absolutely free! (for first 100 users only) 🎁

This masterclass will teach you how to:

• Build business strategies & solve problems like a pro

• Write content for emails, socials & more in minutes

• Build AI assistants & custom bots in minutes

• Research 10x faster, do more in less time & make your life easier

You’ll wish you knew about this FREE AI masterclass sooner 😉

Register & save your seat now! (valid for next 24 hours only!)

TECHNICAL, RESEARCH & OPEN SOURCE

• Pytorch introduces torchchat: Accelerating Local LLM Inference on Laptop, Desktop and Mobile

• AI reprograms brain cancer cells into dendritic cells for cancer immunotherapy

• Apple releases its Apple Intelligence Foundation Language Model technical report

• French startup Mistral AI releases Mistral Large 2: pushing the boundaries of cost efficiency, speed, and performance

• Google expands Gemma 2 open-source family of models, including Gemma 2 2B which outperforms GPT-3.5 with only 2 billion parameters

• Meta Research’s Segment Anything Model 2 (SAM 2): an advanced AI model that can identify and track objects across video frames in real-time

• exo: Run your own AI cluster at home with everyday devices. Forget expensive NVIDIA GPUs, unify your existing devices into one powerful GPU: iPhone, iPad, Android, Mac, Linux, pretty much any device

• Google DeepMind’s AlphaProof + AlphaGeometry teams: AI achieves silver-medal standard solving International Mathematical Olympiad problems

• Stable Video 4D from Stable Diffusion: a new AI model that can turn single object videos into multiple videos - with potential applications in game development, video editing, and virtual reality

That’s all for this week! We’ll see you next Thursday.