The week in AI: Have LLMs already hit a scaling wall?

In partnership with

Welcome to The Dispatch! We are the newsletter that keeps you informed about AI. Each Thursday, we aggregate the major developments in artificial intelligence - we pass along the news, useful resources, tools and services; we highlight the top research in the field as well as exciting developments in open source. Even if you aren’t a machine learning engineer, we’ll keep you in touch with the most important developments in AI.

NEWS & OPINION

Has LLM scaling hit a wall?

-------------------------

There has been some serious debate this week about how the AI industry should view scaling as a path to further progress/AGI. For over a decade, “scaling laws” have dominated AI, with companies pursuing exponential growth in model parameters, data, and compute power to achieve impressive leaps in performance. However, upcoming models like OpenAI’s Orion, Anthropic’s Claude 3.5 Opus, and Google’s Gemini 2.0 have all revealed unexpectedly rapid diminishing returns from scaling that has led to product delays, raising questions about whether this approach can sustain further advancements given the immense cost.

So, what sparked the debate? In addition to the aforementioned reports on diminishing returns, Ilya Sutskever (co-founder of OpenAI, which pioneered scaling laws), initially told Reuters that the 2010s era of scaling may have plateaued, suggesting that the field now requires a new phase of discovery to unlock further capabilities. This interview led to Meta AI Chief Yann LeCun making an “I told you so” post on X, which prompted a scathing response from famous AI skeptic Gary Marcus. Even OpenAI CEO Sam Altman cheekily chimed in:

there is no wall

^ That’s perhaps not a surprising take from Altman, AI’s Chief Hype Officer; but the central challenge underlying this plateau is a scarcity of high-quality, human-generated data. Research from Epoch AI warns that existing public text data sources may be exhausted by 2026, leaving fewer options for high-quality training material. Companies are experimenting with synthetic data to bypass this shortage, but reliance on artificial data may result in a recursive “model collapse”. Other companies like Outlier, TELUS International and Data Annotation have been hiring tens of thousands of remote workers to create domain-specific, human-curated datasets that are effectively useless for anything besides training AI models.

In the face of these challenges, the question remains: has deep learning reached a wall? The emerging consensus suggests that while traditional scaling approaches are beginning to plateau, it’s not entirely a dead end. AI will also continue evolving rapidly through specialization and innovation in reasoning rather than sheer size, regardless. The future of AI will depend on a balance of technical breakthroughs, resource allocation, and economic feasibility. If the industry can navigate these challenges, AI will continue to advance - albeit along a more nuanced, less exponential path to AGI than scaling laws might have predicted.

Agent wars: OpenAI hopes to launch its first AI agent, called ‘Operator,’ in January

-------------------------

In the last couple months, we’ve seen a flurry of AI agent announcements: Google’s Jarvis, Anthropic’s Computer Use, Microsoft’s Copilot agents - now OpenAI’s Operator agent has been announced and is expected to be controlling computers near you by January.

An AI agent is an intelligent system designed to autonomously handle and streamline tasks, acting as a proactive assistant that collaborates with individuals and teams and can interact within apps or other environments, not just a chat interface. AI agents vary in capability: some are skilled in retrieving and analyzing data to provide summaries and answer questions, while more advanced agents can execute tasks upon request, coordinating actions across different platforms.

Operator is expected to be capable of performing multi-step tasks autonomously, such as booking flights or writing code, through browser-based interactions with minimal human oversight. Operator will act more like a digital assistant, fulfilling OpenAI CEO Sam Altman’s recent assertion that AI’s “next giant breakthrough” will come from agentic capabilities.

As we noted, Operator joins a growing list of AI agents developed by Anthropic, Google, and Microsoft - all of which are exploring different applications of autonomous task management. With so many similar offerings, what differentiator will make these tools stand out above each other? We’re just getting started with the agent wars.

Slack's latest survey offers a surprising reality check on AI usage at work

-------------------------

A new survey from Slack reveals the stark disconnect between executive-level commitment to AI and the workforce's perception of its value. While 99% of executives plan to invest in AI within the coming year, uptake among desk workers is stalling - and enthusiasm for AI is waning, marking the first cooling of sentiment since generative AI’s debut. The survey was conducted with over 17,000 desk workers worldwide and examines the challenges and opportunities for organizations aiming to incorporate AI effectively into their operations. Here are some key insights from the report:

• Executives show overwhelming support for AI, with 99% planning investments this year, and 97% reporting urgency to embed AI into business processes. However, this enthusiasm isn’t mirrored among desk workers, as AI adoption has started to plateau globally.

• Excitement around AI dropped six percentage points (more in the U.S. and France) signaling a cooling trend among employees who are uncertain about the impact of AI on their daily tasks.

• 48% of desk workers feel uncomfortable sharing their AI use with managers, citing fears that using AI may make them seem lazy, less skilled, or like they’re “cheating” at their work.

• Lack of training persists as a major barrier to AI uptake, with 61% of employees having spent less than five hours on AI training, and 30% reporting no exposure to AI training resources.

• Younger generations, especially Gen Z and Millennials, are more inclined to explore AI and self-identify as AI experts, with new workforce entrants twice as likely to consider themselves AI-savvy compared to their older counterparts.

MORE IN AI THIS WEEK

• I’m a neurology ICU nurse. The creep of AI in our hospitals terrifies me

• Is “AI welfare” the new frontier in ethics? Anthropic's new hire is preparing for a future where advanced AI models may experience suffering

• Vatican, Microsoft create AI-generated St. Peter’s Basilica to allow virtual visits, log damage

• Portrait of Alan Turing made by an AI robot sells for $1m

• OpenAI, Google and Anthropic are struggling to build more advanced AI

• Wendy’s needs Palantir’s AI to handle its $1 Frosty demand

• Jerry Garcia's estate announced a partnership with ElevenLabs, bringing the late Grateful Dead icon's AI-recreated voice to audio content

• New secret math benchmark stumps AI models and PhDs alike

• Online education company Chegg is on its last legs after ChatGPT sent its stock down 99%

• The Beatles’ ‘Now and Then’ makes history as first AI-assisted song to earn Grammy nomination

• OpenAI defeats news outlets' copyright lawsuit over AI training, for now

When Beyoncé Gets Paid, So Could You

JKBX (pronounced “Jukebox”) lets you invest in royalty shares tied to real revenue streams. This isn’t crypto or real estate—it’s the songs people stream, hum, and love every day.

You can potentially earn quarterly income as music royalties flow from platforms like Spotify, YouTube, and TikTok.

What’s unique? Music doesn’t crash. It just plays.

Add a New Note to Your Portfolio

Visit www.jkbx.com/legal/offering-circulars for important Reg A disclosures. This content is not investment advice, nor is it an offer of securities. All investments involve risk and may result in loss.

TECHNICAL NEWS, DEVELOPMENT, RESEARCH & OPEN SOURCE

• AI code completion platform Codeium released Windsurf Editor, the first-ever agentic IDE that writes, edits, and runs your code

• Alibaba’s Qwen2.5-Coder series: current state-of-the-art (GPT-4o+ level benchmarks)

• Microsoft’s TinyTroupe: LLM-powered multiagent persona simulation for imagination enhancement and business insights

• Google research: generating zero-shot personalized portraits

• Supermaven is joining Cursor to build the best AI code editor

• Google DeepMind open-sources its Nobel Prize-winning AlphaFold 3 protein prediction model

• Nous Research just introduced the Forge Reasoning API Beta - system dramatically enhances LLM capabilities through advanced reasoning techniques

  That’s all for this week! We’ll see you next Thursday.