The week in AI: Anthropic makes the world's most powerful AI free to use

In partnership with

Welcome to The Dispatch! We are the newsletter that keeps you informed about AI. Each Thursday, we aggregate the major developments in artificial intelligence - we pass along the news, useful resources, tools and services; we highlight the top research in the field as well as exciting developments in open source. Even if you aren’t a machine learning engineer, we’ll keep you in touch with the most important developments in AI.

NEWS & OPINION

Anthropic’s Claude: 3.5 Sonnet, “Artifacts”, and “Projects” - everything you need to know about the world’s most powerful publicly available artificial intelligence

-------------------------

It’s time to talk about Claude. In the last week, Anthropic’s flagship LLM (and ChatGPT’s most powerful chatbot competitor) has received a slew of impressive and important updates and upgrades. These changes are cumulatively pointing at Anthropic’s long-term vision for Claude: not just an extremely powerful conversational chatbot and AI assistant, but a collaborative work environment where entire organizations will be able to securely centralize their knowledge, documents, and ongoing work in one shared space. Here’s what you should know, starting with Claude 3.5 Sonnet:

• The upgraded model surpasses rivals like GPT-4o (and Anthropic’s previous top model, Claude Opus) across key benchmarks for reasoning, coding, math, and knowledge abilities - but also with significant speed boosts and cost reductions for the API. Claude 3.5 Sonnet boasts 2x the speed of Opus, while its token pricing comes in at 1/5 the cost. The model also has state-of-the-art vision capabilities.

• For most of those benchmarks, there are only a few percentage points of difference; sometimes Claude is better at a certain task or metric, sometimes GPT-4o is, sometimes (gasp!) Google’s Gemini is, etc. But none of these tests capture some of the truly important things that any good LLM needs - nuance, character, depth, warmth, genuineness, and so on. Claude 3.5 Sonnet has all of these intangibles in spades compared to the majority of competing LLM’s.

• It’s free to use, though output limits are capped within timed windows unless you subscribe.

• Anthropic has stated that 3.5 versions of Haiku (faster and cheaper) and Opus (more compute intensive and powerful) are coming ‘later this year’, along with more new features like Memory.

The major upgrade for 3.5 Sonnet, though, is (in our opinion) the new “Artifacts” feature. Most succinctly, the feature allows you to edit and iterate on a wide variety of content with Claude directly within the chat interface. When Claude creates an Artifact, you'll see the Artifact content displayed in a new dedicated window to the right side of the main chat. Common examples of Artifact content include:

• Documents (Markdown or Plain Text)

• Code snippets

• Websites (single page HTML)

• Scalable Vector Graphics (SVG) images

• Diagrams and flowcharts

• Interactive React components

Here are a few examples of how some users are utilizing Artifacts right now:

• Making a fully functional Blackjack game complete with game logic, a betting system, and a game area with card design

• Simulating physics in a 3D solar system with three.js and cannon.js

• Creating a basic DOOM-like 3D shooter

• Making interactive educational tools on the fly

• Leveraging external API’s to create apps

And for devs: Pietro Schirano unveiled Claude Engineer, an open-source tool enabling 3.5 Sonnet to interact with your local files and create and edit code through a chat interface.

Last up is the “Projects” update. In May, Anthropic announced Claude for Teams. Now, both Pro subscribers and Team users will be able to collaborate with Claude on Projects:

• Projects allows teams to curate relevant documents, code, and insights in a single location. With a 200,000-token context window (equivalent to a 500-page book), you can ground Claude’s outputs in your internal knowledge (style guides, codebases, interview transcripts, past work) and essentially give Claude a crash course in your organization’s expertise.

• Anthropic stated that its internal teams and early-access partners have already utilized the new feature to accomplish a wide variety of tasks. For example, marketers have uploaded style guides in order to generate emails in a specific voice, legal teams have uploaded case law and statutes to help draft legal memos, and investment firms have uploaded market reports and financial statements to predict future stock performance.

• Users can share individual Projects - and a new activity feed helps to collaborate with teammates across conversation snapshots. Additionally, custom instructions (similar to ChatGPT) are also now available for every Project, allowing users to quickly and easily tailor the AI’s responses.

Anthropic CEO Dario Amodei was also featured in a wide-ranging interview with TIME magazine this week. Quite a week for Anthropic and Claude! We hope you get the opportunity to experiment with this outstanding chatbot.

Y Combinator joins over 140 AI startup founders in signing a letter opposing California’s Senate Bill 1047 on AI regulation

-------------------------

In February, California State Senator Scott Wiener proposed Senate Bill 1047, which aims to require significant safety testing for large/frontier AI models in California. The bill primarily focuses on a) establishing safety and security protocols for developers of large AI models, and b) creating a regulatory framework to prevent potentially critical harms that could arise from unchecked AI development. The bill managed to glide through its house of origin without major resistance, but is receiving some major pushback in the Assembly.

Y Combinator (a huge venture capital firm for AI development) signed off on a letter with 140 AI startup founders arguing that the bill could harm California's ability to retain AI talent and remain competitive in the industry. The legislation targets AI "frontier models" costing $100 million or more to train, requiring risk assessments to prevent potential catastrophic harm. While Wiener characterizes the bill as having a "light touch," many tech companies, including those in the letter, strongly disagree.

As the debate has intensified, opponents are raising concerns about liability issues, vague language, and potential inadvertent crackdowns on smaller AI companies. Y Combinator argues that the bill could threaten California's technology economy and undermine competition. Wiener has disputed claims that the bill could lead to prison sentences for developers, emphasizing that it's not intended to criminalize the industry.

Recent polling suggests strong public support for SB 1047, with 77% of likely voters in favor. The bill has already been amended twice this month in Assembly as it moves through the legislative process.

MORE IN AI THIS WEEK

• Bloomberg Special Report: AI is already wreaking havoc on global power systems

• The world's biggest music labels are suing two AI startups over copyright

• Apple says it won't roll out AI features in Europe due to regulatory concerns

• Citigroup released a new GPS report on AI in finance

• NBC to use AI version of announcer Al Michaels’ voice for Olympics recaps

• Swallow this robot: Endiatx’s tiny pill examines your body with cameras, sensors

• AI’s $600B question: the AI bubble is reaching a tipping point

• Top AI companies really want to make their chatbots funnier

• Stability AI stabilized by investment from Silicon Valley royalty, new executive team

• OpenAI acqui-hires Multi (formerly Remotion), a startup focused on remote collaboration and screen sharing tech, and Rockset, a company specializing in real-time data analysis

• Death, taxes, and AI: how generative AI will change accounting

• OpenAI is reportedly set to restrict access to its API and platform in China and other unsupported regions

Learn AI-led Business & startup strategies, tools, & hacks worth a Million Dollars (free AI Masterclass) 🚀

This incredible 3-hour Crash Course on AI & ChatGPT (worth $399) designed for founders & entrepreneurs will help you 10x your business, revenue, team management & more.

It has been taken by 1 Million+ founders & entrepreneurs across the globe, who have been able to:

• Automate 50% of their workflow & scale your business

• Make quick & smarter decisions for their company using AI-led data insights

• Write emails, content & more in seconds using AI

• Solve complex problems, research 10x faster & save 16 hours every week

Register & save your seat now (100 free seats only)

TECHNICAL, RESEARCH & OPEN SOURCE

Etched is launching its custom chip Sohu, specifically designed for transformer models - and it’s an order of magnitude faster than Nvidia's upcoming monster GPU (the GB200)

-------------------------

A major bottleneck in AI development is hardware. GPUs are increasingly expensive and inefficient for a number of AI processes - Meta used 50,000x more compute to train Llama 400B than OpenAI used on GPT-2 in 2019. Continuing this game of scaling with GPU’s alone will be almost unfathomably expensive.

AI startup Etched is stepping in here, with a radical approach to AI hardware – a specialized chip designed specifically for transformers only, the architecture behind most cutting-edge language models. The chip is called Sohu, and it’s the world's first Application-Specific Integrated Circuit (ASIC) designed exclusively for transformer architectures. Some highlights:

• Sohu achieves 500,000 tokens per second throughput for Llama 70B, outpacing current GPUs by an order of magnitude.

• One 8x Sohu server replaces 160 Nvidia H100 GPUs, drastically reducing infrastructure costs - and energy consumption.

• By focusing solely on transformers, Sohu attains 90% FLOPS utilization, tripling the typically inefficient ~30% seen in GPUs.

By specializing in transformer architectures, Sohu could lead to more accessible and responsive AI applications, accelerate AI development cycles, and significantly improve energy efficiency in the face of growing AI workloads.

Sohu's success is not without challenges. The technology's future hinges on the continued dominance of transformer architectures in AI (definitely not a sure thing - transformers are quadratic and inefficient by nature, and researchers are currently developing entirely new architectures for language models), and Etched faces the formidable task of convincing developers and companies to adopt a new hardware platform. Established players like Nvidia are likely to respond with their own innovations.

MORE IN T/R/OS:

• Building a personalized code assistant with open-source LLMs using RAG Fine-tuning

• Optimizing AI inference at Character.AI

• Researchers upend AI status quo by eliminating matrix multiplication in LLMs

• Why we no longer use LangChain for building our AI agents

• DCLM-POOL is the largest public data corpus ever assembled for LLM training

• Poolside is raising $400M+ at a $2B valuation to build a supercharged coding co-pilot

• Introducing Claudette, a new friend that makes Claude 3.5 Sonnet even nicer

That’s all for this week! We’ll see you next Thursday