The week in AI: Elon Musk's Neuralink performs first human brain implant surgery

Sponsored by

Welcome to The Dispatch! We are the newsletter that keeps you informed about AI. Each Thursday, we aggregate the major developments in artificial intelligence; we pass along the news, useful resources, tools and services, and highlight the top research in the field as well as exciting developments in open source. Even if you aren’t an engineer, we’ll keep you in touch with what’s going on in AI.

This week’s issue comes a day early as we’ll be out of the office for the next few days! We’re also in the process of evaluating our format to maximize valuable content. Starting this week we’ll be adding accompanying breakdowns to our linked technical articles, research, and open source developments near the bottom of the newsletter. If you have a friend or colleague who is interested in the AI space, please forward The Dispatch along - and thanks for reading!

NEWS & OPINION

Elon Musk’s Neuralink conducts first surgical human brain implant

-------------------------

An unidentified person has received Neuralink’s first human brain implant, according to owner/founder Elon Musk. Musk stated that the patient is recovering well, adding that initial results show promising cellular activity between the patient’s brain and nervous systems. Further comments from Musk indicated that the implant device will be named Telepathy. It’s designed to be the first implant device to enable wireless brain-to-device communication - hopefully allowing individuals with severe mobility impairments (quadriplegia, ALS, etc.) to gain control over digital devices through thought alone.

Despite the promise of the technology, Neuralink has faced scrutiny over its research practices and the ethical implications of merging human cognition with AI. The company went viral in 2021 when it showcased a macaque playing “MindPong” with an implanted Neuralink device (the implant takes over at 1:36 in the video).

Meta’s upgraded, free Code Llama 70B model rivals GPT-4 and Google Gemini for coding prowess

-------------------------

On Monday, Meta announced a new update (note: the August 24 date prominently displayed is from the initial Code Llama release; scroll down to see the update) to their open source programming-centric model, Code Llama, that can write code in Python, Java/Script, C++, etc. from natural language prompts or existing code snippets. The ability to write and analyze code is emerging as one of the most important uses of language models today. Here are our notes on the updated Code Llama version, 70B:

• How good is it? The new model scored a 67.8 on HumanEval Pass@1, on par with GPT-4 (67) and Google Gemini Pro’s (67.7) initial Pass@1 results. The HumanEval benchmark ranks models based on a practical assessment of the model's ability to solve coding problems. Benchmarking these models accurately is difficult and comes with a host of issues - so the evaluation is a ‘best-available’, but imperfect indicator of coding ability.
  
  One of the highest-ranking open source coding models, Phind, was fine-tuned on Code Llama 34B, the version released 5 months ago by Meta that only scored a 48.8 on HumanEval Pass@1. Code Llama 70B is a big improvement over that as a foundation model, so we’re excited to see what the open source community can build on it.

• So it’s free and available for commercial use? Yes, with some particulars that distinguish it from being ‘truly’ open source: unlike true open source, in order to use the Llama model family you must request access to use them by accepting Meta’s licensing agreement. The license is not overly restrictive (unless your commercial use case happens to garner over 700 million monthly users), but does challenge the traditional concept of open source.

• Can I use it if I know nothing about coding? Can this and other similarly powerful coding models replace working programmers? Yes, and no. One of the most exciting aspects of these models is the ability to provide guidance and feedback based on natural language instructions. They don’t just generate or diagnose code - they can act as a dedicated tutor/co-pilot. But they aren’t perfect or even remotely autonomous; they’re a tool. All of these models still very much need a deft human supervisor for real and business-world applicable development tasks.

• What’s the context window? There’s still some confusion about this based on Meta’s documentation. We believe the Code Llama 70B model itself has a 16k token context window, while the two specialized 70B models (for Python/Instruct) have a 4k window. That latter window is unfortunately quite small for coding purposes, so hopefully improvements are coming quickly down the context window pipeline!

Foreign access to American cloud computing for AI training under scrutiny as US continues high-stakes AI/tech war with China

-------------------------

The US government is introducing a proposal to potentially block foreign entities, particularly tech companies from China, from using American cloud computing for AI model training. The primary component of this initiative is called ‘Know Your Customer,’ and it requires US cloud companies to identify their foreign users rigorously. ‘KYC’ would impose significant responsibilities on cloud computing firms, forcing them to verify foreign customers’ identity, maintain user identification standards, and certify their compliance annually.

The proposal is a strategy to ensure that US cloud platforms are not exploited for potentially hostile AI development more broadly - but it’s also the latest volley in a sustained barrage of measures aimed at curbing China's burgeoning AI/technology prowess.

More Google privacy concerns ahead of Bard’s integration with Android phones

-------------------------

Google's rumored upcoming integration of its large language model, Bard, into the ‘Messages’ app on Android smartphones is sparking significant privacy concerns. Bard for Messages will understand conversation contexts, tones, interests, and relationship dynamics. It will be able to do this based on reading and analyzing your private messages - dating back forever. These will be sent to the cloud for processing, used for training and may be seen by humans (albeit anonymized). This data will be stored for 18-months, and will persist for a few days even if you disable/opt out from the AI.

Contrary to this approach, Apple emphasizes on-device processing for all of its AI applications, limiting the exposure of user data to external servers. This could become yet another wrinkle in the longstanding iPhone vs. Android battlefield. Google has a poor track record in data privacy.

• What can be done with AI in 59 seconds: an oportunity and a crisis

• ChatGPT is violating Europe’s privacy laws, Italian data protection authority tells OpenAI

• Robots are fighting robots in Russia’s war against Ukraine

• The Taylor Swift deepfake debacle was frustratingly preventable

• The hottest job in corporate America? Executive in Charge of AI

• Microsoft and Alphabet earnings reports are in; Apple, Amazon and Meta report Thursday

• Tech layoffs not tied to economic struggles, but AI investments

• Following lawsuit, rep admits “AI” George Carlin was human-written

• Morpheus-1: the AI system that wants to help you control your dreams

Your info is on the dark web

Every day, data brokers profit from your sensitive info—phone number, DOB, SSN—selling it to the highest bidder. And who’s buying it? Best case: companies target you with ads. Worst case: scammers and identity thieves.

It's time you check out Incogni. It scrubs your personal data from the web, confronting the world’s data brokers on your behalf. And unlike other services, Incogni helps remove your sensitive information from all broker types, including those tricky People Search Sites.

Help protect yourself from identity theft, spam calls, ID theft and health insurers raising your rates. Plus, just for our readers: Get 55% off Incogni using code PRIVACY.

TECHNICAL, RESEARCH & OPEN SOURCE

OpenAI announces new embedding models, API usage management tools, and reduced prices

-------------------------

OpenAI has announced a slew of updates to its models and upgraded developer tools:

• GPT-4 Turbo preview: The updated gpt-4-0125-preview focuses on improving task completion, specifically addressing issues of "laziness" in the model's responses. It also resolves a bug affecting non-English UTF-8 generations. OpenAI also plans to launch GPT-4 Vision for general availability in the coming months.

• GPT-3.5 Turbo update: The new gpt-3.5-turbo-0125 model will feature a 50% reduction in input prices - to $0.0005 per 1,000 tokens and a 25% reduction in output prices to $0.0015 per 1,000 tokens. The new model also includes enhancements for higher accuracy in requested formats and fixes a text encoding issue for non-English language function calls.

• Two new embedding models: text-embedding-3-small and text-embedding-3-large offer a notable performance upgrade, The small model is not only better than the previous generation (ada-002), it’s much cheaper - with prices 5x lower than previous gen at $0.00002 per 1,000 tokens. The larger model creates embeddings up to 3072 dimensions, with even better performance, but priced at $0.00013 per 1,000 tokens.

• New API management tools for devs: First, developers can now customize permissions for API keys directly from the API keys page, allowing for more tailored access control, such as read-only permissions for specific applications or access limitations to certain endpoints. Second, the enhanced usage dashboard and export function now provide detailed API key-level metrics, enabling developers to easily monitor and analyze usage data for different features, teams, products, or projects. OpenAI will further refine these tools around API usage and key management in the future!

The Strategy Deck has a great overview on how OpenAI is designing and expanding its ecosystem for mainstream businesses in AI, and the opportunities and challenges ahead in 2024.

Accessing Perplexity AI’s online LLM’s programmatically via API

-------------------------

We’ve been using Perplexity AI for search at The Dispatch since its inception in early 2023; it’s a relatively quiet but important player in the AI space. One special function of Perplexity’s API is that it can retrieve real-time information because it is online. Perplexity is intended for search, not reasoning (although the 70B API is much better at the latter), and the online LLM is not capable of multi-turn conversations like we’re used to in most LLM chats. But the API can be used to build RAG-based apps that work with OpenAI, and frameworks like LangChain and LlamaIndex are highly compatible.

Vectors as AI data primitives

-------------------------

As AWS CEO Adam Selipsky puts it: “Customers are finding that different models actually work better for different use cases or on different sets of data. Some models are great for summarization, others are good for reasoning and integration, and still others have awesome language support.” Organizations now often require infrastructure that allows them to switch between different models quickly and even combine them for specific use cases. The growing importance of unstructured data, coupled with the emerging capability of these various models, means we need new architectures and methods to work with data.

That’s where Pinecone fits in. Vector embeddings are a way of representing data that AI models can understand. Traditional databases are not equipped to handle vector embeddings efficiently, leading to issues with scalability and performance. There is a growing need for new infrastructure and databases capable of handling vector embeddings and the unique demands of these technologies. Pinecone represents a solution specifically designed for this new era of data management.

Meta-prompting: a new scaffolding technique to enhance the functionality of language models

-------------------------

Researchers from Stanford and OpenAI have developed a new technique called "meta prompting" to enhance the capabilities of language models. Meta prompting allows a single LM to be transformed into a multi-functional entity by breaking down complex tasks into smaller, manageable subtasks. These subtasks are then assigned to different instances of the same LM, with each instance acting as an "expert" in a specific area. The LM serves as a conductor, managing the experts and integrating their outputs to produce a comprehensive final response.

This approach enables a single LM to simultaneously function as an orchestrator and diverse panel of specialists. Meta prompting has shown efficacy across tasks like mathematical puzzles and programming challenges. One of the standout features of meta-prompting is its task-agnostic nature. Unlike traditional methods that require detailed, task-specific instructions, this new approach utilizes a universal set of high-level instructions - simplifying the user experience substantially. The research touches closely on GPT-4’s long-rumored Mixture of Experts architecture.

MORE IN T/R/OS

• InternLM-XComposer2: mastering free-form text-image composition and comprehension in a vision-language large model

• Hugging Face and Google partner for open AI collaboration

• Sharper LLMs: enhancing math and reasoning abilities with Abacus AI

• The promise and challenges of crypto + AI applications

• More than just chatbots: using context to build the future of in-app AI experiences

That’s it for the week! We’ll see you next Thursday.