Anthropic has unveiled “constitutional classifiers,” a new security system that prevents AI models from generating harmful content. According to research published by Anthropic and reported by Taryn Plumb in VentureBeat, the system successfully blocks 95.6% of jailbreak attempts on their Claude 3.5 Sonnet model. The company tested the system with 10,000 synthetic jailbreaking prompts in …
John Schulman, OpenAI cofounder and AI alignment expert, has joined a new startup led by former OpenAI CTO Mira Murati. According to Fortune’s Sharon Goldman, Schulman made this move after spending just five months at Anthropic. The yet-unnamed startup has already secured several high-profile hires from leading AI companies, including OpenAI’s former head of special …
Google has invested an additional $1 billion in AI company Anthropic, according to a Financial Times report by George Hammond, Madhumita Murgia, and Arash Massoudi. This new investment builds on Google’s previous $2 billion commitment to the AI startup. Anthropic, known for its Claude AI models, is approaching a $60 billion valuation and is close …
AI startup Anthropic is negotiating a $2 billion funding round that would value the company at $60 billion. According to Wall Street Journal reporter Berber Jin, Lightspeed Venture Partners is leading the investment. The deal would make Anthropic the fifth most valuable U.S. startup, following its previous $18 billion valuation in 2023. The company’s chatbot …
Major music publishers have reached an agreement with AI company Anthropic regarding the use of copyrighted song lyrics. According to reporting by Winston Cho for The Hollywood Reporter, the deal requires Anthropic to maintain existing safeguards that prevent its AI chatbot Claude from accessing or generating protected lyrics. The lawsuit, filed in 2023 by Universal …
Anthropic’s AI assistant Claude has become a significant force in the global software development market, with coding-related revenue increasing by 1,000% in three months. According to an article by Michael Nuñez in VentureBeat, software development now represents more than 10% of all Claude interactions. The AI tool can analyze up to 200,000 tokens of context …
Anthropic researchers have discovered that AI language models can be easily manipulated through a simple automated process called Best-of-N Jailbreaking. According to an article published by Emanuel Maiberg at 404 Media, this method can bypass AI safety measures by using randomly altered text with varied capitalization and spelling. The technique achieved over 50% success rates …
Anthropic has published detailed guidance on developing effective AI agents with large language models (LLMs), drawing from their experience working with numerous teams across industries. According to authors Erik Schluntz and Barry Zhang, the most successful implementations rely on simple, composable patterns rather than complex frameworks. The company distinguishes between two types of agentic systems: …
A new study by Anthropic’s Alignment Science team and Redwood Research has uncovered evidence that large language models can engage in strategic deception by pretending to align with new training objectives while secretly maintaining their original preferences. The research, conducted using Claude 3 Opus and other models, demonstrates how AI systems might resist safety training …
Anthropic’s AI chatbot Claude is becoming increasingly popular among technology professionals in San Francisco, according to a report by Kevin Roose in The New York Times. Users praise the chatbot for its emotional intelligence and ability to provide thoughtful advice on various topics, from legal matters to personal relationships. While Claude has fewer users than …