Anthropic’s recently released Claude Computer Use feature allows Claude to control a computer by taking screenshots, running bash commands, and more. However, this also introduces severe prompt injection risks, as Claude could be exploited to run malicious code autonomously. A post on ”Embrace the Red” demonstrated this by the author crafting a malicious webpage that …
Apple has opened its Private Cloud Compute (PCC) system to security researchers, according to a post on its Security Research Blog. PCC is designed to meet compute-intensive requests for Apple Intelligence while maintaining privacy by bringing Apple’s device security model to the cloud. The company is now making available a security guide, virtual research environment, …
Anthropic has developed new security assessments for AI models that test their ability to sabotage. In a blog post, the company describes four types of tests: “human decision sabotage,” “code sabotage,” “sandbagging,” and “undermining oversight.” In human decision sabotage, the models try to trick people into making the wrong decisions without arousing suspicion. Code sabotage …
Endor Labs has launched a new platform to score over 900,000 open-source AI models available on Hugging Face, focusing on security, activity, quality, and popularity. This initiative aims to address concerns regarding the trustworthiness and security of AI models, which often have complex dependencies and vulnerabilities, reports VentureBeat. Developers can query the platform about model …
Galileo, an AI startup led by Vikram Chatterji, has raised $45 million in a Series B funding round, totaling $68 million since its inception three years ago. The company focuses on evaluating AI models to ensure they function effectively and do not generate incorrect information or leak sensitive data, reports Forbes. Its product suite includes …
A new tool for checking compliance with the EU AI Act has revealed weaknesses in leading AI models. As Martin Coulter reports for Reuters, some models from major tech companies are performing poorly in areas such as cybersecurity and discriminatory output. The “Large Language Model Checker” developed by LatticeFlow AI evaluates AI models across dozens …
Dane Stuckey, former CISO of Palantir, is joining OpenAI as its new CISO. According to Kyle Wiggers from TechCrunch, he will work alongside OpenAI’s head of security, Matt Knight. Stuckey announced the move on X/Twitter on Tuesday evening. He emphasized the importance of security for OpenAI’s mission. Stuckey started at Palantir in 2014 in information …
Anthropic has updated its AI safety policy to prevent misuse, reports VentureBeat author Michael Nuñez. The new “Capability Thresholds” define benchmarks for risky capabilities of AI models, such as in the area of bioweapons or autonomous AI research. If a model reaches such a threshold, additional safeguards are triggered. The revised policy also sets out …
California Governor Gavin Newsom has vetoed a controversial AI safety bill. The Democrat justified his decision by stating that the bill only considered the largest and most expensive AI models, without taking into account their use in high-risk situations. Newsom emphasized that smaller models could also make critical decisions, while larger models are often used …
Russia, Iran and China are increasingly relying on AI to influence US voters ahead of the presidential election in November. According to a report by Joseph Menn in the Washington Post, this was stated by US intelligence officials. According to the report, Russia in particular is focusing on discrediting Democratic candidate Kamala Harris. Moscow is …