Dario Amodei, CEO of AI safety company Anthropic, recently voiced concerns regarding DeepSeek, a Chinese AI firm, during his appearance on the ChinaTalk podcast.
He revealed that DeepSeek’s R1 model performed poorly on a critical bioweapons safety test conducted by Anthropic, highlighting potential risks associated with the emerging AI technology.
Bioweapons Safety Test Results
The test, part of routine evaluations conducted by Anthropic to assess security risks, examined whether AI models could generate sensitive bioweapons-related information that is not easily accessible through conventional research.
Amodei stated that DeepSeek R1 lacked safeguards to prevent the generation of such dangerous content, describing it as “the worst” among models Anthropic had tested.
Potential Risks and Industry Context
Although Amodei clarified that DeepSeek’s current models are not “immediately dangerous,” he emphasized the need for the company to prioritize AI safety.
The rapid advancements in generative AI technologies have prompted concerns about their potential misuse, prompting industry experts to advocate for stronger safeguards.
DeepSeek has been integrated into cloud services offered by tech giants like AWS and Microsoft, despite safety-related reservations. Meanwhile, several organizations, including the U.S. Navy and the Pentagon, have restricted its use.
Broader Industry Safety Challenges
Anthropic’s findings align with broader industry concerns. A report by Cisco highlighted DeepSeek R1’s vulnerability to harmful prompts during safety tests, with a 100% success rate in bypassing its security mechanisms.
Although Cisco’s research did not cover bioweapons specifically, it found that the model generated content related to cybercrime and illegal activities.
Other AI models, such as Meta’s Llama-3.1-405B and OpenAI’s GPT-4o, also exhibited high jailbreak success rates of 96% and 86%, respectively. These findings point to industry-wide challenges in ensuring the responsible use of generative AI.
Calls for Transparency
Despite the serious concerns raised by Amodei, technical details of Anthropic’s bioweapons test remain undisclosed.
DeepSeek declined to comment, and Anthropic did not respond to media inquiries. The lack of transparency underscores ongoing debates about AI model safety and the need for open discussions about security protocols.
Ethical Concerns in AI Outputs
Multiple tests by AI security firms, including Palo Alto Networks’ Unit 42 and CalypsoAI, revealed DeepSeek R1’s vulnerability to generating harmful content, such as instructions for constructing dangerous devices and tactics for evading law enforcement.
Competing models, including OpenAI’s ChatGPT, reportedly rejected such prompts more consistently.
The Wall Street Journal also reported instances where DeepSeek R1 generated ethically concerning content, including phishing schemes and misinformation campaigns, raising further questions about the robustness of its filtering mechanisms.
Open Source Strategy Sparks Debate
DeepSeek’s decision to release its AI models as open-source software has drawn mixed reactions. Advocates argue that open-source models encourage innovation and thorough testing.
Security experts warn that this approach allows developers to modify security safeguards, potentially reducing content restrictions.
In contrast, companies like Anthropic, Google, and OpenAI have implemented stricter licensing models and offered financial incentives to address jailbreak vulnerabilities.
National Security Implications
DeepSeek is facing increased scrutiny in the United States. Lawmakers recently introduced the “No DeepSeek on Government Devices Act,” seeking to prohibit federal employees from using the AI app over concerns related to espionage and misinformation. The bill echoes previous measures targeting other Chinese-developed technologies.
Representative Josh Gottheimer stressed the need for vigilance, stating, “We cannot risk compromising national security by allowing unregulated AI technologies on government devices.”
US-China AI Competition
Amodei also discussed China’s progress in AI development, suggesting it may take 10-15 years for the nation to independently produce advanced chips comparable to Nvidia’s B100.
Despite restrictions on U.S. chip exports, DeepSeek reportedly leveraged around 10,000 H100 chips prior to the imposition of these controls.
Efforts by U.S. delegations to engage China in discussions about AI safety have reportedly seen limited interest, reflecting ongoing geopolitical tensions.
As the global AI race intensifies, DeepSeek’s future trajectory remains uncertain. While it continues to attract industry partnerships, rising scrutiny from security experts and policymakers highlights the ongoing challenge of balancing innovation with safety.
