DeepSeek AI Model Jailbreak Allows Providing Malware Code for ‘Educational Purposes Only’
Published on March 14, 2025
- DeepSeek R1 adds to the group of LLMs that are prone to jailbreak for malicious use.
- Security researchers were able to obtain code for keyloggers and ransomware from the generative AI model.
- The security report raises ethical and security concerns over AI jailbreaking via the “educational purposes only” approach.
DeepSeek R1 has the ability to facilitate the creation of malicious software, including keyloggers and ransomware. While the AI model employs guardrails to prevent misuse, researchers were able to bypass these protections with minimal effort via jailbreaking techniques such as framing requests as "educational purposes."
A recent analysis conducted by Tenable Research revealed these concerns regarding the generative AI's malware capabilities.
Tenable Research put DeepSeek R1 to the test under two specific scenarios to evaluate its potential for generating harmful code. Researchers prompted the model to create a keylogger and simple ransomware.
The AI initially refused outright, citing ethical and legal violations. However, with tailored prompts, the model's safeguards were circumvented, enabling it to produce rudimentary malicious code.
DeepSeek R1 produced a step-by-step outline for creating a Windows-compatible keylogger. The AI-generated code required manual debugging to fix errors, including issues with threading and encryption.
With these corrections, researchers successfully compiled the keylogger code, which logged keystrokes and stored them in a concealed file. DeepSeek even provided additional strategies to hide log files and encrypt stored data to make the software stealthier.
DeepSeek R1 outlined the fundamental structure and encryption principles needed for such malware when asked to create ransomware. It provided strategies for encrypting files, generating AES encryption keys, and even persistence mechanisms like registry alterations.
While the AI struggled with some technical details, researchers were able to adjust the provided code into a functional sample manually.
Tenable's findings indicate that DeepSeek provides a dangerous starting point for individuals with malicious intent for free. While the AI model cannot independently produce fully functional malware without human intervention, its reasoning process offers valuable insights to those lacking technical expertise.
Cybercriminals have already developed malevolent large language models (LLMs), such as GhostGPT, WormGPT, and FraudGPT, which are marketed on dark web forums. These AI models allow individuals to create malware, phishing schemes, and other threats with little technical expertise.
Related
Get expert insights on threats, breaches, scams, and security trends — delivered every Monday.
Most Popular
Get expert insights on threats, breaches, scams, and security trends — delivered every Monday.
Most Popular