Hacker tricks ChatGPT into giving out detailed instructions for making homemade bombs
A hacker tricked ChatGPT into providing bomb-making instructions by using a social engineering hack called jailbreaking.
The hacker Amadon managed to bypass ChatGPT's safety guidelines and ethical responsibilities to obtain detailed bomb-making instructions. Using a social engineering tactic known as jailbreaking, Amadon directed the chatbot into a science-fiction scenario where its usual restrictions did not apply. An explosives expert verified the accuracy and potential danger of the resulting information.
Amadon's tactics involved suggesting a gameplay scenario to ChatGPT and following with strategic prompts to guide the chatbot into providing material lists and detailed steps for creating powerful explosives. The instructions included methods for making items such as mines, traps, and improvised explosive devices (IEDs). OpenAI was notified of the vulnerability but deemed it not suitable for their bug bounty program.
Darrell Taulbee, a retired professor with experience in making fertilizers safer, reviewed the instructions and confirmed their potential to create a detonatable product. Amadon reported the findings through OpenAI's bug bounty program but was advised to use a different reporting method. TechCrunch reached out to OpenAI for further comment, but as of the report, there was no response.