A post-quantum cryptography algorithm that was being evaluated for possible adoption as a US standard has been withdrawn after Anthropic’s AI security model identified a weakness that significantly reduced its effective security.
The algorithm, known as HAWK, is a digital signature system designed to protect communications and data against attacks from future quantum computers. It had previously cleared two rounds of evaluation conducted by the National Institute of Standards and Technology (NIST) as part of its assessment of post-quantum cryptographic (PQC) algorithms. HAWK was undergoing a third round of testing when Anthropic’s Mythos security model uncovered a previously unknown attack method.
After Anthropic announced its findings on Monday, HAWK’s developer confirmed on Tuesday that the algorithm was being withdrawn from consideration.
Anthropic said its Mythos AI model had identified weaknesses in the mathematical foundations behind HAWK and had also improved an existing attack technique targeting the widely deployed AES encryption standard.
However, the findings need to be viewed in context. The discoveries do not compromise cryptographic systems currently used to protect data. Instead, they demonstrate techniques that can reduce the computational effort required to attack certain weakened versions of cryptographic systems under controlled testing conditions.
There are several other limitations. The systems examined were deliberately weakened “challenge instances” created for security researchers to test and review. Production implementations are significantly stronger. In addition, the fundamental mathematical primitives underpinning both systems remain considered secure. The attacks also require resources that would make them impractical in real-world environments.
HAWK Withdrawn After AI-Assisted Attack
Anthropic reported that Mythos was able to improve the strongest previously known attack against HAWK after approximately 60 hours of work and around $100,000 in computing costs. The researcher guiding the system reportedly did not have specialist cryptography expertise.
HAWK's security is based on the Lattice Isomorphism Problem, a mathematical problem believed to offer protection against quantum attacks. Existing classical attacks attempt to identify mathematical symmetries known as automorphisms.
Mythos generated a previously unknown approach for locating those symmetries, effectively reducing HAWK's key strength by half.
Although increasing the key size could address the weakness, doing so would introduce additional computational overhead and make HAWK less attractive compared with other post-quantum signature systems.
Matthew Green, a cryptography expert and professor at Johns Hopkins University, said the significance of the discovery came from combining several established techniques in an unexpected way.
“What’s particularly concerning (and so especially ripe for AI) is that the attack does not invent fundamentally new mathematics,” Green wrote. “It simply extends a bunch of tools that were lying around and well-known, and gets a good result.”
Anthropic explained that Mythos worked with limited human intervention and used extensive literature research, mathematical analysis and computational testing before producing an end-to-end verification process for the attack.
“To find the attack, Claude Mythos Preview worked semi-autonomously in an agentic harness, with occasional human guidance and nontechnical direction. Mythos found the attack after an extensive literature review to understand the state of the art, and substantial mathematical reasoning and computational experiments. After finding the attack, Mythos implemented an end-to-end verification pipeline to convince itself—and the human operator—of the attack’s correctness.”
The company said two separate AI agents initially worked independently on the problem. While one rejected the proposed approach, the other found a way to make it viable. The systems eventually converged on the conclusion that the improved attack worked.
Sophie Schmieg, a post-quantum cryptography specialist at Google, noted that researchers had already suspected HAWK could have weaknesses. However, the newly demonstrated reduction in effective key strength made the algorithm less competitive with alternatives such as ML-DSA and FN-DSA.
“Basically with this paper, HAWK is dead,” she wrote.
AI Also Improves an AES Attack
The second discovery involving AES was less consequential but still demonstrated how AI could assist cryptanalysis.
Mythos helped improve a “meet-in-the-middle” attack, a technique that attempts to recover encryption keys by analyzing large quantities of known plaintext and corresponding ciphertext.
The previous best-known approach required approximately 2^105 plaintext inputs, making it impractical. Anthropic said Mythos developed an improved technique using a Möbius Bridge, a more advanced fingerprinting approach.
According to Green, the resulting code reduced the
[…]
Content was trimmed to protect the source. Please visit the original article for the full text.
Read the original article: