Study Reveals LLM Limitations in Cryptographic Protocol Compliance Testing

Researchers have evaluated the effectiveness of large language models (LLMs) combined with grammar-level mutation techniques for automated compliance testing of cryptographic protocols. The study, submitted to arXiv on 16 June 2026, focuses on PKCS#1 v1.5 signature verification—a widely deployed standard using Type-Length-Value (TLV) encoding—and tests the approach across 48 cryptographic library implementations.

The investigation compares LLM-based code synthesis against traditional testing methods, which rely on purely random generation and primitive mutations that often fail to explore semantically meaningful behaviors in binary protocols. The researchers used a formally verified testing oracle called Morpheus as a baseline for their evaluation. They successfully reproduced 10 of 13 previously identified specification violation categories, including all 5 signature forgery categories, and discovered 1 previously unreported discrepancy.

However, the study identifies critical limitations in the LLM approach. LLM hallucination—where the model generates plausible-sounding but incorrect code—occurred in 82.5% of generated scripts and emerged as the primary factor limiting effectiveness, rather than deficiencies in mutation strategies themselves. The researchers identified five distinct types of hallucination with varying distributions across mutation categories: structural mutations achieved only 13.3% fidelity in correct implementation, while constraint mutations reached 30.3% correctness but exhibited the highest rate of completely ignored mutations at 8.1%.

The findings reveal a significant gap between operational reliability and semantic fidelity in LLM-based code synthesis. While the systems demonstrated 99.8% operational reliability, semantic fidelity—the correctness of the generated test logic—reached only 17.5%. These results provide practical guidance on the trustworthiness of LLM-based code synthesis in specification-driven testing pipelines and highlight when such automation can be reliably deployed versus when human expertise remains essential.