CloakLM: New Defense Against AI Model Theft from GPU Memory

Researchers have introduced CloakLM, a software-based security framework designed to protect large language models from exfiltration attacks when deployed on shared or third-party GPU infrastructure. The work addresses a significant vulnerability in current model-serving deployments: while service providers control the software stack running on virtual machines or bare-metal servers, they lack control over the underlying hardware substrate, including GPU interconnects and neighboring infrastructure components.

The threat is substantial and multifaceted. Previous research has demonstrated multiple attack vectors: Hermes can reconstruct deep neural networks losslessly by observing PCIe traffic passively, while TunnelS extracts high-bandwidth memory (HBM) contents through driver-level access without disrupting inference. Additionally, co-tenant virtual machines can access memory-mapped interfaces or misconfigured RDMA regions without requiring physical proximity. These attacks succeed because machine learning systems typically store model weights in large, contiguous memory regions that are repeatedly accessed—making intercepted PCIe transfers and HBM dumps sufficiently detailed to expose model structure and parameters.

CloakLM employs a three-pronged obfuscation strategy: PCIe traffic shaping to mask data patterns, inter- and intra-layer weight shuffling to fragment model structure across memory, and physical HBM page remapping to disrupt contiguity. Critically, the framework operates entirely in software without requiring hardware modifications, and it preserves the logical memory view for authorized execution with negligible performance overhead. Unauthorized observers encounter only fragmented and semantically incoherent memory state.

The framework integrates seamlessly with existing deep learning infrastructure, including vLLM and PyTorch, and complements confidential computing approaches. Evaluation on distributed inference workloads using LLaMA and Qwen foundation models demonstrated near-native performance while substantially increasing resistance to both PCIe snooping and HBM dump attacks. This makes inference-time model exfiltration significantly less practical for adversaries with access to the hardware substrate.