TIGER: New Gradient Inversion Attack Threatens Federated Learning Security

Researchers have presented TIGER, a novel gradient inversion attack that poses significant security risks to federated learning systems using transformer models. Federated learning enables multiple clients to collaboratively train machine learning models by transmitting only gradient updates to a central server while keeping raw input data local. However, the new attack demonstrates that these gradient updates can leak sufficient information to reconstruct sensitive client inputs.

Existing gradient inversion attacks on transformers face substantial limitations. Previous approaches either attempt to reconstruct inputs by optimizing dummy data to match true client gradients—a process that is computationally expensive and unstable for large modern models—or exploit the low-rank structure of attention gradients to identify a subspace containing true layer embeddings. The latter methods rely on discrete membership tests to identify candidate tokens, but these tests prove fragile when exposed to numerical noise from quantization or differential privacy defenses, and scale poorly for encoder models with non-causal attention mechanisms.

TIGER addresses these limitations by introducing a continuous optimization approach. Rather than searching across discrete tokens or attempting to match complete gradients, the attack directly optimizes token embeddings to minimize their distance to the identified subspace. This differentiable objective provides a more stable and efficient attack mechanism. Experimental results show that on encoder-only transformer models, TIGER substantially improves both reconstruction quality and runtime compared to existing attacks. For decoder models, TIGER demonstrates greater robustness than prior subspace-based methods, and notably enables the first successful reconstructions against federated learning systems defended with differential privacy.

The research highlights a critical vulnerability in federated learning architectures, particularly for transformer-based models. The ability to reconstruct inputs even from privacy-defended systems suggests that additional security measures may be necessary to protect sensitive data in collaborative machine learning environments. The findings underscore the ongoing tension between the efficiency benefits of federated learning and the privacy risks posed by gradient-based attacks.