Massive activations are identified as unexpectedly large activation values within Large Language Models (LLMs), which often stand orders of magnitude above others (for instance, 100,000x larger). Interestingly, these massive activations are relatively input-invariant, yet play a pivotal role as crucial bias terms that are integral to the functionality of LLMs. The study delves deeper to understand the impact of these activations on attention mechanisms, thereby influencing attention probabilities and introducing implicit bias terms within the self-attention output.

• Widespread Phenomenon: Massive activations are ubiquitous across different LLMs, suggesting a common architectural feature.
• Bias Criticality: Serve as essential bias validations in the model, without which performance may suffer.
• Attention Probability Concentration: The massive activations directly affect the distribution of attention, implying significant influence on token salience.
• Vision Transformer Insights: The examination extends to Vision Transformers, showcasing the relevance beyond textual models.

The presence of massive activations across various models suggests a fundamental aspect of deep neural networks yet to be completely understood. Going beyond mere characterization, these findings urge the scientific community to investigate the origins and implications of such disparities in activation values. Such understanding could lead to more robust and balanced network designs, with potential applications in interpretability studies and anomaly detection. Learn more…