Introduction

The extensive computational and memory demands of Large Language Models necessitate advancements in inference efficiency, especially in environments with limited resources. This survey delves deep into the causes of inefficiencies and presents a comprehensive review of tactics aimed to enhance the operation of these powerful models without compromising their capabilities.

Key Points

• Identification of key factors causing inefficiencies in large model operations, such as attention mechanisms and auto-regressive decoding.

• Presentation of a thorough taxonomy of existing optimization strategies at different levels: data, model, and system.

• Comparative analysis showing improvements from various optimization techniques.

Conclusion

The ongoing efforts to enhance LLM inference underscore the importance of efficiency in deploying AI technologies in resource-limited scenarios. This comprehensive survey serves as a guide for researchers and practitioners aiming to optimize LLM performance while considering practical deployment constraints.