This extensive survey reviews the state of continual learning (CL) within the context of Large Language Models. The main takeaways include:

• Overview of continual learning methods in LLMs, focusing on vertical and horizontal continuity.
• Describes the stages involved like CPT, DAP, and CFT.
• Evaluative protocols and data sources used in this research field are also discussed.

Key Insights:

• The challenge of integrating pre-trained LLMs to continual learning setups is considerable.
• Insights from this survey help in understanding how LLMs can be better tailored for specific applications.

Opinion: This survey provides a solid foundation for future innovations, offering in-depth insights into making LLMs flexible for changing tasks and data environments. It highlights the critical research areas to focus on for enhancing the adaptability of AI systems.