Mobile devices increasingly host large language models (LLMs) directly to safeguard user data privacy. This study proposes a new mobile AI paradigm, referred to as LLM as a system service (LLMaaS). Key innovations include:

• Tolerance-Aware Compression: Adjusts data compression based on how it affects system accuracy.
• IO-Recompute Pipelined Loading: Combines data loading and recalculating processes to speed up data retrieval.
• Chunk Lifecycle Management: Implements a management strategy for memory components to enhance performance.

This approach significantly lowers switching latency by magnitudes, making mobile applications both faster and more secure. Such advancements are pivotal as they offer a glimpse into the future where mobile computing meets the demands of advanced AI, suggesting more localized, privacy-focused solutions.

Relevance:

• Enhances mobile AI applications with reduced reliance on remote servers.
• Opens new avenues for privacy-preserving, low-latency mobile computing.