Abstract:

Paramanu-Ganita is a novel Auto Regressive (AR) decoder based language model developed for specialized mathematical tasks. Despite being significantly smaller than its mainstream counterparts, this model demonstrates excellent performance, outperforming even larger, more established LLMs.

• Model Size: 208 million parameters
• Context Size: 4096
• Evaluation: Powerful performance on GSM8k; surpasses larger LLM models
• Training: 146 hours on A100, efficient compared to larger models
• Future Prospects: The paper highlights the potential for further development by implementing additional parts of the mathematical corpus.

Importance:

Paramanu-Ganita redefines the scalability and efficacy anticipated from high-performing LLMs, particularly in domain-specific applications like mathematics. Its success points to a shifting paradigm where smaller, more tailored models may outperform generalized giant models not only in effectiveness but also in efficiency.