A new study explores how LLMs can be instructed to generate internal rationales as an intermediate step in producing external text, akin to pausing to think before speaking. Named the Self-Taught Reasoner (STaR), this approach hints at a more general intent of instilling LMs with reasoning capabilities.

Highlights include:

• Quiet-STaR: LMs learn to generate rationales at each token level to better predict future text.
• The model addresses key challenges such as the cost of generation and initial ignorance of rationale creation and use.
• Success is measured by the model’s improved ability to predict difficult tokens and directly answer challenging questions.

The potential of Quiet-STaR in teaching LMs to reason in more scalable and general ways indicates an exciting direction for further research into self-reflective AI. Read the full paper.