Large Language Models (LLMs) face challenges in generating highly structured outputs accurately. Grammar-Aligned Decoding techniques mitigate these challenges but may distort the LLM’s output likelihoods. ASAp algorithm ensures grammatical outputs aligned with the LLM’s distribution by approximating future grammaticality of output prefixes. ASAp outperforms existing techniques in code generation and structured NLP tasks, enforcing grammatical constraints effectively.