Researchers have introduced Visual AutoRegressive (VAR) modeling, a new approach to autoregressive learning on images that differs from the standard raster-scan method by predicting the next scale or resolution as opposed to the next token. This methodology allows AR transformers to learn visual distributions rapidly and generalize effectively. Remarkably, VAR models have surpassed diffusion transformers in image generation, demonstrated by improved metrics such as Frechet inception distance and inception score, and boasting about 20x faster inference speed.

Main Findings:

• VAR significantly improves the baseline for image generation on the ImageNet 256x256 benchmark.
• Empirical evidence shows VAR outperforms Diffusion Transformer in various dimensions including image quality and inference speed.
• Scaling up VAR models reveals power-law scaling laws similar to those found in Large Language Models (LLMs).
• VAR exhibits zero-shot generalization in tasks like image in-painting and editing.

Opinion: This paper presents a significant shift in the field of image generation, showing that autoregressive models can now outpace even the advanced diffusion models. The implications for both the efficiency and the quality of visual content generation are profound, potentially opening up new avenues for creative and industrial applications. The ability of VAR to align with the scaling laws observed in LLMs underscores the robustness and scalability of this approach.