The Rise of AI Image Generators: A Deep Dive into the Text to Image Leaderboard

In just two years since the introduction of diffusion-based image generators, AI image models have made remarkable strides, achieving near-photographic quality that continues to captivate users and researchers alike. With the rapid advancement in this field, a pressing question emerges: how do these models compare against each other? Furthermore, are open-source alternatives holding their ground against proprietary giants? The Artificial Analysis Text to Image Leaderboard seeks to provide clarity with human preference-based rankings, showcasing the leading image models like Midjourney, OpenAI’s DALL·E, Stable Diffusion, and Playground AI.

Explore the leaderboard here: Artificial Analysis Text to Image Leaderboard. Moreover, you can actively participate in the Text to Image Arena and receive your personalized model ranking after casting just 30 votes!

Methodology

Evaluating the quality of image models presents unique challenges compared to other AI modalities, such as language models. This difficulty arises from the inherent variability in individual preferences regarding how images should be rendered. Faced with this complexity, early objective metrics have evolved into more nuanced human preference studies, especially as image models reach increasingly high levels of accuracy.

The Image Arena adopts a crowdsourcing approach to gather human preference data on a large scale, allowing for comparisons among major image models for the first time. Each model is assigned an ELO score, derived from regression analysis of all preferences, akin to the methodology used in the Chatbot Arena. In this setup, participants are presented with a prompt alongside two images and must select the one that best reflects the given prompt. To ensure a comprehensive evaluation, over 700 images are generated for each model, covering a wide range of styles and categories such as human portraits, animals, nature, and abstract art.

Early Insights From the Results 👀

• Proprietary vs. Open Source: While proprietary models like Midjourney, Stable Diffusion 3, and DALL·E 3 HD currently dominate the leaderboard, open-source models are gaining momentum. Notably, Playground AI v2.5 has risen to prominence, even outpacing OpenAI’s DALL·E 3 in some comparisons.
• Rapid Advancements: The image generation landscape is evolving at an unprecedented pace. Just a year ago, DALL·E 2 was the leading model, but it now features prominently in less than 25% of selections in the arena, highlighting the swift changes in user preferences.
• Impact of Open Sourcing: The announcement that Stable Diffusion 3 Medium will be open-sourced on June 12 could significantly influence the community. While it may not match the performance quality of the full version currently provided by Stability AI, its release is expected to spur innovation and fine-tuning within the open-source community, similar to the impact seen with previous versions.

How to Contribute or Get in Touch

To view the leaderboard, visit the Artificial Analysis Text to Image Leaderboard. You can also participate in the ranking process by navigating to the ‘Image Arena’ tab and selecting the image that you believe best represents each prompt. After casting your votes on 30 images, you can check your personalized ranking under the ‘Personal Leaderboard’ tab.

Stay updated with the latest developments by following us on Twitter and LinkedIn. We also provide comparisons of the speed and pricing of Text to Image model API endpoints on our website at Artificial Analysis.

We value your feedback! Feel free to reach out via Twitter or through our website’s contact form.

Other Image Model Quality Initiatives

The Artificial Analysis Text to Image leaderboard is just one of several quality ranking initiatives aimed at evaluating image models. Our leaderboard specifically aims to provide a comprehensive overview by including both proprietary and open-source models, ensuring a complete picture of how leading Text to Image generators compare.

Explore additional initiatives that focus on image model quality, and join the growing community of AI enthusiasts and researchers dedicated to advancing this exciting field!