More

    Pioneering LLM Creativity Benchmark Reveals AI Tool Insights

    Understanding the Power of AI in Advertising: Insights from the New Study

    Recent developments in artificial intelligence have opened doors to new possibilities in creativity across various sectors, particularly advertising. A groundbreaking study by Springboards, an AI platform, reveals that popular AI tools like ChatGPT, Gemini, and Claude are not as disparate in their creative capabilities as many once believed. This insightful research, conducted in collaboration with industry organizations such as the 4As and D&AD, presents compelling findings that could reshape how advertising agencies approach the use of AI in creativity.

    Behind the Study

    The Creativity Benchmark study aimed to challenge the prevailing notion that certain AI platforms are superior in generating creative outputs. Sixteen distinct AI systems, including those from renowned tech giants like OpenAI, Google, and Meta, were put to the test. The dataset comprised contributions from over 600 creative professionals across notable brands, with more than 11,000 comparisons made to evaluate each AI’s performance on creative tasks.

    The research sought to dissect three primary types of creative challenges:

    1. Identifying Consumer Insights: Gaining surprising insights about audience preferences.
    2. Developing Campaign Ideas: Crafting comprehensive ideas for marketing campaigns.
    3. Generating Bold Concepts: Coming up with innovative and attention-grabbing ideas.

    Surprising Findings

    The study unveiled some intriguing conclusions that challenge traditional perceptions:

    No Clear Champion Among AI Tools

    Despite expectations, there was no decisive winner among the AI systems. The variations in performance were surprisingly minimal, suggesting that the differences between the best and the worst tools in the context of creative outputs were less pronounced than anticipated.

    “Everyone assumes some AI tools are way better than others for creative work,” noted Pip Bingemann, CEO and co-founder of Springboards. He emphasized that AI models primarily recognize patterns and provide the most probable responses, which does not always equate to creativity.

    Task-Specific Strengths

    Another key takeaway was that different AI tools excelled in distinct areas. Some systems were more adept at strategic insights, while others thrived in producing imaginative ideas. This suggests that a multi-faceted approach could be more beneficial for agencies looking to harness the full potential of AI in advertising.

    The Human Element in Creativity

    While AI has shown potential in generating ideas, the study highlighted the critical role that human judgment plays in the creative process. When AI systems were tasked with evaluating their own creative outputs, their assessments significantly differed from those made by human experts. This discrepancy underscores a crucial insight: while AI can assist in idea generation, it cannot replace the nuanced understanding and instinct of human creativity.

    The Importance of Variety

    Beyond quality, the diversity of ideas generated by AI tools emerged as a crucial factor. Some AI systems tended to produce similar ideas repeatedly, while others could generate a wider range of creative options from the same brief. In the world of advertising, offering varied choices is often as important as delivering quality ideas.

    Revisiting Creativity Assessment

    Another layer to the research revealed that conventional creativity tests commonly employed in psychological contexts do not effectively measure AI performance in marketing-specific tasks. The study advocates for the development of specialized metrics tailored to the unique nuances of advertising creativity.

    Cultural Influences on Preferences

    Interestingly, the study also unearthed cultural differences in creative preferences among advertising professionals across various countries. The findings indicate that what one group considers highly creative may differ significantly from the views of another, showcasing the global diversity in creative judgment.

    Methodology Overview

    The research incorporated a robust methodology involving 678 advertising professionals participating in a blind A/B idea comparison, reminiscent of a “Tinder for Ideas.” Over four weeks, from June 10, 2025, data was meticulously collected and analyzed, covering numerous brands and creative prompts.

    This thorough study employed four distinct evaluation strategies to assess AI creativity:

    1. Human Comparisons: Nearly 700 professionals compared AI-generated ideas without knowledge of their sources, ensuring objectivity.
    2. Diversity Measurement: Each AI system was challenged to create multiple responses to the same brief, with a focus on the variety in outputs.
    3. Self-Evaluation Tests: Leading AI systems were tasked with judging the same creatives previously scored by humans, revealing a lack of agreement on evaluations.
    4. Creativity Testing: AI systems underwent adapted psychological tests to evaluate their creativity in a structured manner.

    Looking Ahead: The Future of AI in Creativity

    These findings prompt a re-evaluation of how advertising professionals incorporate AI into their creative workflows. As Jeremy Lockhorn, SVP of Creative Technologies & Innovation at 4As, stated, “LLMs aren’t a one-size-fits-all solution—they’re general-purpose tools that require human creativity to unlock breakthrough outcomes.” The essence of the study emphasizes learning to collaborate effectively with AI systems to amplify, rather than replace, the creative process.

    Supporting Resources

    For those interested in delving deeper into the research, the full white paper is available for review here.

    About Springboards

    Springboards, founded by industry veterans birthed from a desire to inspire creativity in advertising, continues to forge partnerships and deliver innovative solutions. With a commitment to enhancing the creative landscape, they remain at the forefront of AI integration in marketing.


    This article highlights the transformative impact AI continues to have on creativity in advertising, emphasizing a future where collaboration between AI and human creativity thrives.

    Latest articles

    Related articles

    Leave a reply

    Please enter your comment!
    Please enter your name here

    Popular