top of page

I asked four AI models for one brand, 12,000 times.

Writer: Dimitris Drakatos
Dimitris Drakatos
9 hours ago
6 min read

GPT-5.6 gave one brand over 90% in 31/50 tests. Developer-access (API) test with search disabled: 50 categories, 30 attempts each, collected 10 October 2026. Not a measurement of the consumer apps.

The question never changed: "Name one [category] brand. Answer with only the brand name." I put it to four models, across 50 categories, 30 times each, once with web search disabled and once with search available. Four models, two conditions, 50 categories, 30 attempts: 12,000 attempts in total.

I wanted to know something simple. If you ask again, do you get a different name?

In most categories, you don't.

Bar chart: categories where one brand took at least 28 of 30 attempts, search disabled. GPT-5.6 31 of 50, Sonnet 5 39 of 49 eligible, Gemini Flash 38 of 50, Sonar 33 of 50.

What I tested

  • Prompt: "Name one [category] brand. Answer with only the brand name." No country, no context, no follow-up.

  • Categories: 50, fixed before collection: 20 retail and service categories, 15 consumer products, 15 business software categories.

  • Models: GPT-5.6 (gpt-5.6-luna), Sonnet 5 (claude-sonnet-5), Gemini Flash (gemini-3.6-flash) and Sonar (perplexity/sonar). Each was chosen as the closest developer-access match to what the product's free tier serves by default, so treat them as proxies, not as the apps themselves.

  • Conditions: "search disabled" and "search available, optional". In the second condition the model could search but was not made to.

  • Attempts: 30 per model, per category, per condition. 12,000 scheduled, 11,992 usable.

  • Collected: 10 October 2026, in one window, with no retries and no model substitution.

The design, the thresholds and the wording of the GPT-5.6 result at the top were pre-registered before collection. A headline could only be used if its rule passed.

Finding 1: ask again, get the same brand

I counted a category as concentrated when one brand took more than 90% of a model's 30 attempts, which means at least 28 of 30.

Model

Search disabled

Search available, optional

GPT-5.6

31 of 50 categories

33 of 50

Sonnet 5

39 of 49 eligible categories

46 of 50

Gemini Flash

38 of 50

35 of 50

Sonar

33 of 50

33 of 50

Sonnet 5's search-disabled figure is out of 49, not 50, because one category (broadband provider) fell below the minimum of 27 valid answers. More on that in the limitations.

Many of these were not just above 90% but unanimous. With search disabled, GPT-5.6 gave the same brand 30 times out of 30 in 28 categories, Sonnet 5 in 39, Gemini Flash in 29 and Sonar in 23.

Two things to keep in mind when reading this table. The 31 of 50 counts categories, not individual answers. And the repeated brand is each model's own: a model can be completely consistent with itself and still disagree with the model next to it.

Finding 2: the four models often repeat the same brand, but not always

With search available, all four models shared the same most-frequent brand in 31 of 50 categories.

Grid of the 31 brands that all four AI models gave most often in the same category, with search available, out of 50 categories. Includes Starbucks, IKEA, Salesforce, Zoom and Shopify.

The 31: Comcast (Xfinity), Chase, The Home Depot, Best Buy, IKEA, FedEx, Starbucks, McDonald's, Barnes & Noble, Petco, Staples, Samsung (television), Dyson, Sony (headphones), BIC, Tide, Cheerios, Salesforce, Mailchimp, Asana, QuickBooks, Workday, Zendesk, Zoom, Slack, Dropbox, Wix, Shopify, Bluehost, DocuSign and Calendly.

Looking at where that agreement sits: 14 of the 15 business software categories had one shared leader (payroll software was the exception), against 17 of the other 35 categories. That split is descriptive. It was not a pre-registered comparison, the categories were selected by me, and it does not establish a pattern for software as an industry.

In the other 19 categories, the models had different leaders. A few examples, with search available, out of 30 attempts each:

Category

GPT-5.6

Sonnet 5

Gemini Flash

Sonar

Washing machine

Whirlpool 30

Samsung 30

Bosch 25

Whirlpool 30

Washing-up liquid

Dawn 30

Fairy 30

Fairy 30

Dawn 18

Clothing retailer

Gap 10

Zara 30

Zara 30

Uniqlo 26

Insurer

State Farm 17

GEICO 30

GEICO 17

Allstate 27

Supermarket

Kroger 28

Walmart 22

Walmart 30

Walmart 14

Estate agent

RE/MAX 20

Century 21 30

Savills 21

RE/MAX 22

Payroll software

Gusto 18

Gusto 30

Gusto 22

ADP 28


Table of seven categories where four AI models had different top brands. Washing machine: GPT-5.6 Whirlpool 30 of 30, Sonnet 5 Samsung 30, Gemini Flash Bosch 25, Sonar Whirlpool 30.

The washing machine row is the one I keep coming back to. Three of the four models gave a single brand in all 30 attempts, and they did not give the same one. Disagreement between models can sit alongside total consistency within each model.

The prompt names no country, and the answers mix markets: Fairy and Dawn, Savills and Century 21, Octopus Energy next to US retailers. I did not test why.

Finding 3: search availability rarely changed the top brand

I compared each model's most-frequent brand with search disabled against the same model with search available. A switch only counted as robust under a pre-registered rule that sets aside part of the attempts to confirm it.

Model

Robust top-brand switches (of 50 categories)

GPT-5.6

7

Sonnet 5

4

Gemini Flash

3

Sonar

2

That is 16 switches counted as model and category pairs, not 16 different categories. Washing-up liquid alone accounts for three of them, and in opposite directions: GPT-5.6 and Sonar moved from Fairy to Dawn, Sonnet 5 moved from Dawn to Fairy.

The pre-registered bar for a "search changed the defaults" headline was 10 categories for a model. No model reached it, so I am not making that claim.

That is not the same as saying search did nothing. Between conditions, each model had 2 to 7 robust top-brand switches, and the overall mix of answers differed for all four (pre-registered permutation test, Holm-adjusted p = 0.0004 for each model). The top name mostly held. The spread underneath it moved.

Bar chart: categories of 50 where a model's top brand changed between search disabled and search available. GPT-5.6 7, Sonnet 5 4, Gemini Flash 3, Sonar 2. Pre-registered threshold was 10.

The switches, for the record:

  • GPT-5.6: supermarket (Walmart to Kroger), insurer (Allstate to State Farm), airline (Emirates to Delta), estate agent (Savills to RE/MAX), headphones (Sennheiser to Sony), washing-up liquid (Fairy to Dawn), payroll software (ADP to Gusto).

  • Sonnet 5: pet supplies retailer (PetSmart to Petco), television (Sony to Samsung), mattress (Sealy to Casper), washing-up liquid (Dawn to Fairy).

  • Gemini Flash: mobile network (Verizon to Vodafone), washing machine (Whirlpool to Bosch), mattress (Casper to Sealy).

  • Sonar: hotel chain (Marriott to Hilton), washing-up liquid (Fairy to Dawn).

What this does not show

It is not the apps. These are measurements of each provider's developer access (API) on one day. Consumer apps add their own instructions, routing, location and personalisation. The consumer check is pending and will be reported as a dated addendum.

The prompt forces a single answer. The prompt imposed one brand per response, but not the same brand across repetitions; these results describe that prompt under the tested developer-interface settings. A real buying question is longer and messier, and I did not test one here.

Category selection and geography shape the result. These 50 selected categories and the English prompt, which specifies no country, limit generalisation; the software split does not establish an industry-wide pattern.

"Search available" is not "search used". The model decided whether to search. This experiment compares the two settings, not searched answers against unsearched ones.

Most frequent is not recommended. Nothing here measures quality, ranking, market share or what a model would say to a shopper with a specific need.

How the answers were checked

Each response was reduced to a brand by an automated extractor, then audited.

  • 1,000 records were reviewed by two AI reviewers, with a third on flagged cards.

  • I checked a fixed random sample of 75 by hand: zero discrepancies. I also reviewed a sample of 40 for plausibility: 39 plausible, 0 implausible, 1 unsure (eXp Realty as an estate agent).

  • The audit found one systematic defect. The extractor read "RE/MAX" as two brands because of the slash. The repair changed 71 of 12,000 records, all in the estate agent category. I record the original extraction as having failed that check.

The count-affecting RE/MAX repair preceded final analysis and changed 71 records, but it was not blind, one adjudication was author-decided, and zero discrepancies in 75 human spot checks does not prove error-free extraction.

Outcome analysis followed the 14 November lock; extraction correction and adjudication preceded it. Every change to the original pre-registration was made as a versioned amendment.

Eight attempts were not usable. Six were Sonnet 5 answers for broadband provider: three corrected themselves mid-answer ("Comcast Xoom", then "Wait, let me correct that"), and three gave a name that could not be resolved to a brand ("Comcast Xoty", "Comcast Xenforo", "Comcast Xantech"). The other two were a timeout and a failed request. Percentages use all 30 attempts as the denominator, so a failed attempt counts against concentration rather than being dropped.

Disclosure

I run AI Search Experiments independently. I also built Wellknown (getwellknown.ai), a tool that checks how AI agents see a website, so I have an interest in this topic. No brand in this experiment had any involvement.

Collected 10 October 2026. Analysis locked 14 November 2026. Protocol v1.7.2. The consumer check is pending and will be reported as a dated addendum.

Comments


Illustrated portrait of Dimitris Drakatos, SEO and AI search consultant.

Hi, I'm Dimitris

I work on SEO, AI search and organic growth, in-house and independently. I write here about what is actually working, and I run my own research into how AI search behaves.

Let the posts come to you.

Thanks for submitting!

Have something you want to talk through?

Thanks, that has come through.

Tell me a bit about the company, where organic sits for you today, and what you are trying to fix. If I am not the right person for it, I will tell you.

© 2026 by Dimitris Drakatos

bottom of page