September 2026 measurement · compared with —
Can AIs read small-business websites? We measured 3,054 sites across 12 countries.
Nobody was publishing the figure for France. We measure it regularly, on a sample of shops, tradespeople, restaurants and practices drawn at random across fifteen départements. The first measurement contradicted our own headline — we had written that “many sites” had shut themselves off from AI without knowing. That is false, and we corrected it the same day. Here is what is true, and how it moves.
The sample: small businesses, not media
The addresses come from OpenStreetMap, which records the website of hundreds of thousands of French shops, tradespeople, offices and restaurants. Fifteen départements, from Paris to the Cantal, so as not to measure a single metropolis. Above all: no popularity ranking. A top 1,000 is full of media outlets, and the press blocks heavily — 49% of US and UK news sites shut the door on OAI-SearchBot, against 0.2% of ordinary business sites. Inferring anything about a bakery from that would be a reasoning error. Random draw with a fixed seed after de-duplication by domain, so the measurement can be replayed identically. Facebook pages, booking platforms and directories are excluded: those are somebody else’s sites.
The results, line by line
| What was found | Sites | Share | Change |
|---|---|---|---|
| At least one search bot blocked — the site cannot be quoted | 13 / 1,200 | 1.1% | — |
| of which OAI-SearchBot (ChatGPT) | 11 / 1,200 | 0.9% | — |
| of which Claude-SearchBot | 7 / 1,200 | 0.6% | — |
| of which PerplexityBot | 11 / 1,200 | 0.9% | — |
| GPTBot blocked — training only, costs no citation | 42 / 1,200 | 3.5% | — |
| robots.txt unreadable: the server refuses or does not answer | 239 / 1,200 | 19.9% | — |
| noindex or nosnippet tag on the homepage | 11 / 896 | 1.2% | — |
| Content that probably depends on JavaScript | 41 / 896 | 4.6% | — |
| No price written out in full | 638 / 896 | 71.2% | — |
| No readable update date | 671 / 896 | 74.9% | — |
Two denominators, and they must not be mixed: what is read in robots.txt covers every site in the sample, what is read in the page covers only the pages actually read. Each row therefore shows its own. The “Change” column is in percentage points since the previous measurement; it stays empty in the first month, for want of a comparison.
And elsewhere? The same measurement, country by country
The same diagnostic, on an OpenStreetMap sample specific to each country. Each row has its own month and its own sample: nothing is mixed, only like is compared with like.
| Country | Measured | Sites read | At least one search bot blocked — the site cannot be quoted | No price written out in full | No readable update date |
|---|---|---|---|---|---|
| United Arab Emirates | September 2026 | 198 | 1.5% | 93.6% | 79.8% |
| Belgium | September 2026 | 198 | 1.0% | 71.2% | 80.1% |
| Switzerland | September 2026 | 191 | 0.5% | 94.6% | 81.3% |
| Egypt | September 2026 | 177 | 1.7% | 86.5% | 85.4% |
| Jordan | September 2026 | 117 | 1.7% | 91.5% | 85.4% |
| Kuwait | September 2026 | 142 | 0.7% | 94.1% | 87.1% |
| Lebanon | September 2026 | 142 | 1.4% | 75.6% | 77.9% |
| Morocco | September 2026 | 215 | 0.0% | 82.1% | 73.7% |
| Qatar | September 2026 | 137 | 2.2% | 90.1% | 84.6% |
| Saudi Arabia | September 2026 | 165 | 1.2% | 93.8% | 87.5% |
| Tunisia | September 2026 | 172 | 1.7% | 84.9% | 66.7% |
New: brand websites →
The same diagnostic on well-known brands in cosmetics, SaaS and fashion.
What it means: blocking is rare, missing information is not
About one site in a hundred has removed itself from AI answers. That is rare — but it is total, it lasts as long as the line stays in the file, and nobody is ever notified: it is an argument about severity, not frequency. The gesture itself is common: 3.5% block GPTBot, which only trains models and costs no citation. Many sites therefore believe they have protected themselves while closing nothing that matters. What is genuinely massive lies elsewhere: seven sites in ten do not write their price, three in four show no date. Yet the SIGIR 2026 study, over 252,000 matched trials, ranks explicit price among the four decisive factors for citation, and Ahrefs, over 17 million citations, shows AIs quote fresher content than organic search does.
What it does not mean
That writing a price will get your client quoted. Nobody has published a before/after study with a control group showing that, and we will not promise it while that remains true. What these figures establish is a frequency: how many sites carry a verifiable obstacle, documented by the providers themselves or measured with a control group. Removing a verifiable obstacle is useful work; selling it as a guarantee of citation would be a lie.
The limits, spelled out
1,200 sites are not a census: the intervals above state the margin. The sample only contains businesses visible enough to have been mapped along with their website — the smallest are absent. One site in five did not let its robots.txt be read, usually because its host refuses any unknown bot; those are counted as “unknown”, never as “open”. Only the homepage was read, not product pages. And it is a snapshot taken at the time of measurement: a robots.txt changes in one line.
Redo the measurement
The diagnostic behind these figures is exactly the Free diagnostic: two HTTP requests per site, no model call at all, so it costs nobody anything. You can run it on any address, five times a day without an account, up to 40 times a day with a subscription. The sample data comes from OpenStreetMap’s Overpass API, freely queryable: the method described here is enough to redo the measurement and contradict us, which is the only serious way to validate it.
And your client’s site — where does it stand?
The same diagnostic, on any address you choose: the Free diagnostic, no account needed.
Check a site →