Your website has a decision to make before September 15

A dentist in Fishers logged into his website's back end last week to update his holiday hours. He found a screen he had never seen before: a list of automated visitors with a switch next to each one. Allow. Charge. Block. His web host had never walked him through it, so he closed the tab and left every switch where he found it.

That default is about to flip on its own. Starting September 15, 2026, Cloudflare, the company that quietly runs security and traffic for a large share of the small business web, is changing how it treats AI crawlers. Any "mixed use" bot, one that blends search, AI agents, and AI model training into a single crawler, gets blocked by default on any page that carries ads, unless the site owner says otherwise. If your host sits behind Cloudflare and you never opened that settings screen, the choice gets made for you in nine days.

What is changing on September 15?

Cloudflare announced the policy on July 1, 2026, giving AI companies about two and a half months to sort their crawlers into clear categories: search, AI agent, or AI training. Pure search crawlers, the kind that feed a normal Google or Bing results page, stay allowed by default. Crawlers that mix search with agent use or model training do not. Starting September 15, those mixed crawlers get blocked by default on any page that shows an ad, for three groups: new Cloudflare customers, new sites set up by existing customers, and every free-tier account that has not changed its settings.

That third group is the one that matters here. A lot of small business sites, built years ago and left alone since, sit on Cloudflare's free tier without the owner knowing it. If that describes your site and you run any ads on it, a banner, an affiliate widget, a sponsored post, the new default applies to you on September 15 whether you log in or not.

I wrote earlier this year about how Google's AI Overviews changed what happens after someone searches from their phone. This Cloudflare deadline is the other half of that same shift. That post was about how an answer gets shown to a searcher. This one is about whether your content gets read at all before an answer ever gets built.

Does this change anything if you don't run ads on your site?

Not right away. Cloudflare's new default only touches pages that carry ads. A five-page site for a bakery or a body shop with no ad units on it is not covered by the automatic switch. But the settings screen is worth understanding anyway, because it is the same screen that decides whether ChatGPT, Perplexity, or another AI assistant can read your menu, your service list, or your hours when a customer asks it a question.

The deadline disappears without ads. The decision does not.

How do you find out if you're behind Cloudflare?

Most owners have never seen this screen because someone else set up their hosting years ago. Ask whoever manages your domain and DNS: your web developer, your hosting company, or the freelancer who built the site. If you manage the site yourself, check your domain registrar or hosting dashboard for Cloudflare in the list of connected services. Free Cloudflare accounts run through cloudflare.com directly, so log in there if you signed up for one at some point and forgot.

Once you're in, look for a section labeled Security or Bots. Cloudflare's bot management tools list every crawler that has visited your site by name, tagged as search, AI agent, or AI training, with a switch for each: Allow, Charge, or Block. Leave the search crawlers alone. The decision that matters is what you do with the other two categories.

Should you block AI crawlers or let them in?

Destiny would run the math on this one, so here is the short version. If a customer near your shop asks ChatGPT or Google's AI Mode something like "best tacos on the east side" instead of typing it into a search box, and the answer names your business, an AI crawler read your site to get there. Block every AI crawler and you close that door along with the rest.

If almost none of your customers are asking chatbots for recommendations yet, allowing costs you nothing today and keeps that door open for later. Charging sits in the middle: some AI companies now pay when your content shows up inside an answer, instead of paying every time a bot fetches the page. Cloudflare rolled this out in July as Pay Per Use, working first with two partners, Ceramic.ai and You.com. The payout on a single small business site will be small for a while yet. The visibility question underneath it is worth answering for yourself before September 15 answers it for you.

Where does Google Search fit into this?

Google keeps a side door. Googlebot crawls for regular search results and for AI Overviews and AI Mode as one combined crawler, and Cloudflare's new default does not touch it. Google also runs a separate bot called Google-Extended that lets a site opt out of AI training specifically, for the Gemini app and related products, without affecting whether the site shows up in a normal Google search. Turning off Google-Extended will not hide your shop from search. Leaving every other AI crawler on the default setting past September 15 is the choice that changes what happens with everyone besides Google.

This is not the first hard deadline a platform has dropped into a dashboard small business owners rarely open. Shopify did something similar with checkout tracking back in August, and the pattern repeats: if you never open the dashboard, the platform decides for you.

If auditing bot settings and DNS records is not how you want to spend an afternoon, it is exactly the kind of maintenance we handle for the sites we build and manage. Most owners would rather run their shop.

Nine days is enough time to open one settings screen and make this decision on purpose instead of by accident. Whatever you land on, allow, charge, or block, at least it will be yours.

---

Sources

#	Source	What it backed up
1	Cloudflare Allows the Agentic Internet to Flourish, Cloudflare press release	The September 15, 2026 deadline, the ads-only scope, which accounts the new default applies to, and the Pay Per Use rollout with Ceramic.ai and You.com.
2	Cloudflare's new policy pushes AI companies to pay for publishers' content, TechCrunch	The definition of a mixed-use crawler and Google's Google-Extended opt-out bot.
3	Google Common Crawlers documentation, Google for Developers	Confirmation that Google-Extended controls AI training use without affecting Search inclusion.
4	Introducing Pay Per Crawl, Cloudflare Blog	Background on the 2025 program that Pay Per Use replaces.