01
Overview
Cloudflare’s September 15, 2026 AI crawler policy change creates an SEO-sensitive edge case: a crawler that performs both Search and Training can be affected by a rule intended to block Training.
Cloudflare explicitly names Googlebot, Applebot and BingBot as examples of multi-purpose crawlers in its announcement. That makes it important to understand what Cloudflare actually says—and what it does not say—before changing AI crawler settings on a site that depends on organic discovery.
02
Why mixed-purpose crawlers are different
Cloudflare now classifies AI-related crawler behavior as Search, Agent or Training. A single crawler can have more than one behavior.
Cloudflare says the most restrictive applicable rule is enforced for multi-purpose crawlers. Therefore, if a crawler is classified for both Search and Training and the site blocks Training, that crawler can be blocked even if Search is otherwise allowed.
03
Does blocking Training block Googlebot?
It can under the policy conditions Cloudflare describes.
Cloudflare’s July announcement specifically says multi-purpose crawlers such as Googlebot, Applebot and BingBot will be blocked for customers who selected to block Training through affected new AI-traffic options or the legacy Block AI bots service.
This is a crawler-access statement from Cloudflare. It is not a statement that a site will automatically lose rankings.
04
Why “Googlebot” does not equal one SEO outcome
Search platforms operate multiple crawlers, services and indexing systems. A blocked request only tells you that a particular request did not reach the content through that path.
Ranking and indexing consequences depend on which crawler was blocked, how often, which URLs were affected, whether another crawler can access them, and how the search engine uses that content.
That is why a configuration change should be followed by measurement rather than a prediction of guaranteed traffic loss.
05
What to check after changing Cloudflare AI bot policies
First, inspect robots.txt. If Bot Preference Sync is active, policy preferences may be reflected there. Remember that robots.txt and Cloudflare edge enforcement are different layers.
Second, inspect Cloudflare analytics/security events for relevant crawler blocks where your plan exposes them.
Third, monitor Search Console. Watch for changes in crawl behavior, indexing, discovered URLs and search performance. A short-term fluctuation alone does not prove causation, but a crawler-policy change gives you a concrete event to correlate against.
Fourth, test important templates separately. Cloudflare supports blocking on pages displaying ads, so behavior may differ across sections of the same site.
06
Search visibility versus training control
The policy problem exists because some crawler operators combine purposes. A publisher may want Search indexing but not model training. Cloudflare’s controls can express that preference, but a mixed-purpose crawler can force a stricter result.
Cloudflare has argued that bot operators should separate behaviors more transparently so site owners do not have to choose between discoverability and training access.
07
What should SEO teams do?
Treat AI crawler settings as part of technical SEO governance.
Before changing Training rules, identify which search and answer-engine crawlers matter to the business. After the change, verify access and first-party search data. If a desired crawler is being blocked because it is classified as mixed-purpose, decide whether the training restriction or the discovery benefit matters more for that site.
There is no universal correct answer. A publisher with valuable licensed content may choose differently from a product site whose priority is maximum discovery.
08
FAQ
Did Cloudflare start blocking Googlebot globally on September 15?
No. Cloudflare describes specific policy conditions under which mixed-purpose crawlers can be blocked. It does not say Googlebot is globally blocked for every Cloudflare site.
Does blocking AI training guarantee an SEO penalty?
No. Cloudflare does not document a guaranteed ranking penalty. The immediate technical question is crawler access; SEO impact must be measured.
Why can Search be allowed while Googlebot is blocked?
Because Cloudflare can classify one crawler as serving multiple purposes. When Search is allowed but Training is blocked, the most restrictive applicable behavior can win for a mixed-purpose crawler.
What data should I monitor?
Cloudflare crawler/security data where available, robots.txt, crawl tests, and Google Search Console indexing and performance evidence.
Sources
Primary and supporting sources
Use the linked documentation to check current pricing, access and product behaviour. The dates above describe this article; they do not guarantee that every vendor claim remains current.