100+ Technical SEO Statistics for 2026: Key Trends and Insights

Technical SEO Statistics for 2026

Technical SEO statistics for 2026 describe a web that is easier to crawl and still slow on mobile. In the 2025 Web Almanac, 48% of mobile sites pass all three Core Web Vitals, 91.5% of mobile pages use HTTPS, and the median mobile home page weighs 2.6 MB. Outside the crawl data, 68.01% of US Google searches ended without a click in early 2026.

This page collects 100+ sourced numbers on speed, page weight, crawling, indexing, AI crawlers and AI search. Every figure names its source and year, so you can check it and benchmark your own site against it.

Most numbers come from the 2025 Web Almanac (HTTP Archive), published in January 2026 from a July 2025 crawl. The rest come from Cloudflare, SparkToro, Ahrefs and Seer Interactive data from 2025 and 2026.

What do the technical SEO statistics for 2026 say at a glance?

Mobile speed is the weakest area. Crawl hygiene improves every year, and AI bots now decide how robots.txt gets used. These 11 numbers are the ones to put in a client report first.

StatisticNumberSource
Mobile sites passing all 3 Core Web Vitals48%Web Almanac 2025
Desktop sites passing all 3 Core Web Vitals56%Web Almanac 2025
Mobile sites with good Largest Contentful Paint62%Web Almanac 2025
Mobile sites with good Interaction to Next Paint77%Web Almanac 2025
Median mobile home page weight2.6 MBWeb Almanac 2025
Mobile pages served over HTTPS91.5%Web Almanac 2025
Mobile pages with a canonical tag67%Web Almanac 2025
Sites with a valid llms.txt fileabout 2.1%Web Almanac 2025
Sites naming GPTBot in robots.txt (desktop)4.5%Web Almanac 2025
US Google searches ending with no click68.01%SparkToro, Jan to Apr 2026
CTR drop for the top result when an AI Overview shows58%Ahrefs, 2026

How many websites pass Core Web Vitals in 2026?

48% of mobile sites and 56% of desktop sites had good Core Web Vitals in the 2025 Web Almanac, which uses July 2025 Chrome UX Report data. Mobile has climbed 16 points since 2021. Desktop has climbed 15.

YearMobile sites with good Core Web VitalsDesktop sites with good Core Web Vitals
202132%41%
202231%44%
202336%48%
202444%55%
202548%56%
  • Among the top 1,000 mobile sites, 51% pass. The rate drops to 42% for the next 10,000 sites and 37% for the next 100,000. It recovers to 48% across the next 10 million (Web Almanac 2025).
  • On desktop, 59% of the top 1,000 sites pass, and every other rank band stays between 55% and 57% (Web Almanac 2025).
  • Home pages pass less often than inner pages: 47% vs 61% on desktop, 45% vs 56% on mobile (Web Almanac 2025).
  • DebugBear’s reading of HTTP Archive data puts desktop origins at 57.1% and mobile origins at 49.7% (DebugBear, 2025).
  • Chrome UX Report origin data for January 2026 shows 55.7% of origins passing all three metrics across devices (reported by WhiteHat SEO).
  • In the same January 2026 data, 68.3% of origins had good LCP, 87.1% had good INP and 80.9% had good CLS (reported by WhiteHat SEO).

Which Core Web Vital fails most often?

Largest Contentful Paint on mobile. Only 62% of mobile sites have a good LCP, against 77% for INP and 81% for CLS.

MetricGood (desktop)Good (mobile)Poor (desktop)Poor (mobile)
Largest Contentful Paint74%62%7%13%
Interaction to Next Paint97%77%under 1%3%
Cumulative Layout Shift72%81%10%9%

Source for the table: Web Almanac 2025, Performance chapter.

What slows down Largest Contentful Paint?

The LCP element is usually an image, and most of those images are still JPG. Late discovery and slow delivery are common.

  • An image is the LCP element on 85.3% of desktop pages and 76.0% of mobile pages. Text is the LCP element at 14.4% and 23.7% (Web Almanac 2025).
  • JPG is the format for 57% of LCP images, PNG for 26% and WebP for 11%. AVIF reaches 0.7% (Web Almanac 2025).
  • JPG use fell 4 points since 2024 while WebP rose 4 points (Web Almanac 2025).
  • The LCP image comes from the same host as the page on 51% of desktop pages and 44% of mobile pages (Web Almanac 2025).
  • fetchpriority=”high” appears on 17% of mobile pages with an LCP image, up from 15% in 2024 (Web Almanac 2025).
  • Preload is used on only 2.1% to 2.2% of pages (Web Almanac 2025).
  • 16% to 17% of pages lazy-load their LCP image, which delays the main content (Web Almanac 2025).

Is interactivity getting better?

On mobile, yes. Good INP rose from 74% in 2024 to 77% in 2025, and the mobile-to-desktop gap shrank from 23 points to 20.

  • Desktop good INP sits at 97% (Web Almanac 2025).
  • Good INP among the top 1,000 mobile sites jumped from 53% to 63% (Web Almanac 2025).
  • Mobile home pages reach 80% good INP, while mobile secondary pages reach 69% (Web Almanac 2025).
  • Median Total Blocking Time on mobile rose 58%, from 1,209 ms to 1,916 ms (Web Almanac 2025).
  • Median desktop Total Blocking Time rose from 67 ms to 92 ms (Web Almanac 2025).
  • At the 90th percentile, mobile pages block the main thread for 7,555 ms (Web Almanac 2025).

Why do page layouts still shift?

Images without set dimensions are among the most common causes of layout shift, and most pages still have at least one.

  • 62% of mobile pages and 65% of desktop pages leave at least one image without explicit dimensions. That is down from 66% and 69% in 2024 (Web Almanac 2025).
  • The median page has 2 unsized images. At the 90th percentile it is 26 on desktop and 23 on mobile (Web Almanac 2025).
  • Good CLS on mobile climbed from 62% in 2021 to 81% in 2025. Desktop climbed from 62% to 72% (Web Almanac 2025).
  • 87% of mobile pages use at least one web font, yet only 15% to 16% preload fonts (Web Almanac 2025).
  • 40% of mobile pages and 44% of desktop pages run non-composited animations (Web Almanac 2025).
  • 23% of sites send Cache-Control: no-store, up from 21% in 2024, which can block the back/forward cache (Web Almanac 2025).

How fast do pages load, and what does a slow page cost?

On mobile, 55% of pages have a good First Contentful Paint and 44% have a good server response time. Desktop does better on both. Slow loads cost visits, though the best-known bounce data is from 2017.

  • Good First Contentful Paint: 70% on desktop (68% in 2024) and 55% on mobile (51% in 2024) (Web Almanac 2025).
  • Poor First Contentful Paint: 9% on desktop and 16% on mobile (Web Almanac 2025).
  • Good Time to First Byte: 55% on desktop and 44% on mobile (Web Almanac 2025).
  • Poor Time to First Byte: 12% on desktop and 17% on mobile (Web Almanac 2025).
  • Only 13% of desktop pages and 15% of mobile pages pass Lighthouse’s render-blocking resources audit (Web Almanac 2025).
  • Vodafone improved LCP by 31% and saw 8% more sales and a 15% better lead-to-visit rate (reported by WhiteHat SEO).

Google and SOASTA published these older figures in 2017. Use them as a rough guide only, since the data is nine years old:

  • 53% of mobile visitors leave a page that takes longer than 3 seconds to load.
  • Bounce probability rises 32% as load time goes from 1 to 3 seconds, 90% from 1 to 5 seconds, and 113% from 1 to 7 seconds.
  • The average mobile landing page took 22 seconds to load fully.

How much do web pages weigh in 2026?

The median home page weighs about 2.6 MB on mobile and 2.9 MB on desktop, and weight is growing 7% to 8% a year. Inner pages are lighter than home pages.

Page typeDesktopMobile
Median home page2,862 KB2,559 KB
Median inner page1,963 KB1,769 KB
Median of all pages2,412 KB2,164 KB
90th percentile of all pages9,179 KB8,337 KB

Source for the table: Web Almanac 2025, Page Weight chapter.

  • The median home page grew 7.8% in a year to 2.7 MB. Mobile grew 8.4% from 2.4 MB, and desktop grew 7.3% (Web Almanac 2025).
  • Inner page weight rose 9.5% year over year. Since 2022, the median mobile inner page is up 27.8% and desktop is up 25.2% (Web Almanac 2025).
  • Home pages are about 45.8% larger than inner pages (Web Almanac 2025).
  • The median mobile home page carries 911 KB of images, 632 KB of JavaScript, 122 KB of fonts, 77 KB of CSS and 22 KB of HTML (Web Almanac 2025).
  • The median desktop home page carries 1,058 KB of images, 697 KB of JavaScript, 139 KB of fonts, 82 KB of CSS and 22 KB of HTML (Web Almanac 2025).
  • Median video bytes grew 28% in a year, from 246 KB to 315 KB (Web Almanac 2025).
  • The median page makes 77 requests on desktop and 72 on mobile, up 8% and 9% year over year (Web Almanac 2025).
  • The median home page makes 80 requests on desktop and 75 on mobile (Web Almanac 2025).
  • The median desktop page requests 17 images and the median mobile page requests 15 (Web Almanac 2025).

How much JavaScript, CSS and HTML do pages ship?

The median page ships 646 KB of compressed JavaScript on mobile and 708 KB on desktop. JavaScript is the second-heaviest resource type after images, and it costs CPU time on top of bytes.

  • 98.1% of pages request at least one JavaScript file (Web Almanac 2025).
  • At the 90th percentile, pages ship 2,003 KB of JavaScript on desktop and 1,910 KB on mobile (Web Almanac 2025).
  • The median single JavaScript file is small: 4.6 KB on desktop, 4.7 KB on mobile (Web Almanac 2025).
  • The median page makes 23 JavaScript requests on desktop and 22 on mobile (Web Almanac 2025).
  • The median page carries 280 KB of unused, uncompressed JavaScript on desktop and 251 KB on mobile (Web Almanac 2025).
  • Median CSS is 83 KB on desktop and 79 KB on mobile, spread over 8 requests (Web Almanac 2025).
  • Median HTML is 35 KB on desktop and 33 KB on mobile (Web Almanac 2025).
  • 71% of desktop and 72% of mobile home pages compress text resources correctly (Web Almanac 2025).
  • 61% of desktop and 62% of mobile home pages minify CSS correctly. For JavaScript the figures are 62% and 63% (Web Almanac 2025).
  • 77% of desktop home pages and 41% of mobile home pages could lazy-load third-party embeds with facades (Web Almanac 2025).
  • The median home page could save about 303 KB on desktop and 270 KB on mobile with longer cache lifetimes (Web Almanac 2025).
  • JavaScript minification is a top technical issue for about 50% of websites audited (SE Ranking, 2025).
  • Early Hints reached roughly 4% of pages by 2025, up from just over 1% in 2023 (Web Almanac 2025).
  • Speculation Rules appear on 3% of top-1,000 desktop home pages and 5% of mobile ones (Web Almanac 2025).

How well do sites handle robots.txt and crawling?

85% of robots.txt requests return a valid 200 responses, up from 84% in 2024. A missing file is the main remaining problem, and an empty file is the second.

  • 84.9% of robots.txt requests return 200 on both desktop and mobile (Web Almanac 2025).
  • 13.3% on desktop and 13.2% on mobile return 404, down from about 14% in 2024 (Web Almanac 2025).
  • Timeouts hit about 1.0% of requests, 403 errors about 0.5% and 500 errors about 0.1% (Web Almanac 2025).
  • Empty robots.txt files appear on 1.8% of desktop sites and 1.7% of mobile sites (Web Almanac 2025).
  • 98% of robots.txt files are under 100 KB, and only 0.1% exceed Google’s 500 KB parsing limit (Web Almanac 2025).
  • The wildcard * user agent appears in 77% of robots.txt files (Web Almanac 2025).
  • AdsBot-Google is named in 9.8% of desktop files and AhrefsBot in 9.3% (Web Almanac 2025).
  • MJ12Bot appears in 7.3% of files, Googlebot in about 6.2% on desktop and 6.7% on mobile, and SemrushBot in about 3% (Web Almanac 2025).
  • Bingbot ranks 22nd among named user agents and appears in under 3% of files (Web Almanac 2025).

Are canonical tags and meta robots set up correctly?

Two thirds of pages have a canonical tag, and canonical use is rising. Roughly one page in three still lacks one, which leaves duplicate-content signals to Google’s guesswork.

  • 68% of desktop pages and 67% of mobile pages have a canonical tag, up from 65% in 2024 (Web Almanac 2025).
  • Nearly 33% of pages have no canonical implementation (Search Engine Land, April 2026).
  • 7% of desktop pages and 9% of mobile pages point their canonical URLs at a different URL (Web Almanac 2025).
  • 64.4% of desktop pages and 64.27% of mobile pages carry the canonical in the raw HTML (Web Almanac 2025).
  • Only about 2% of pages add the canonical in the rendered DOM alone (Web Almanac 2025).
  • HTTP header canonicals appear on 0.6% of desktop pages and 0.5% of mobile pages (Web Almanac 2025).
  • Canonical mismatches affect about 0.7% of pages (Web Almanac 2025).
  • Meta robots appears on 47.0% of desktop and 47.9% of mobile pages, up from 45.5% and 46.2% (Web Almanac 2025).
  • X-Robots-Tag appears on 0.6% of desktop and 0.7% of mobile pages, and 0.4% use both methods at once (Web Almanac 2025).
  • Inner pages are 50% likely to carry a robots directive. Home pages are 46% likely (Web Almanac 2025).
  • noindex appears on 3.5% of desktop and 2.4% of mobile pages. nofollow appears on 2.4% and 2.8% (Web Almanac 2025).
  • 10.1% of desktop pages and 10.3% of mobile pages have invalid elements in the <head>, down from 10.6% and 10.9% (Web Almanac 2025).
  • Yoast SEO runs on about 16% of home pages, Rank Math on about 3.6% and All in One SEO on about 3% (Web Almanac 2025).

Which AI crawlers do sites name in robots.txt, and who uses llms.txt?

GPTBot is the most-named AI crawler, at 4.5% of desktop robots.txt files, up from 2.9% in 2024. ClaudeBot nearly doubled. About 2% of sites have a valid llms.txt file.

CrawlerDesktop 2025Mobile 2025Desktop 2024Mobile 2024
GPTBot4.5%4.2%2.9%2.7%
PetalBot4.0%4.4%not trackednot tracked
ClaudeBot3.6%3.4%1.9%1.6%
CCBot3.5%3.2%2.7%2.4%
Google-Extended3.4%3.0%not comparednot compared
Amazonbot3.3%3.0%not comparednot compared
PerplexityBot2.8%2.7%not comparednot compared
ChatGPT-User2.8%2.5%2.0%1.7%
Meta-ExternalAgent2.8%2.5%not comparednot compared

Source for the table: Web Almanac 2025, SEO chapter.

  • A valid llms.txt file exists on 2.13% of desktop sites and 2.10% of mobile sites. That is 324,184 valid files across mobile sites (Web Almanac 2025).
  • 39.6% of those files come from All in One SEO and 3.6% from Yoast SEO, so many owners may not have chosen the file on purpose (Web Almanac 2025).
  • An early-2025 crawl of the Majestic Million found llms.txt on 0.015% of sites, or 15 domains (Chris Green, Search Engine Land).

How much crawler traffic comes from Googlebot and AI bots?

Googlebot alone made 4.5% of HTML requests on Cloudflare’s network in 2025. All other AI bots combined made 4.2%. AI platforms crawl heavily and send few visitors back.

  • Googlebot generated about 4.5% of all HTML requests, more than all other AI bots combined at 4.2% (Cloudflare, 2025).
  • Googlebot accounted for more than 25% of verified bot traffic (Cloudflare, 2025).
  • Googlebot crawled 11.6% of unique pages in Cloudflare’s October to November 2025 sample (Cloudflare, 2025).
  • AI “user action” crawling grew more than 15 times year over year (Cloudflare, 2025).
  • Global internet traffic grew 19% in 2025 (Cloudflare, 2025).
  • Google’s crawl-to-refer ratio stayed between roughly 3:1 and 30:1 (Cloudflare, 2025).
  • Anthropic’s crawl-to-refer ratio peaked near 500,000:1, then settled between about 25,000:1 and 100,000:1 (Cloudflare, 2025).
  • OpenAI’s ratio peaked at 3,700:1. Perplexity stayed mostly below 400:1 (Cloudflare, 2025).

Is HTTPS and mobile-first design the default in 2026?

Yes for both. HTTPS covers about 91.5% of pages, and Google finished its move to mobile-first indexing in 2023. Separate mobile sites are nearly gone.

  • HTTPS reached 91.7% of desktop pages and 91.5% of mobile pages, up from about 89% in 2024 (Web Almanac 2025).
  • The viewport meta tag appears on 93.1% of desktop pages and 95.2% of mobile pages (Web Almanac 2025).
  • Only about 1% of pages send Vary: User-Agent, the old dynamic-serving signal (Web Almanac 2025).
  • 92% of mobile pages pass Lighthouse’s legible font size audit (Web Almanac 2025).
  • 84.64% of desktop and 86.64% of mobile home pages use descriptive link text. Inner pages reach 92.42% and 93.45% (Web Almanac 2025).

What do on-page technical elements look like across the web?

Title tags are almost universal, meta descriptions sit near two thirds, and H1s are present on 7 pages in 10. Most elements changed by a point or two in a year.

  • Title tags appear on 98.6% of desktop and 98.5% of mobile pages (Web Almanac 2025).
  • Meta descriptions appear on 67.7% of desktop and 67.2% of mobile pages, up from 66.7% and 66.4% in 2024 and down from 71% in 2022 (Web Almanac 2025).
  • The median title is 12 words and 77 characters on desktop, 79 on mobile. At the 90th percentile it reaches 150 to 154 characters (Web Almanac 2025).
  • The median meta description is 40 words and 274 characters (Web Almanac 2025).
  • H1 appears on 71% of desktop and 70% of mobile pages, with 66% non-empty. H2 appears on 72% and 71%, H3 on 60% and H4 on 37% and 36% (Web Almanac 2025).
  • The median page has 60% of its images with an alt attribute, up from 58%, and about 15% of images carry a blank one (Web Almanac 2025).
  • The median home page has 20 images on desktop and 19 on mobile (Web Almanac 2025).
  • Video elements appear on 6.40% of desktop and 5.68% of mobile pages (Web Almanac 2025).
  • The median desktop page has 43 internal links and the median mobile page has 39. At the 90th percentile the counts are 174 and 161 (Web Almanac 2025).
  • The median page has 6 external links on both devices (Web Almanac 2025).
  • rel=”nofollow” appears on 32.3% of mobile pages. sponsored and ugc each sit at about 0.5% (Web Almanac 2025).
  • The median desktop home page has 413 rendered words and the median mobile home page has 378. Inner pages have 339 and 323 (Web Almanac 2025).

How common are lazy loading and structured data?

Native lazy loading is used on about a quarter of images, and structured data covers about half of home pages. Both are still uneven.

  • Nearly 67% of images have no loading attribute, and over 91% of iframes have none (Search Engine Land, April 2026).
  • 27.3% of desktop and 27.7% of mobile home page images use loading=”lazy” (Web Almanac 2025).
  • Lazy-loading an iframe can save about 500 KB for a YouTube embed and 200 KB for a Facebook widget (Web Almanac 2025, citing web.dev).
  • Structured data appears on about 50% of home pages (Web Almanac 2025, as reported by SEOmator).
  • Hreflang appears on 20.3% of desktop sites, roughly double the year before (Web Almanac 2025, as reported by SEOmator).
  • FAQPage schema use keeps rising across the last three Web Almanac editions (Search Engine Land, April 2026).
  • AMP still appears on more than 38,000 home pages (Search Engine Land, April 2026).

How have AI Overviews changed SEO statistics in 2026?

AI Overviews cut clicks hard, and zero-click searches are at a record. The drop is largest for the top-ranked result. Early signs of partial recovery appeared in February 2026.

  • 68.01% of US Google searches ended without a click from January to April 2026, up from 60.45% in 2024 (SparkToro).
  • The share of searches producing at least one click fell 9.51 points, a 22.9% decline (SparkToro).
  • About 49% of searches ended without a click in 2019, though that figure came from a different data provider (SparkToro).
  • AI Overviews appear on more than 20% of Google searches, and click-through drops by nearly 60% when they do (SparkToro).
  • AI Mode took only 0.34% of searches in the study window (SparkToro).
  • 92% to 94% of AI Mode sessions end without a click to an external site (Semrush).
  • In a randomized test of 1,065 US participants, AI Overviews cut organic clicks by 38% and raised zero-click searches from 54% to 72% (Agarwal and Sen, SSRN, April 2026).
  • Across 300,000 keywords, AI Overviews cut the top result’s CTR by 58%, up from 34.5% in the April 2025 version of the study (Ahrefs).
  • The CTR hit runs from about 50.8% at position 2 to 19.4% at position 10 (Ahrefs).
  • Position 1 desktop CTR on affected keywords fell from 0.073 in December 2023 to 0.016 in December 2025 (Ahrefs).
  • 21% of keywords triggered an AI Overview in a 146-million-SERP study from September 2025 (Ahrefs).
  • Organic CTR on AI Overview searches rose from 1.3% in December 2025 to 2.4% in February 2026 (Seer Interactive).
  • CTR is about 3.3% with no AI Overview, 2.1% when the page is cited and 0.9% when it is not (Seer Interactive).
  • CTR on searches without an AI Overview rose from 2.8% in early 2025 to 3.8% in February 2026 (Seer Interactive).
  • AI Overviews show about 36% of informational queries, 86% of question queries, 95% of comparison queries and 5% of transactional ones (Seer Interactive).
  • Brands cited in AI Overviews earned a 15.74% paid CTR on the same queries, against 11.19% when not cited (Seer Interactive, 2026).
  • Google reported 2.5 billion monthly users for AI Overviews and more than 1 billion for AI Mode at I/O 2026 (Google, via TechWyse).

How big is the technical SEO market in 2026?

Technical SEO services are a USD 16.89 billion market in 2026, growing about 11% a year. North America holds the largest regional share.

  • The technical SEO services market was USD 15.73 billion in 2025 (Mordor Intelligence).
  • It is forecast at USD 16.89 billion in 2026 and USD 28.37 billion by 2031, a CAGR of 10.93% (Mordor Intelligence).
  • Technical SEO held a 32.41% share of the services market in 2025 (Mordor Intelligence).
  • North America held 34.62% of the market in 2025 (Mordor Intelligence).

Which technical SEO trends in 2026 should you act on first?

Start with mobile loading speed, then fix the crawl signals, then decide your AI bot policy. These 7 steps follow the numbers above in order of payoff.

  1. Fix mobile LCP first. Only 62% of mobile sites have a good LCP. Check that the LCP image is not lazy-loaded, add fetchpriority=”high” to it, and serve WebP or AVIF instead of JPG.
  2. Cut JavaScript on mobile. The median page carries about 251 KB of unused JavaScript on mobile. Remove what the page never runs and split the rest by template.
  3. Set width and height on every image. 62% of mobile pages still miss at least one. This removes a common source of layout shift.
  4. Add a raw-HTML canonical to every indexable page. Roughly one page in three has none.
  5. Write an AI bot policy in robots.txt. GPTBot, ClaudeBot and CCBot each appear in 3% to 5% of files. Review your server logs first, then allow or block each bot on purpose.
  6. Put key content in the raw HTML. The Web Almanac’s page weight chapter, citing Vercel, says AI crawlers do not render JavaScript.
  7. Track clicks and AI citations along with rankings. With 68.01% of searches ending without a click, a number one ranking no longer guarantees traffic.

What is the takeaway from these technical SEO statistics?

The technical SEO statistics for 2026 show steady gains in basics like HTTPS, canonicals and robots.txt, and slow progress on mobile speed. Pull your own Core Web Vitals report from Google Search Console and compare it with the tables above. If your mobile LCP is under 62% good, start there.

How were these technical SEO statistics collected?

Each number comes from a named public source, and I left out figures I could not trace to one. The Web Almanac measures a July 2025 crawl of millions of sites, mostly home pages with some inner pages. Chrome UX Report figures cover real Chrome users. Cloudflare data covers its own network. SparkToro data covers US Google searches only.

Definitions differ between sources, so totals will not always match. Web Almanac pass rates count sites on one device at a time, while the CrUX figure above covers all devices. The 2017 Google and SOASTA figures are old and flagged as such. Check the original source before quoting a number in a client deck.

FAQ's

What counts as a good LCP, INP and CLS score?

 

Good LCP is 2.5 seconds or less. Good INP is 200 milliseconds or less. Good CLS is 0.1 or less. These are the thresholds the Web Almanac uses.

Google folded its standalone page experience system into its core ranking framework. The Web Almanac says speed measured through Core Web Vitals feeds that framework, so slow pages can rank lower.

Google has said it does not use llms.txt, and no Google service currently reads it, according to the Web Almanac. Some AI tools may read it.

You can block other AI crawlers and keep Googlebot. You cannot separate Googlebot’s search crawling from its AI training crawling, per Cloudflare.

Run a full audit every quarter. Add one after any redesign, migration or CMS update, since those changes break canonicals, redirects and speed.

Share it :