Guide

How to keep a page out of AI answers

Sometimes the problem isn't being missing from AI answers, it's being in them with the wrong page: an old price list, a service you've dropped, a promotion that ended. We pulled together what Google, Microsoft, OpenAI, Anthropic, Perplexity, Apple, Amazon, Brave and DuckDuckGo document about keeping content out of their AI answers, from pages our team opened for earlier guides between September 30 and November 11, 2026. There is no single switch. Here is each one, and when fixing the page beats hiding it.

Quick answer

Engine by engine. For Google's AI Overviews and AI Mode, use nosnippet, max-snippet or noindex; Google-Extended won't do it. For ChatGPT, Claude and Perplexity, block their search bots for that path in robots.txt. For Copilot, Bing's NOARCHIVE works today. Often the better fix is to update the page instead.

What are the key facts about keeping a page out of AI answers?

EngineControlWhat it doesSource
Google AI Overviews and AI ModenosnippetStops the content being used as "a direct input for AI Overviews and AI Mode"Google
Google AI Overviews and AI Modemax-snippet"Will also limit how much of the content may be used" thereGoogle
Google Search, AI features includednoindexTakes the page out of Search, and so out of its AI answersGoogle
Google, whole siteSearch Console setting, still a test (UK sites first)Opts a site out of appearing in, and grounding, Google's generative AI featuresGoogle
GoogleGoogle-ExtendedGemini training and Gemini app grounding only; "does not impact a site's inclusion in Google Search"Google
ChatGPT searchDisallow OAI-SearchBotOpted-out sites are not shown in ChatGPT search answers; about 24 hours to take effectOpenAI
ClaudeDisallow Claude-SearchBotBlocking "may reduce your site's visibility and accuracy in user search results"Anthropic
PerplexityDisallow PerplexityBotIts search crawler; Perplexity-User "generally ignores robots.txt rules"Perplexity
Microsoft CopilotNOARCHIVE"Prevents content from being used in Copilot responses and grounding results"Bing
Microsoft Copilot, from 1 Feb 2027AI Grounding preferencePer-page Allow, DisallowExtended or Disallow; Allow is the defaultBing
DuckDuckGoDisallow DuckAssistBotOut of AI answers; "does not impact organic search rankings"; about 72 hoursDuckDuckGo
Brave Searchnoindex"robots.txt is not used to prevent a page from being indexed"Brave

Can I keep one page out of AI answers and leave the rest of my site in?

Yes, for most engines, but each has its own control and there is no single switch. The controls come in two kinds: a tag on the page itself (a robots meta tag, or an X-Robots-Tag HTTP header for files), and a rule in robots.txt that names a crawler and a path.

First decide which of two things you mean:

  • Out of AI answers. The search crawlers and snippet rules decide this: OAI-SearchBot for ChatGPT search, Claude-SearchBot for Claude, PerplexityBot for Perplexity, and Google's snippet and index rules for AI Overviews and AI Mode.
  • Out of AI training. Different names: GPTBot, ClaudeBot, Google-Extended, Applebot-Extended. OpenAI says each of its bot settings is independent, so blocking GPTBot keeps pages out of training while OAI-SearchBot decides ChatGPT search. OpenAI Apple says pages that block Applebot-Extended "can still be included in search results." Apple

Google's Search team is direct about the trade-off: "More restrictive permissions will limit how your content is featured in our AI experiences." Google Search Central Blog That's the point when you want a page out, and a risk when a tag lands on the wrong page.

How do I keep a page out of Google's AI Overviews and AI Mode?

With the same controls that shape regular Search. Google lists nosnippet, data-nosnippet, max-snippet and noindex as the ways to limit what shows from your pages in Search, including its AI features. These apply to Search as a whole, so they also change how the page looks in ordinary results. Google

  • Keep the page listed but not quoted: <meta name="robots" content="nosnippet">. Google's robots meta tag docs say nosnippet also stops the content being used as "a direct input for AI Overviews and AI Mode," and that max-snippet "will also limit how much of the content may be used." Google
  • Take the page out of Search altogether: noindex, which tells Google "Do not show this page" in results. Google
  • For a PDF or other file: send the same rule as an X-Robots-Tag HTTP header, since you can't put a meta tag inside a PDF. Google

Don't also block the page in robots.txt. If crawling is blocked, Google never sees the tag: the rules "will be ignored." Google And robots.txt "is not a mechanism for keeping a web page out of Google"; a blocked page can still be listed, it just "won't have a description." Google

For the whole site, Google describes a Search Console setting that opts a site out of appearing in, and helping to ground, its generative AI features. It says sites that opt out "will not receive traffic or impressions" from those features. It is still a test: Google is rolling it out to a subset of site owners in the UK first, before offering it everywhere, so most sites won't see the setting yet. Google

Google-Extended is not the switch. Google says it controls training of future Gemini models and grounding in the Gemini app, and "does not impact a site's inclusion in Google Search nor is it used as a ranking signal in Google Search." Google For Search, Google says the control is your robots.txt rules for Googlebot, because AI "is built into Search." Google

How do I keep a page out of ChatGPT, Claude and Perplexity?

With robots.txt rules for their search crawlers, limited to the page's path. For example, to keep an old price list out of all three:

User-agent: OAI-SearchBot
User-agent: Claude-SearchBot
User-agent: PerplexityBot
Disallow: /old-price-list/

One catch: under the robots.txt rules Google documents, "only one group is valid for a particular crawler," the most specific one that names it. Google Our advice: once a bot has its own group, it no longer reads your User-agent: * rules, so copy any other Disallow lines it should follow into the new group.

  • OpenAI. OAI-SearchBot "is used to surface websites in search results in ChatGPT's search features," and sites that opt out of it are not shown in ChatGPT search answers. OpenAI says "it can take ~24 hours from a site's robots.txt update" for its systems to adjust. OpenAI
  • Anthropic. Blocking Claude-SearchBot "may reduce your site's visibility and accuracy in user search results." Anthropic says its bots follow robots.txt, and warns that blocking its IP addresses instead "may not work correctly or persistently guarantee an opt-out." Anthropic
  • Perplexity. PerplexityBot "is designed to surface and link websites in search results on Perplexity." Perplexity says robots.txt changes can take up to 24 hours to apply. Perplexity

The gap: fetchers that act for a user. When someone asks about your page directly, a different agent may fetch it. OpenAI says for ChatGPT-User "robots.txt rules may not apply," and Perplexity says Perplexity-User "generally ignores robots.txt rules." OpenAI Perplexity Anthropic's Claude-User does follow robots.txt; blocking it stops Claude retrieving your page for that user. Anthropic So a person who pastes your URL into a chat may still get a summary. That's a person asking for your page, not the AI finding it on its own.

noindex doesn't reach them. OpenAI, Anthropic and Perplexity document robots.txt controls for their bots, not robots meta tags or X-Robots-Tag. OpenAI Anthropic Perplexity

How do I keep a page out of Microsoft Copilot?

With Bing's meta tags today, and with a new AI Grounding preference from 2027. Bing says to use NOINDEX when a URL should not appear in Bing search, Copilot experiences or grounding results, and that "NOARCHIVE prevents content from being used in Copilot responses and grounding results." Bing Webmaster Help

Bing's 2023 announcement, from before the Copilot name, spelled out the difference: "Content tagged NOARCHIVE will not be included in Bing Chat answers, not be linked to in the answers," while for NOCACHE "We will only display URL/Snippet/Title in the answer." Microsoft Bing

What changes in 2027. Microsoft announced on 30 October 2026 that separate AI Training and AI Grounding preferences replace NOARCHIVE and NOCACHE for those uses, and it starts honoring them on 1 February 2027. Grounding is set per page with a meta tag or HTTP header (Allow, DisallowExtended or Disallow), and Allow is the default. A site that uses NOARCHIVE and wants to stay out of grounding must publish a page-level AI Grounding preference, and record a temporary one in Bing Webmaster Tools, before 1 March 2027. Bing Webmaster Help

Check this first: many sites added NOARCHIVE years ago to hide cached copies. On Bing that same tag now keeps the page out of Copilot, wanted or not.

What about Apple, Amazon, Brave and DuckDuckGo?

  • Apple (Siri, Spotlight, Safari). Applebot-Extended only controls whether your content trains Apple's models; it "does not crawl webpages," and pages that block it "can still be included in search results." Apple says the X-Robots-Tag header is "useful for non-HTML resources (such as PDFs or images)" when you want to give Applebot rules. Apple
  • Amazon (Alexa+). Amazon honors robots meta tags: noarchive means "do not use the page for model training" and noindex means do not index it. Amazon says Amzn-SearchBot makes content eligible for Alexa and Amzn-User fetches pages for live answers, so on our reading those are the two to name in robots.txt. Amazon's crawlers don't support crawl-delay. Amazon Developer
  • Brave Search. Brave's crawler has no separate user agent and follows your robots.txt rules for Googlebot. It says "robots.txt is not used to prevent a page from being indexed": add noindex, then submit the URL so Brave fetches it again. For pages that no longer exist, you can also email not-found@brave.com. Brave Search Help
  • DuckDuckGo. DuckAssistBot "crawls pages in real-time for our AI-assisted answers." Blocking it "does not impact organic search rankings" and removes your pages from AI answers in about 72 hours. DuckDuckGo Help

What doesn't keep a page out of AI answers?

  • Google-Extended, for AI Overviews and AI Mode. It doesn't affect inclusion in Search. Google
  • robots.txt alone, for Google and Brave. Both say it isn't a way to remove a page from their index. Google Brave Search Help
  • robots.txt plus noindex on the same page. Google can't see a tag on a page it may not crawl. Google
  • Blocking training bots. GPTBot, ClaudeBot and Applebot-Extended govern training, not search answers. OpenAI Apple
  • Blocking IP addresses. Anthropic warns it "may not work correctly or persistently guarantee an opt-out." Anthropic
  • Expecting user-triggered fetchers to obey. ChatGPT-User and Perplexity-User may not follow robots.txt. OpenAI Perplexity

Should I hide the outdated page or fix it?

Usually fix it. Our advice, not a vendor statement: if AI keeps quoting a 2024 price list, hiding that page doesn't put this year's prices in front of anyone. The engine just quotes something else, possibly a directory with the same old number. Replacing the content at the same address gives every engine the right facts the next time it reads the page.

Hide a page when nothing true is left on it, or when it shouldn't be public at all, such as a page for a staff member who has left or an offer you can no longer honor. Otherwise:

  1. Update the page at the same URL with the current facts, and say plainly what changed.
  2. Tell the engines it changed: update lastmod in your sitemap and send the URL through IndexNow. See do sitemaps help AI search?
  3. Fix the copies elsewhere: your Google Business Profile, Yelp and directory listings should show the same facts. See should you show prices to AI?
  4. Check again in a day or two, with the questions customers actually ask. When AI is simply wrong about you, our guide to what to do when AI gets your business wrong has the rest.

How do I take one page out of AI answers, step by step?

  1. Decide what you want gone: the page from search entirely, or only from being quoted in AI answers. The first is noindex; the second is nosnippet plus crawler rules.
  2. For Google, add the tag and keep the page crawlable. nosnippet keeps it out of AI Overviews and AI Mode as a direct input; noindex removes it from Search. Don't block it in robots.txt, or Google won't see the tag. Google
  3. For ChatGPT, Claude and Perplexity, add a robots.txt rule. Disallow the page's path for OAI-SearchBot, Claude-SearchBot and PerplexityBot. OpenAI Anthropic Perplexity
  4. For Copilot, add NOARCHIVE. Before 1 March 2027, switch to a page-level AI Grounding preference. Bing Webmaster Help Bing Webmaster Help
  5. For DuckDuckGo and Alexa, add their bots to the same robots.txt rule: DuckAssistBot, Amzn-SearchBot and Amzn-User. DuckDuckGo Help Amazon Developer
  6. For Brave, rely on the noindex, then submit the URL so Brave fetches it again. Brave Search Help
  7. For PDFs and images, send the rule as an X-Robots-Tag header. Google
  8. Check it took. Search Console's URL Inspection live test "fetches and examines the URL in real time" and reports robots.txt blocks and noindex. Google Search Console Help Then allow about 24 hours for OpenAI and Perplexity and about 72 hours for DuckDuckGo before asking the AI again. OpenAI Perplexity DuckDuckGo Help

Which control should I use for which engine?

GoalUseEngines it coversSource
Listed, but not quoted by Google's AInosnippet or a small max-snippetGoogle AI Overviews and AI Mode (and regular snippets)Google
Gone from Google entirelynoindex, page left crawlableGoogle Search, AI features includedGoogle
Whole site out of Google's AI featuresSearch Console AI setting (a test, UK sites first)GoogleGoogle
Out of ChatGPT searchDisallow OAI-SearchBot for the pathChatGPT searchOpenAI
Out of Claude's search resultsDisallow Claude-SearchBot (and Claude-User)ClaudeAnthropic
Out of Perplexity searchDisallow PerplexityBotPerplexity searchPerplexity
Out of CopilotNOARCHIVE now; AI Grounding preference from 2027Microsoft CopilotBing
Out of DuckDuckGo AI answersDisallow DuckAssistBotDuckDuckGo Search AssistDuckDuckGo
Out of Alexa+Disallow Amzn-SearchBot and Amzn-UserAlexaAmazon
Out of Bravenoindex, then resubmit the URLBrave Search and its AI answersBrave
Out of training onlyDisallow GPTBot, ClaudeBot, Google-Extended, Applebot-ExtendedTraining, not answersOpenAI; Anthropic; Google; Apple

To see which pages AI crawlers can read on your site right now, and spot a tag you didn't mean to add, run the free AI visibility check.

Sources

  1. AI features and your website, Google Search Central. Read Nov 11, 2026.
  2. Robots meta tag specifications, Google Search Central. Read Nov 11, 2026.
  3. Introduction to robots.txt, Google Search Central. Read Oct 12, 2026.
  4. How Google interprets the robots.txt specification, Google Search Central. Read Nov 11, 2026.
  5. Google's common crawlers (Google-Extended), Google Search Central. Read Nov 11, 2026.
  6. New opportunities, control and insights for website owners, Google. Read Oct 8, 2026.
  7. Top ways to ensure your content performs well in Google's AI experiences on Search, Google Search Central. Read Nov 5, 2026.
  8. URL Inspection tool, Search Console Help. Read Oct 12, 2026.
  9. Overview of OpenAI Crawlers, OpenAI. Read Nov 11, 2026.
  10. Does Anthropic crawl data from the web, and how can site owners block the crawler?, Anthropic. Read Nov 11, 2026.
  11. Perplexity Crawlers, Perplexity. Read Nov 11, 2026.
  12. Bing Webmaster Guidelines, Microsoft Bing. Read Nov 11, 2026.
  13. Announcing new options for webmasters to control usage of their content in Bing Chat (September 2023), Microsoft Bing Webmaster Blog. Read Nov 11, 2026.
  14. FAQ - New AI Preferences, Bing Webmaster Help. Read Nov 5, 2026.
  15. About Applebot, Apple Support. Read Nov 5, 2026.
  16. About AmazonBot, Amazon Developer. Read Oct 7, 2026.
  17. Brave Search Crawler, Brave Search Help. Read Sep 30, 2026.
  18. Is DuckAssistBot related to DuckDuckGo?, DuckDuckGo Help. Read Oct 3, 2026.
FAQ

Common questions

Can I stop Google's AI Overviews quoting a page but keep it in search?

Partly. nosnippet stops the content being a direct input for AI Overviews and AI Mode, and max-snippet limits how much can be used, but both also apply to the page's regular search snippet. noindex removes the page from Search altogether. Google Google

Does blocking Google-Extended remove me from AI Overviews?

No. Google says Google-Extended controls Gemini training and Gemini app grounding, and does not affect inclusion in Google Search, which is where AI Overviews and AI Mode get their links. Google

Does blocking GPTBot keep my page out of ChatGPT?

Not out of ChatGPT search. GPTBot governs training; OpenAI says each bot setting is independent, and sites that opt out of OAI-SearchBot are the ones not shown in ChatGPT search answers. OpenAI

How do I keep a page out of Microsoft Copilot?

Today, Bing says NOARCHIVE prevents content being used in Copilot responses and grounding results, and NOINDEX keeps a URL out of Bing and Copilot. From 1 February 2027 Bing honors a page-level AI Grounding preference instead. Bing Webmaster Help Bing Webmaster Help

How long until AI answers stop using the page?

OpenAI says about 24 hours after a robots.txt change, Perplexity up to 24 hours, and DuckDuckGo about 72 hours. Google and Brave apply noindex once they fetch the page again. OpenAI Perplexity DuckDuckGo Help Brave Search Help

Find out which of your pages AI can read, and which it's quoting.

Enter your domain. The free AI visibility check shows which pages ChatGPT, Gemini, Perplexity and Google's AI answers can reach, flags noindex and blocking rules, and shows where AI already mentions you.