# How to keep a page out of AI answers

> Sometimes the problem isn't being missing from AI answers, it's being in them with the wrong page: an old price list, a service you've dropped, a promotion that ended. We pulled together what Google, Microsoft, OpenAI, Anthropic, Perplexity, Apple, Amazon, Brave and DuckDuckGo document about keeping content out of their AI answers, from pages our team opened for earlier guides between September 30 and November 11, 2026. There is no single switch. Here is each one, and when fixing the page beats hiding it.

Updated November 13, 2026 · AI Syndicate team · https://www.aisyndicate.com/blog/how-to-keep-a-page-out-of-ai-answers/

**Quick answer:** Engine by engine. For Google's AI Overviews and AI Mode, use **nosnippet**, **max-snippet** or **noindex**; **Google-Extended** won't do it. For ChatGPT, Claude and Perplexity, block their **search bots** for that path in robots.txt. For Copilot, Bing's **NOARCHIVE** works today. Often the better fix is to **update the page** instead.

**Our take:** Hiding a page from AI is the last resort, not the first. Correcting what the page says usually solves the problem that made you want to hide it. Every control here works one engine at a time, and several also change normal search: Google's snippet rules apply to all of Search, and NOARCHIVE takes a page out of Copilot entirely. Robots.txt alone doesn't remove a page from Google or Brave, and fetchers acting for a user may ignore it. So we'd put the right facts at the same address first, and hide a page only when nothing true is left on it.

## What are the key facts about keeping a page out of AI answers?

| Engine | Control | What it does | Source |
| --- | --- | --- | --- |
| Google AI Overviews and AI Mode | `nosnippet` | Stops the content being used as "a direct input for AI Overviews and AI Mode" | Google |
| Google AI Overviews and AI Mode | `max-snippet` | "Will also limit how much of the content may be used" there | Google |
| Google Search, AI features included | `noindex` | Takes the page out of Search, and so out of its AI answers | Google |
| Google, whole site | Search Console setting, still a test (UK sites first) | Opts a site out of appearing in, and grounding, Google's generative AI features | Google |
| Google | `Google-Extended` | Gemini training and Gemini app grounding only; "does not impact a site's inclusion in Google Search" | Google |
| ChatGPT search | Disallow `OAI-SearchBot` | Opted-out sites are not shown in ChatGPT search answers; about 24 hours to take effect | OpenAI |
| Claude | Disallow `Claude-SearchBot` | Blocking "may reduce your site's visibility and accuracy in user search results" | Anthropic |
| Perplexity | Disallow `PerplexityBot` | Its search crawler; `Perplexity-User` "generally ignores robots.txt rules" | Perplexity |
| Microsoft Copilot | `NOARCHIVE` | "Prevents content from being used in Copilot responses and grounding results" | Bing |
| Microsoft Copilot, from 1 Feb 2027 | AI Grounding preference | Per-page Allow, DisallowExtended or Disallow; Allow is the default | Bing |
| DuckDuckGo | Disallow `DuckAssistBot` | Out of AI answers; "does not impact organic search rankings"; about 72 hours | DuckDuckGo |
| Brave Search | `noindex` | "robots.txt is not used to prevent a page from being indexed" | Brave |

## Can I keep one page out of AI answers and leave the rest of my site in?

Yes, for most engines, but each has its own control and there is no single switch. The controls come in two kinds: a tag on the page itself (a robots meta tag, or an `X-Robots-Tag` HTTP header for files), and a rule in robots.txt that names a crawler and a path.

First decide which of two things you mean:

- **Out of AI answers.** The search crawlers and snippet rules decide this: OAI-SearchBot for ChatGPT search, Claude-SearchBot for Claude, PerplexityBot for Perplexity, and Google's snippet and index rules for AI Overviews and AI Mode.
- **Out of AI training.** Different names: GPTBot, ClaudeBot, Google-Extended, Applebot-Extended. OpenAI says each of its bot settings is independent, so blocking GPTBot keeps pages out of training while OAI-SearchBot decides ChatGPT search. [OpenAI](https://developers.openai.com/api/docs/bots) Apple says pages that block Applebot-Extended "can still be included in search results." [Apple](https://support.apple.com/en-us/119829)

Google's Search team is direct about the trade-off: "More restrictive permissions will limit how your content is featured in our AI experiences." [Google Search Central Blog](https://developers.google.com/search/blog/2025/05/succeeding-in-ai-search) That's the point when you want a page out, and a risk when a tag lands on the wrong page.

## How do I keep a page out of Google's AI Overviews and AI Mode?

With the same controls that shape regular Search. Google lists `nosnippet`, `data-nosnippet`, `max-snippet` and `noindex` as the ways to limit what shows from your pages in Search, including its AI features. These apply to Search as a whole, so they also change how the page looks in ordinary results. [Google](https://developers.google.com/search/docs/appearance/ai-features)

- **Keep the page listed but not quoted:** `<meta name="robots" content="nosnippet">`. Google's robots meta tag docs say `nosnippet` also stops the content being used as "a direct input for AI Overviews and AI Mode," and that `max-snippet` "will also limit how much of the content may be used." [Google](https://developers.google.com/search/docs/crawling-indexing/robots-meta-tag)
- **Take the page out of Search altogether:** `noindex`, which tells Google "Do not show this page" in results. [Google](https://developers.google.com/search/docs/crawling-indexing/robots-meta-tag)
- **For a PDF or other file:** send the same rule as an `X-Robots-Tag` HTTP header, since you can't put a meta tag inside a PDF. [Google](https://developers.google.com/search/docs/crawling-indexing/robots-meta-tag)

**Don't also block the page in robots.txt.** If crawling is blocked, Google never sees the tag: the rules "will be ignored." [Google](https://developers.google.com/search/docs/crawling-indexing/robots-meta-tag) And robots.txt "is not a mechanism for keeping a web page out of Google"; a blocked page can still be listed, it just "won't have a description." [Google](https://developers.google.com/search/docs/crawling-indexing/robots/intro)

**For the whole site,** Google describes a Search Console setting that opts a site out of appearing in, and helping to ground, its generative AI features. It says sites that opt out "will not receive traffic or impressions" from those features. It is still a test: Google is rolling it out to a subset of site owners in the UK first, before offering it everywhere, so most sites won't see the setting yet. [Google](https://blog.google/products-and-platforms/products/search/new-controls-website-owners/)

**Google-Extended is not the switch.** Google says it controls training of future Gemini models and grounding in the Gemini app, and "does not impact a site's inclusion in Google Search nor is it used as a ranking signal in Google Search." [Google](https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers) For Search, Google says the control is your robots.txt rules for Googlebot, because AI "is built into Search." [Google](https://developers.google.com/search/docs/appearance/ai-features)

## How do I keep a page out of ChatGPT, Claude and Perplexity?

With robots.txt rules for their search crawlers, limited to the page's path. For example, to keep an old price list out of all three:

```
User-agent: OAI-SearchBot
User-agent: Claude-SearchBot
User-agent: PerplexityBot
Disallow: /old-price-list/
```

One catch: under the robots.txt rules Google documents, "only one group is valid for a particular crawler," the most specific one that names it. [Google](https://developers.google.com/search/docs/crawling-indexing/robots/robots_txt) **Our advice:** once a bot has its own group, it no longer reads your `User-agent: *` rules, so copy any other `Disallow` lines it should follow into the new group.

- **OpenAI.** OAI-SearchBot "is used to surface websites in search results in ChatGPT's search features," and sites that opt out of it are not shown in ChatGPT search answers. OpenAI says "it can take ~24 hours from a site's robots.txt update" for its systems to adjust. [OpenAI](https://developers.openai.com/api/docs/bots)
- **Anthropic.** Blocking Claude-SearchBot "may reduce your site's visibility and accuracy in user search results." Anthropic says its bots follow robots.txt, and warns that blocking its IP addresses instead "may not work correctly or persistently guarantee an opt-out." [Anthropic](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler)
- **Perplexity.** PerplexityBot "is designed to surface and link websites in search results on Perplexity." Perplexity says robots.txt changes can take up to 24 hours to apply. [Perplexity](https://docs.perplexity.ai/guides/bots)

**The gap: fetchers that act for a user.** When someone asks about your page directly, a different agent may fetch it. OpenAI says for ChatGPT-User "robots.txt rules may not apply," and Perplexity says Perplexity-User "generally ignores robots.txt rules." [OpenAI](https://developers.openai.com/api/docs/bots) [Perplexity](https://docs.perplexity.ai/guides/bots) Anthropic's Claude-User does follow robots.txt; blocking it stops Claude retrieving your page for that user. [Anthropic](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler) So a person who pastes your URL into a chat may still get a summary. That's a person asking for your page, not the AI finding it on its own.

**`noindex` doesn't reach them.** OpenAI, Anthropic and Perplexity document robots.txt controls for their bots, not robots meta tags or `X-Robots-Tag`. [OpenAI](https://developers.openai.com/api/docs/bots) [Anthropic](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler) [Perplexity](https://docs.perplexity.ai/guides/bots)

## How do I keep a page out of Microsoft Copilot?

With Bing's meta tags today, and with a new AI Grounding preference from 2027. Bing says to use `NOINDEX` when a URL should not appear in Bing search, Copilot experiences or grounding results, and that "NOARCHIVE prevents content from being used in Copilot responses and grounding results." [Bing Webmaster Help](https://www.bing.com/webmasters/help/webmaster-guidelines-30fba23a)

Bing's 2023 announcement, from before the Copilot name, spelled out the difference: "Content tagged NOARCHIVE will not be included in Bing Chat answers, not be linked to in the answers," while for `NOCACHE` "We will only display URL/Snippet/Title in the answer." [Microsoft Bing](https://blogs.bing.com/webmaster/september-2023/Announcing-new-options-for-webmasters-to-control-usage-of-their-content-in-Bing-Chat)

**What changes in 2027.** Microsoft announced on 30 October 2026 that separate AI Training and AI Grounding preferences replace NOARCHIVE and NOCACHE for those uses, and it starts honoring them on 1 February 2027. Grounding is set per page with a meta tag or HTTP header (Allow, DisallowExtended or Disallow), and Allow is the default. A site that uses NOARCHIVE and wants to stay out of grounding must publish a page-level AI Grounding preference, and record a temporary one in Bing Webmaster Tools, before 1 March 2027. [Bing Webmaster Help](https://www.bing.com/webmasters/help/new-ai-preferences-2t1cc16d)

> **Check this first:** many sites added NOARCHIVE years ago to hide cached copies. On Bing that same tag now keeps the page out of Copilot, wanted or not.

## What about Apple, Amazon, Brave and DuckDuckGo?

- **Apple (Siri, Spotlight, Safari).** Applebot-Extended only controls whether your content trains Apple's models; it "does not crawl webpages," and pages that block it "can still be included in search results." Apple says the `X-Robots-Tag` header is "useful for non-HTML resources (such as PDFs or images)" when you want to give Applebot rules. [Apple](https://support.apple.com/en-us/119829)
- **Amazon (Alexa+).** Amazon honors robots meta tags: `noarchive` means "do not use the page for model training" and `noindex` means do not index it. Amazon says Amzn-SearchBot makes content eligible for Alexa and Amzn-User fetches pages for live answers, so on our reading those are the two to name in robots.txt. Amazon's crawlers don't support `crawl-delay`. [Amazon Developer](https://developer.amazon.com/amazonbot)
- **Brave Search.** Brave's crawler has no separate user agent and follows your robots.txt rules for Googlebot. It says "robots.txt is not used to prevent a page from being indexed": add `noindex`, then submit the URL so Brave fetches it again. For pages that no longer exist, you can also email not-found@brave.com. [Brave Search Help](https://search.brave.com/help/brave-search-crawler)
- **DuckDuckGo.** DuckAssistBot "crawls pages in real-time for our AI-assisted answers." Blocking it "does not impact organic search rankings" and removes your pages from AI answers in about 72 hours. [DuckDuckGo Help](https://duckduckgo.com/duckduckgo-help-pages/results/duckassistbot)

## What doesn't keep a page out of AI answers?

- **Google-Extended, for AI Overviews and AI Mode.** It doesn't affect inclusion in Search. [Google](https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers)
- **robots.txt alone, for Google and Brave.** Both say it isn't a way to remove a page from their index. [Google](https://developers.google.com/search/docs/crawling-indexing/robots/intro) [Brave Search Help](https://search.brave.com/help/brave-search-crawler)
- **robots.txt plus noindex on the same page.** Google can't see a tag on a page it may not crawl. [Google](https://developers.google.com/search/docs/crawling-indexing/robots-meta-tag)
- **Blocking training bots.** GPTBot, ClaudeBot and Applebot-Extended govern training, not search answers. [OpenAI](https://developers.openai.com/api/docs/bots) [Apple](https://support.apple.com/en-us/119829)
- **Blocking IP addresses.** Anthropic warns it "may not work correctly or persistently guarantee an opt-out." [Anthropic](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler)
- **Expecting user-triggered fetchers to obey.** ChatGPT-User and Perplexity-User may not follow robots.txt. [OpenAI](https://developers.openai.com/api/docs/bots) [Perplexity](https://docs.perplexity.ai/guides/bots)

## Should I hide the outdated page or fix it?

Usually fix it. **Our advice, not a vendor statement:** if AI keeps quoting a 2024 price list, hiding that page doesn't put this year's prices in front of anyone. The engine just quotes something else, possibly a directory with the same old number. Replacing the content at the same address gives every engine the right facts the next time it reads the page.

Hide a page when nothing true is left on it, or when it shouldn't be public at all, such as a page for a staff member who has left or an offer you can no longer honor. Otherwise:

1. **Update the page** at the same URL with the current facts, and say plainly what changed.
2. **Tell the engines it changed:** update `lastmod` in your sitemap and send the URL through IndexNow. See [do sitemaps help AI search?](/blog/do-sitemaps-help-ai-search/)
3. **Fix the copies elsewhere:** your Google Business Profile, Yelp and directory listings should show the same facts. See [should you show prices to AI?](/blog/should-you-show-prices-to-ai/)
4. **Check again in a day or two,** with the questions customers actually ask. When AI is simply wrong about you, our guide to [what to do when AI gets your business wrong](/blog/ai-wrong-about-your-business/) has the rest.

## How do I take one page out of AI answers, step by step?

1. **Decide what you want gone:** the page from search entirely, or only from being quoted in AI answers. The first is `noindex`; the second is `nosnippet` plus crawler rules.
2. **For Google, add the tag and keep the page crawlable.** `nosnippet` keeps it out of AI Overviews and AI Mode as a direct input; `noindex` removes it from Search. Don't block it in robots.txt, or Google won't see the tag. [Google](https://developers.google.com/search/docs/crawling-indexing/robots-meta-tag)
3. **For ChatGPT, Claude and Perplexity, add a robots.txt rule.** Disallow the page's path for OAI-SearchBot, Claude-SearchBot and PerplexityBot. [OpenAI](https://developers.openai.com/api/docs/bots) [Anthropic](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler) [Perplexity](https://docs.perplexity.ai/guides/bots)
4. **For Copilot, add NOARCHIVE.** Before 1 March 2027, switch to a page-level AI Grounding preference. [Bing Webmaster Help](https://www.bing.com/webmasters/help/webmaster-guidelines-30fba23a) [Bing Webmaster Help](https://www.bing.com/webmasters/help/new-ai-preferences-2t1cc16d)
5. **For DuckDuckGo and Alexa, add their bots to the same robots.txt rule:** DuckAssistBot, Amzn-SearchBot and Amzn-User. [DuckDuckGo Help](https://duckduckgo.com/duckduckgo-help-pages/results/duckassistbot) [Amazon Developer](https://developer.amazon.com/amazonbot)
6. **For Brave, rely on the noindex,** then submit the URL so Brave fetches it again. [Brave Search Help](https://search.brave.com/help/brave-search-crawler)
7. **For PDFs and images, send the rule as an `X-Robots-Tag` header.** [Google](https://developers.google.com/search/docs/crawling-indexing/robots-meta-tag)
8. **Check it took.** Search Console's URL Inspection live test "fetches and examines the URL in real time" and reports robots.txt blocks and noindex. [Google Search Console Help](https://support.google.com/webmasters/answer/9012289) Then allow about 24 hours for OpenAI and Perplexity and about 72 hours for DuckDuckGo before asking the AI again. [OpenAI](https://developers.openai.com/api/docs/bots) [Perplexity](https://docs.perplexity.ai/guides/bots) [DuckDuckGo Help](https://duckduckgo.com/duckduckgo-help-pages/results/duckassistbot)

## Which control should I use for which engine?

| Goal | Use | Engines it covers | Source |
| --- | --- | --- | --- |
| Listed, but not quoted by Google's AI | `nosnippet` or a small `max-snippet` | Google AI Overviews and AI Mode (and regular snippets) | Google |
| Gone from Google entirely | `noindex`, page left crawlable | Google Search, AI features included | Google |
| Whole site out of Google's AI features | Search Console AI setting (a test, UK sites first) | Google | Google |
| Out of ChatGPT search | Disallow `OAI-SearchBot` for the path | ChatGPT search | OpenAI |
| Out of Claude's search results | Disallow `Claude-SearchBot` (and `Claude-User`) | Claude | Anthropic |
| Out of Perplexity search | Disallow `PerplexityBot` | Perplexity search | Perplexity |
| Out of Copilot | `NOARCHIVE` now; AI Grounding preference from 2027 | Microsoft Copilot | Bing |
| Out of DuckDuckGo AI answers | Disallow `DuckAssistBot` | DuckDuckGo Search Assist | DuckDuckGo |
| Out of Alexa+ | Disallow `Amzn-SearchBot` and `Amzn-User` | Alexa | Amazon |
| Out of Brave | `noindex`, then resubmit the URL | Brave Search and its AI answers | Brave |
| Out of training only | Disallow `GPTBot`, `ClaudeBot`, `Google-Extended`, `Applebot-Extended` | Training, not answers | OpenAI; Anthropic; Google; Apple |

To see which pages AI crawlers can read on your site right now, and spot a tag you didn't mean to add, run the free [AI visibility check](/#tool).

## FAQ

### Can I stop Google's AI Overviews quoting a page but keep it in search?

Partly. nosnippet stops the content being a direct input for AI Overviews and AI Mode, and max-snippet limits how much can be used, but both also apply to the page's regular search snippet. noindex removes the page from Search altogether. [Google](https://developers.google.com/search/docs/crawling-indexing/robots-meta-tag) [Google](https://developers.google.com/search/docs/appearance/ai-features)

### Does blocking Google-Extended remove me from AI Overviews?

No. Google says Google-Extended controls Gemini training and Gemini app grounding, and does not affect inclusion in Google Search, which is where AI Overviews and AI Mode get their links. [Google](https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers)

### Does blocking GPTBot keep my page out of ChatGPT?

Not out of ChatGPT search. GPTBot governs training; OpenAI says each bot setting is independent, and sites that opt out of OAI-SearchBot are the ones not shown in ChatGPT search answers. [OpenAI](https://developers.openai.com/api/docs/bots)

### How do I keep a page out of Microsoft Copilot?

Today, Bing says NOARCHIVE prevents content being used in Copilot responses and grounding results, and NOINDEX keeps a URL out of Bing and Copilot. From 1 February 2027 Bing honors a page-level AI Grounding preference instead. [Bing Webmaster Help](https://www.bing.com/webmasters/help/webmaster-guidelines-30fba23a) [Bing Webmaster Help](https://www.bing.com/webmasters/help/new-ai-preferences-2t1cc16d)

### How long until AI answers stop using the page?

OpenAI says about 24 hours after a robots.txt change, Perplexity up to 24 hours, and DuckDuckGo about 72 hours. Google and Brave apply noindex once they fetch the page again. [OpenAI](https://developers.openai.com/api/docs/bots) [Perplexity](https://docs.perplexity.ai/guides/bots) [DuckDuckGo Help](https://duckduckgo.com/duckduckgo-help-pages/results/duckassistbot) [Brave Search Help](https://search.brave.com/help/brave-search-crawler)

**Get the platform: AI Pulse, $99/month** ($990/year). The complete AI Syndicate platform for one business in one market: every feature, all 12 AI engines, 2,000 tokens a month, cancel anytime. [Compare plans](https://www.aisyndicate.com/pricing/)

## Sources

1. [AI features and your website](https://developers.google.com/search/docs/appearance/ai-features), Google Search Central. Read Nov 11, 2026.
2. [Robots meta tag specifications](https://developers.google.com/search/docs/crawling-indexing/robots-meta-tag), Google Search Central. Read Nov 11, 2026.
3. [Introduction to robots.txt](https://developers.google.com/search/docs/crawling-indexing/robots/intro), Google Search Central. Read Oct 12, 2026.
4. [How Google interprets the robots.txt specification](https://developers.google.com/search/docs/crawling-indexing/robots/robots_txt), Google Search Central. Read Nov 11, 2026.
5. [Google's common crawlers (Google-Extended)](https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers), Google Search Central. Read Nov 11, 2026.
6. [New opportunities, control and insights for website owners](https://blog.google/products-and-platforms/products/search/new-controls-website-owners/), Google. Read Oct 8, 2026.
7. [Top ways to ensure your content performs well in Google's AI experiences on Search](https://developers.google.com/search/blog/2025/05/succeeding-in-ai-search), Google Search Central. Read Nov 5, 2026.
8. [URL Inspection tool](https://support.google.com/webmasters/answer/9012289), Search Console Help. Read Oct 12, 2026.
9. [Overview of OpenAI Crawlers](https://developers.openai.com/api/docs/bots), OpenAI. Read Nov 11, 2026.
10. [Does Anthropic crawl data from the web, and how can site owners block the crawler?](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler), Anthropic. Read Nov 11, 2026.
11. [Perplexity Crawlers](https://docs.perplexity.ai/guides/bots), Perplexity. Read Nov 11, 2026.
12. [Bing Webmaster Guidelines](https://www.bing.com/webmasters/help/webmaster-guidelines-30fba23a), Microsoft Bing. Read Nov 11, 2026.
13. [Announcing new options for webmasters to control usage of their content in Bing Chat (September 2023)](https://blogs.bing.com/webmaster/september-2023/Announcing-new-options-for-webmasters-to-control-usage-of-their-content-in-Bing-Chat), Microsoft Bing Webmaster Blog. Read Nov 11, 2026.
14. [FAQ - New AI Preferences](https://www.bing.com/webmasters/help/new-ai-preferences-2t1cc16d), Bing Webmaster Help. Read Nov 5, 2026.
15. [About Applebot](https://support.apple.com/en-us/119829), Apple Support. Read Nov 5, 2026.
16. [About AmazonBot](https://developer.amazon.com/amazonbot), Amazon Developer. Read Oct 7, 2026.
17. [Brave Search Crawler](https://search.brave.com/help/brave-search-crawler), Brave Search Help. Read Sep 30, 2026.
18. [Is DuckAssistBot related to DuckDuckGo?](https://duckduckgo.com/duckduckgo-help-pages/results/duckassistbot), DuckDuckGo Help. Read Oct 3, 2026.
