Guide

Does AI cite paywalled or gated content?

Members-only articles, paid courses, client portals, white papers behind an email form: many businesses keep their best material behind a gate. Can ChatGPT, Claude, Perplexity, Copilot or Google's AI answers read and cite it? Here is what the official docs say as of November 11, 2026, what nobody has documented, and how to gate content without vanishing from AI answers.

Quick answer

Mostly no. Google says it will not crawl pages that need a login, and its paywall markup only lets Googlebot see full gated text. OpenAI, Anthropic and Perplexity say nothing about paywalls or logins on their crawler pages. An email-gated PDF shows a crawler only the form. Keep a public summary page if you want citations.

What are the key facts about AI and paywalled content?

FactDetailSource
Login pages"If a page is made private, such as requiring a log-in to view it, Googlebot will not crawl it."Google
Public pages onlyGoogle indexes pages "accessible to the public and which don't block our crawler"Google
Paywalls are not cloakingNot cloaking if Google "can see the full content of what's behind the paywall" and you follow Flexible Sampling guidanceGoogle
Paywall markupisAccessibleForFree set to false, plus hasPart with a cssSelector that points at the paywalled sectionGoogle
Free samplesGoogle suggests metering or lead-ins; the exact amount is left to publishersGoogle
Google's AI featuresA page must be indexed and eligible for a snippet; "no additional technical requirements"Google
Snippet limitsnosnippet and max-snippet also limit what AI Overviews and AI Mode can useGoogle
OpenAI, Anthropic, PerplexityTheir crawler pages do not mention logins, paywalls or subscriptionsOpenAI, Anthropic, Perplexity
Anthropic and CAPTCHAsAnthropic says it "will not attempt to bypass CAPTCHAs" on sites it crawlsAnthropic
Bing chat answersNOARCHIVE content is not included or linked in Bing Chat answers; NOCACHE shows only URL, title and snippet (2023)Microsoft Bing
Charging AI crawlersCloudflare pay per crawl answers unpaid crawlers with HTTP 402; in closed betaCloudflare

Can Google read content behind a paywall?

Only if you let Googlebot in and label the gated part. Google is blunt about logins: Google only indexes pages "accessible to the public and which don't block our crawler," and "If a page is made private, such as requiring a log-in to view it, Googlebot will not crawl it." Google

Publishers can show Googlebot the full text of paid articles. Normally, showing Google something different from what visitors see is cloaking, which breaks Google's spam policies. Paywalls are the exception: "we don't consider this to be cloaking" if Google "can see the full content of what's behind the paywall," just like a person with access, and you follow its Flexible Sampling guidance. Google

The label is structured data. Google's paywalled content guide says the markup "helps Google differentiate paywalled content from the practice of cloaking." You set isAccessibleForFree to false, add a hasPart element of type WebPageElement, and give it a cssSelector that names the class wrapping the gated section. Google says to use only .class selectors, and to list several class names as an array if a page has more than one gated section. It works on CreativeWork types such as Article, NewsArticle, Blog, Course and WebPage, and Google adds: "This guide only applies to content that you want crawled and indexed." Google

Flexible sampling is the other half, in two kinds. Metering gives readers "a quota of articles to consume before requiring users to subscribe or log in." A lead-in "offers a portion of an article's content without it being shown in full." For most daily news publishers Google expects 6 to 10 free articles per user per month, but says "We leave the exact number to the discretion of individual publishers." Google

In plain words: a paywall Googlebot can see past, with the right markup, is allowed. A login wall Googlebot can't see past is simply not crawled.

Can paywalled content show up in Google AI Overviews or AI Mode?

Google has not said either way. Its AI features page says a page only has to be "indexed and eligible to be shown in Google Search with a snippet," and "There are no additional technical requirements." The page does not mention paywalls or logins. Google

In practice:

  • A login-only page is not crawled, so it can't be indexed or meet the bar. Google
  • A marked-up paywalled article that Google has indexed meets the stated bar on paper. We found no Google statement confirming that the gated part is used in AI Overviews or AI Mode.
  • Snippet controls apply. Google's robots meta tag docs say nosnippet also stops content being used as a direct input for AI Overviews and AI Mode, and that max-snippet "will also limit how much of the content may be used" there. Google

Our advice, not a vendor statement: to be quoted in Google's AI answers, put the part that answers the question above the gate, in the free lead-in. Don't rely on Google pulling a sentence from the locked section. For more, read how to get cited in Google AI Overviews.

Do ChatGPT, Claude and Perplexity read pages behind a login or paywall?

Nothing in their documentation says they do. We read the official crawler pages from OpenAI, Anthropic and Perplexity on 11 November 2026. None mentions logins, paywalls, subscriptions or gated content. What each does say:

  • OpenAI. OAI-SearchBot "is used to surface websites in search results in ChatGPT's search features." ChatGPT-User fetches pages when a person asks, and "is not used for crawling the web in an automatic fashion." Nothing on the page describes it signing in or using a user's credentials. OpenAI
  • Anthropic. Claude-User fetches pages when people ask Claude questions; Claude-SearchBot improves search results; ClaudeBot collects training data. The page says "Anthropic's Bots respect anti-circumvention technologies" and that "we will not attempt to bypass CAPTCHAs for the sites we crawl." Anthropic
  • Perplexity. PerplexityBot "is designed to surface and link websites in search results on Perplexity." Perplexity-User fetches pages for user requests, and "generally ignores robots.txt rules" because a user asked. Nothing is said about logins. Perplexity

Google's paywall markup is a Google rule: it tells Googlebot the gated text is not cloaking. No other AI company documents an equivalent. If your server unlocks the full article only for Googlebot, ChatGPT, Claude and Perplexity are not covered, and nothing they publish says they see more than a logged-out visitor.

Our advice, not a vendor statement: assume every AI fetcher, including the ones acting for a signed-in user, sees what a stranger with no account sees. If the answer is behind the gate, it is unconfirmed whether any of them can quote it.

Bing is the one engine with a published control for chat answers. In September 2023 Microsoft said "Content tagged NOARCHIVE will not be included in Bing Chat answers, not be linked to in the answers," while for NOCACHE "We will only display URL/Snippet/Title in the answer." Microsoft Bing That post predates Microsoft's Copilot branding, and we found no newer Bing page on paywalls, so treat it as the last published word.

Will AI cite my email-gated white paper or guide?

Usually not the file, because a crawler can't fill in your form. A bot sees the landing page: a headline, a few bullets and an email box. If that page doesn't state the findings, there is nothing to cite. Google's rule for login walls applies to anything private: a page that needs a log-in to view is not crawled. Google

A twist: many "gated" PDFs are not locked at all. The form only hides the link, and the file sits at a public address. If that address leaks into a sitemap, an email or another site, a crawler can fetch it like any other file. Our guide to whether AI cites PDFs covers what Google documents about PDFs and what the AI companies don't.

Our advice, not a vendor statement: treat the landing page as the citable version. Publish an open summary page with the headline findings, the key numbers and the date, written in full sentences. Keep the full report, the spreadsheets and the templates behind the form. If the file must stay private, protect it on the server, not just by hiding the link.

This is not legal advice. If your content is licensed or covered by a contract, check those terms before publishing any part of it openly.

Can I get paid when AI reads my gated content?

A few routes exist, mostly built for large publishers. These we could confirm from the companies' own pages:

  • Perplexity Publishers Program (July 30, 2024). Perplexity said: "When Perplexity earns revenue from an interaction where a publisher's content is referenced, that publisher will also earn a share." The launch partners were "TIME, Der Spiegel, Fortune, Entrepreneur, The Texas Tribune, and WordPress.com." Perplexity
  • Google and Reddit (February 22, 2024). Google said it gained access to "Reddit's Data API, which delivers real-time, structured, unique content," which it can "display, train on, and otherwise use." Google
  • Cloudflare pay per crawl (July 1, 2025). Cloudflare describes it as a way "to allow content creators to charge AI crawlers for access to their content." For each crawler a site can allow free access, charge a flat per-request price, or block. A crawler that hasn't agreed to pay gets "a 402 Payment Required response with pricing." Cloudflare Cloudflare's docs say "Pay per crawl is currently in closed beta." Cloudflare

Other licensing deals between AI companies and news publishers have been widely reported. We could not open their primary announcements on 11 November 2026, so we don't describe their terms.

Our advice, not a vendor statement: for a small business, charging AI crawlers gets you paid for training or bulk access, not cited. A bot that gets a 402 can't read the page, so it can't quote it. Before changing any Cloudflare setting, read is Cloudflare blocking AI crawlers? so you don't block the search bots that send customers.

How do I gate content without disappearing from AI answers?

Split each gated piece into an open answer and paid depth:

  1. List what you gate. Paid articles, member pages, course lessons, white papers, price sheets, client-only FAQs.
  2. Write the open answer for each. One to three paragraphs a stranger can read, with the main claim, the key number and the date. Put it at the top of the public page.
  3. Gate the depth, not the summary. Templates, full datasets, worked examples, video lessons and downloads go behind the login or form.
  4. If you run a true paywall, mark it up for Google. Use isAccessibleForFree: false with hasPart and a .class cssSelector around the locked section, and follow Google's flexible sampling advice. Google Google
  5. Never show bots text visitors can't get. Outside Google's paywall exception, that is cloaking. Google
  6. Protect private files on the server. Hiding a link is not a lock.
  7. Keep the gate itself crawlable. The landing page should not be blocked in robots.txt, or nobody's search bot can read the open part either.
  8. Decide on training separately. To keep gated material out of training, use each vendor's training control rather than hiding the whole page. OpenAI says disallowing GPTBot "indicates a site's content should not be used in training generative AI foundation models." OpenAI

Steps 2, 3, 6 and 7 are our advice, not a vendor statement.

What should be on my gated-content checklist for AI search?

CheckDone whenSource
Open answer existsEach gated piece has a public page that answers the main question in full sentencesOur advice
Answer sits above the gateThe free lead-in holds the claim, the number and the dateGoogle; our advice
Paywall markupisAccessibleForFree is false and hasPart points at the gated section with a .class selectorGoogle
Sampling chosenYou offer metering or a lead-in to searchersGoogle
No cloakingBots never get text visitors can't get, outside Google's paywall exceptionGoogle
Login pages expected to drop outYou accept that pages needing a log-in won't be crawled by GoogleGoogle
Snippet limits checkedNo stray nosnippet or tight max-snippet on pages you want quotedGoogle
Bing chat tags checkedNOARCHIVE only where you really want out of Bing chat answersMicrosoft Bing
Private files lockedGated PDFs need a session or token, not just a hidden linkOur advice
Crawler pricing reviewedAny Cloudflare block or charge setting is a deliberate choiceCloudflare

How do I check what AI can see behind my gate?

Look at the page as a logged-out stranger: open it in a private browser window with no cookies. What you can read there is the most any AI engine can be assumed to read. If your customer's answer isn't on that screen, it is unconfirmed whether any engine can cite it.

For Google, use the URL Inspection tool in Search Console. Google's AI features page suggests it to confirm Googlebot receives the correct HTML. Google

Next, check server logs for OAI-SearchBot, ChatGPT-User, Claude-User, PerplexityBot and Perplexity-User. Note which status codes they get on gated URLs: a login redirect, a 402 or a 403 means they saw nothing. Our guide to finding AI bots in your server logs shows how.

Then ask the engines. Our free AI visibility check shows which of your pages ChatGPT, Gemini, Perplexity and Google's AI answers cite. The Prompt Simulator asks the exact question your gated guide answers and shows whose page each engine quotes instead.

Sources

  1. Google Search technical requirements, Google Search Central. Read Nov 11, 2026.
  2. Spam policies for Google web search, Google Search Central. Read Nov 11, 2026.
  3. Subscription and paywalled content (structured data), Google Search Central. Read Nov 11, 2026.
  4. Flexible sampling, Google Search Central. Read Nov 11, 2026.
  5. AI features and your website, Google Search Central. Read Nov 11, 2026.
  6. Robots meta tag specifications, Google Search Central. Read Nov 11, 2026.
  7. Overview of OpenAI Crawlers, OpenAI. Read Nov 11, 2026.
  8. Does Anthropic crawl data from the web, and how can site owners block the crawler?, Anthropic. Read Nov 11, 2026.
  9. Perplexity Crawlers, Perplexity. Read Nov 11, 2026.
  10. Announcing new options for webmasters to control usage of their content in Bing Chat (September 2023), Microsoft Bing Webmaster Blog. Read Nov 11, 2026.
  11. Introducing the Perplexity Publishers' Program (July 2024), Perplexity. Read Nov 11, 2026.
  12. An expanded partnership with Reddit (February 2024), Google. Read Nov 11, 2026.
  13. Introducing pay per crawl (July 2025), Cloudflare Blog. Read Nov 11, 2026.
  14. What is pay per crawl?, Cloudflare Docs. Read Nov 11, 2026.
FAQ

Common questions

Does Google index pages that need a login?

No. Google says that if a page requires a log-in to view it, Googlebot will not crawl it, and Google only indexes pages accessible to the public. Google

Is showing Googlebot my full paywalled article cloaking?

Not if Google can see the full content just as a paying reader can and you follow its Flexible Sampling guidance. Mark the gated section with paywalled content structured data so Google can tell it apart from cloaking. Google Google

Can ChatGPT, Claude or Perplexity log in to read my members-only pages?

Unconfirmed. Their crawler pages say nothing about logins or paywalls, and Anthropic says it will not attempt to bypass CAPTCHAs. Assume they see what a logged-out visitor sees. OpenAI Anthropic Perplexity

Will AI cite my email-gated white paper?

Not reliably, because a crawler sees only the form page. Publish an open summary with the key findings and keep the full file behind the form. Google won't crawl pages that need a log-in. Google

Can I charge AI crawlers to read my content?

Cloudflare's pay per crawl lets a site allow, charge or block each AI crawler, answering unpaid requests with HTTP 402. It is in closed beta. Cloudflare Cloudflare

See what AI can read past your gate.

Enter your domain. The free AI visibility check shows which of your pages ChatGPT, Gemini, Perplexity and Google's AI answers cite, and which ones they can't get into.