What is PerplexityBot?
PerplexityBot is Perplexity AI’s crawler. It fetches and reads web pages to build and refresh the index behind Perplexity’s answer engine — the thing that lets Perplexity generate a cited answer instead of a list of blue links. It’s not a training-data crawler in the GPTBot/ClaudeBot sense; it’s a search-index crawler, closer in purpose to OAI-SearchBot or Googlebot.
Separately, Perplexity-User is a different UA entirely: Perplexity’s own public framing — most visibly the title of its August 2025 rebuttal post “Agents or Bots?” — casts it as an agent acting on a user’s behalf rather than a bot, triggered when a live user asks a question that requires fetching a specific page in real time. The distinction matters for what blocking each one actually does, covered below.
In our classifier, PerplexityBot lands in the ai_search traffic class — alongside OAI-SearchBot, and distinct from ai_training (GPTBot, ClaudeBot, CCBot) and ai_user_action (ChatGPT-User, Claude-User, and Perplexity-User itself).
The user agent string
The commonly quoted strings, as aggregated across secondary sources:
PerplexityBot: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)
Perplexity-User: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Perplexity-User/1.0; +https://perplexity.ai/perplexity-user)
Worth hedging honestly: we could not confirm these literal strings by fetching Perplexity’s own docs directly — the page returned a JS shell rather than static markup we could grep. The version suffix (/1.0) should be treated like any vendor version number: current as of capture, not a permanent guarantee. Given the controversy below, treat the declared UA as one input, not the only one.
PerplexityBot vs Perplexity-User
This distinction is the one thing worth getting right before you touch robots.txt:
| PerplexityBot | Perplexity-User | |
|---|---|---|
| Triggered by | Scheduled/background index crawl | A specific live user question |
| Perplexity’s own framing | A crawler | An agent acting for a user, not a bot |
| Our traffic class | ai_search | ai_user_action |
| Subject to robots.txt (per Perplexity’s stated position) | Yes | Perplexity’s position is that agent-initiated fetches aren’t bound by robots.txt the same way a crawler is |
That second row isn’t us editorializing — it’s Perplexity’s own stated distinction, and it’s the crux of why blocking one doesn’t necessarily control the other.
The stealth-crawling controversy
This is the section people searching “perplexitybot” usually want, so here it is with dates and sourcing.
June 2024 — developer Robb Knight publicly documented Perplexity fetching pages disallowed in robots.txt using a user agent that didn’t match its declared crawler string. This is a separate, earlier incident from what follows — don’t conflate it with the 2025 report.
August 4, 2025 — Cloudflare published a detailed investigation (prompted by customer complaints) finding Perplexity crawling sites that had explicitly blocked PerplexityBot via both robots.txt and network-level Cloudflare blocks. Cloudflare’s direct quote: “Although Perplexity initially crawls from their declared user agent, when they are presented with a network block, they appear to obscure their crawling identity in an attempt to circumvent the website’s preferences.” Cloudflare also reported Perplexity “repeatedly modifying their user agent and changing their source ASNs to hide their crawling activity,” including impersonating a generic Chrome-on-macOS browser UA, and in some cases not fetching robots.txt at all before crawling. Cloudflare’s response was concrete: it de-listed Perplexity from its verified-bot program and added network rules targeting the identified behavior.
August 5, 2025 — Perplexity published a direct rebuttal the next day, titled “Agents or Bots? Making Sense of AI on the Open Web,” calling Cloudflare’s analysis fundamentally flawed and attributing a chunk of the traffic Cloudflare flagged to BrowserBase, a third-party cloud-browser service Perplexity says it uses for some user-driven fetches — i.e., in Perplexity’s framing, live agent activity on a user’s behalf, not stealth crawling. Cloudflare has not retracted its findings. We’re not adjudicating between the two: if you’re relying on this for a decision, read both posts and draw your own conclusion.
Net effect for a site owner: Perplexity’s own docs say blocking PerplexityBot works. The independently reported evidence above suggests the declared bot can be blocked while traffic attributed to Perplexity under rotated identities may not reliably respect that block. Both things can be true at once, which is an unsatisfying but accurate way to summarize it.
How to verify it’s really PerplexityBot
Perplexity publishes IP range lists as JSON:
https://www.perplexity.ai/perplexitybot.jsonhttps://www.perplexity.ai/perplexity-user.json
(Note: perplexity.com 302-redirects to perplexity.ai for both, as of this writing.) Cross-checking the source IP against these lists is the one verification mechanism we can point to with a primary source. Whether Perplexity also supports reverse-DNS/FCrDNS verification, and what PTR suffix it would use, is unclear — we found no authoritative confirmation, and given the UA-rotation findings above, we’d treat any third-party claim of Perplexity rDNS support with extra skepticism until Perplexity documents it directly.
Our own classifier currently has no IP-based Source entry for Perplexity (unlike OpenAI, Google, and Bing, which get CIDR-trie verification refreshed every 6 hours). PerplexityBot classification in our pipeline is UA-string pattern match only today — a real gap given everything above, and on our list to close by adding Perplexity’s published JSON as a verification source the same way we already do for OpenAI.
What the pipeline does today, precisely: a UA claiming “PerplexityBot” from inside a cloud provider’s or a non-AI vendor’s published range gets demoted to bot_other (the signals disagree); from an IP we know nothing about, the UA is trusted at face value — and because the check compares class families rather than companies, a range published by a different AI vendor currently counts as agreement too. Two honest consequences of that: a spoofer on a residential IP currently gets counted as the real thing, and — since Perplexity crawls from AWS IP space, which our classifier only knows as generic cloud infrastructure — a genuine PerplexityBot hit can itself end up demoted to bot_other. Ingesting Perplexity’s JSON fixes the second problem (a more specific Perplexity range beats the generic AWS match). The first one it won’t fix: catching a residential spoofer would require flipping to positive, allowlist-style verification — only count hits from published ranges — which the classifier deliberately doesn’t do, because it would silently drop every legitimate crawler whose operator publishes no ranges at all.
Should you block it?
The honest tradeoff, not a recommendation:
- Block PerplexityBot and you remove your site from Perplexity’s citation-backed answers going forward, at least for the declared-crawler path. Given the findings above, don’t assume a robots.txt Disallow is airtight against all Perplexity-attributed traffic.
- Allow PerplexityBot and you’re opted into Perplexity’s answer index — citation-traffic upside from being quoted, not training-data exposure (Perplexity’s stated purpose here is answer generation, not model training).
- Perplexity-User is a separate decision. Since Perplexity treats it as agent behavior rather than crawler behavior, blocking it is not guaranteed to be honored the same way a crawler block is, per Perplexity’s own framing.
To block PerplexityBot specifically:
User-agent: PerplexityBot
Disallow: /
We couldn’t independently confirm this exact snippet appears verbatim on Perplexity’s docs, but the token name is corroborated across enough sources to trust the syntax.
How much does PerplexityBot crawl?
We don’t have an official cadence to cite — Perplexity hasn’t published a crawl-frequency statement we could find, and we don’t yet have enough sites in our aggregate view to characterize typical volume ourselves. If you’re asking “is this normal for my site,” your own logs are the only reliable source right now.
See PerplexityBot in your own traffic
This is exactly the kind of question our MCP tools exist for. Ask Claude (or any MCP client connected to mcp-analytics):
“Break down my traffic by class for the last 30 days, then show me the top user agents in the ai_search bucket.”
That calls traffic_class_breakdown (see /mcp/tools/traffic-class-breakdown) for the class split, then top_user_agents (/mcp/tools/top-user-agents) for the exact UA strings hitting your site — including any claiming to be PerplexityBot that land in bot_other because the IP didn’t check out.
Instrumentation caveat: the default JS snippet only counts clients that execute JavaScript, and PerplexityBot doesn’t — crawler capture needs the server-side ingest (a Rack middleware that isn’t on RubyGems yet as we publish this; the install steps already sit on your settings page, waiting on the gem’s public release) alongside the snippet. Humans via snippet, bots via middleware, no double-counting.
Once enough sites contribute data, /ai-crawler-index will show how PerplexityBot’s volume compares across sites — illustrative today, live once we cross 50 contributing sites.
For the sibling bot in the same “AI search, not AI training” bucket, see what is OAI-SearchBot. For the bigger picture on why any of this taxonomy matters for SEO in an AI-search world, see LLM SEO and our general AI traffic analytics piece.