Docs
Search docs...⌘K

Crawled pages

API + MCP
GET/v3/site-explorer/crawled-pages

Query parameters

selectstringRequired

A comma-separated list of fields to return.

order_bystring

A field to order the results by, with optional direction. Format: field_name:asc or field_name:desc (default desc).

wherestring

Supported fields:

targetstring (url)Required

The target of the search: a domain or a URL.

modestring

The scope of the target.

Allowed values:exactprefixdomainsubdomains
Default:subdomains
protocolstring

The protocol of your target.

Allowed values:bothhttphttps
Default:both
limitinteger

The number of results to return.

Default:1000
outputstring

The output format.

Allowed values:jsonphpxml

Responses

pagesarray<object>
first_seenstring (date-time)

The timestamp when the URL was first seen by Ahrefs.

http_codeinteger

The HTTP status code returned when the page was last crawled.

languagesarray<string>

Language codes detected in the page content, most confident first. Equality filters match pages having the given code.

last_attemptstring (date-time)

The timestamp of the last crawl attempt.

last_crawledstring (date-time)

The timestamp of the last successful crawl.

page_categorystring or null

Comma-separated page categories, e.g. '/Arts_and_Entertainment,/Finance'. Only the most nested element per parent group is included. Filter with a quoted path, e.g. ["substring", "\"/Finance\""], to match that path exactly.

page_typestring or null

Comma-separated page types, e.g. '/Article/How_to,/Landing_Page'. Only the most nested element per parent group is included. Filter with a quoted path, e.g. ["substring", "\"/Article/How_to\""], to match that path exactly.

publish_datestring (date-time) or null

The date the page was published, preferring the first publication date over the date stated in the HTML.

titlestring or null

The page's meta title.

urlstring (url)

The URL of the crawled page.

url_ratingnumber (float)

URL Rating of the page at the time of the last crawl, on a 100-point scale.