Docs
Search docs...⌘K

Crawled pages

API only
GET/v3/site-explorer/crawled-pages

Query parameters

selectstringRequired

A comma-separated list of fields to return. Available fields: url, http_code, last_crawled, first_seen, last_attempt, url_rating, title.

order_bystring

A field to order the results by, with optional direction. Format: field_name:asc or field_name:desc (default desc). Available fields: url, http_code, last_crawled, first_seen, last_attempt, url_rating, title.

wherestring

Supported fields:

  • url
  • http_code
  • last_crawled
  • first_seen
  • last_attempt
  • url_rating
  • title

Filter is a JSON string that represents a boolean expression, it should follow the following structure:

<bool_filter> ::= { "and" : <bool_filter>+ }
              |   { "or" : <bool_filter>+ }
              |   { "not" : <bool_filter> }
              |   <expr>

<expr> ::= {
             "field" : <dimension|metric>,
             ? "is": <condition>,
           }

<condition> ::= [ "eq", <value> ]
            |   [ "neq", <value> ]
            |   [ "gt", <value> ]
            |   [ "gte", <value> ]
            |   [ "lt", <value> ]
            |   [ "lte", <value> ]
            |   [ "substring", <value> ]
            |   [ "isubstring", <value> ]
            |   [ "phrase_match", <value> ]
            |   [ "iphrase_match", <value> ]
            |   [ "prefix", <value> ]
            |   [ "suffix", <value> ]
limitinteger

The number of results to return.

Default:1000
protocolstring

The protocol of your target.

Allowed values:bothhttphttps
Default:both
modestring

The scope of the search based on the target you entered.

Allowed values:exactprefixdomainsubdomains
Default:subdomains
targetstringRequired

The target of the search: a domain or a URL.

outputstring

The output format.

Allowed values:jsoncsvxmlphp

Responses

pagesarray<object>
urlstring

The URL of the crawled page.

http_codeinteger

The HTTP status code returned when the page was last crawled.

last_crawledstring

The timestamp of the last successful crawl.

first_seenstring

The timestamp when the URL was first seen by Ahrefs.

last_attemptstring

The timestamp of the last crawl attempt.

url_ratingnumber

URL Rating of the page at the time of the last crawl, on a 100-point scale.

titlestring or null

The page's meta title.