Data sources

Every source Domduck reads, what its numbers mean, which direction is better, and how often it is re-read.

Every source Domduck reads, and what its numbers mean.

SourceReadsCadence
ahrefs_drDomain Rating. 0-100, higher is better.daily
trancoTranco rank. top 1M, lower is better.daily
openpagerankOpenPageRank. 0-10, higher is better.daily
cloudflare_radarCloudflare Radar. rank plus content categories.daily
rdapRegistration. registration date and status.once
self_tlsTLS certificate. issuer, validity, days remaining.daily
self_dnsDNS. A, AAAA, MX, NS, SPF, DMARC, DNSSEC.daily
robots_txtAI crawler policy. count of 8 tracked AI crawlers blocked, lower is better.daily

Direction is not obvious, so it is stated

tranco, cloudflare_radar and robots_txt are better when lower. ahrefs_dr and openpagerank are better when higher. Every history the MCP server returns carries a higherIsBetter field for this reason: a rank series read by something that assumes bigger is better is a chart drawn upside down.

Every reading is a dated observation

Nothing is ever overwritten. A reading is appended with the timestamp it was taken at, and the current value is a query over that history rather than a column.

The consequence worth knowing before you read a series: a missing day means nobody looked. It does not mean the value was zero and it does not mean the value held. That distinction is why the charts draw a gap for a day with no reading and a dashed line for a value carried past the last reading, and it is why null appears in an API response where a lazier design would send 0.

Notes per source

  • **ahrefs_dr** Domain Rating, 0-100, on a logarithmic scale, so 80 to 81 is a much larger move than 20 to 21.
  • **tranco** A research list that averages 30 days of several popularity rankings, so it is far more stable than any single day's list. Only the top 1M exists; outside it, there is correctly no reading rather than a bad one.
  • **openpagerank** 0-10. Free, coarse, and useful mainly as a second opinion when Domain Rating moves.
  • **cloudflare_radar** An exact rank for roughly the top 100 domains and a bucket ceiling below that, plus content categories.
  • **rdap** Registration date, registrar and status, from the registry through IANA's bootstrap. Read once: it does not change.
  • **self_tls** Our own TLS handshake with the domain. Issuer, validity window and days remaining.
  • **self_dns** A, AAAA, MX, NS, SPF, DMARC and DNSSEC, through a fixed DNS over HTTPS resolver.
  • **robots_txt** The AI crawler policy, plus an llms.txt presence probe on the same pass.

The eight AI crawlers

AgentOperatorBlocking it means
GPTBotOpenAINo ChatGPT browsing or training
OAI-SearchBotOpenAINo ChatGPT search surface
ClaudeBotAnthropicNo Claude retrieval
PerplexityBotPerplexityNo Perplexity citation
Google-ExtendedGoogleOut of Gemini and AI Overviews grounding, Search ranking unaffected
Applebot-ExtendedAppleOut of Apple Intelligence training
CCBotCommon CrawlOut of most open training corpora
BytespiderByteDanceOut of TikTok and Doubao

Full detail on the AI crawler page.