What changed
Every crawler policy, certificate, DNS and registration change Domduck has observed across the corpus, newest first. Each line is a difference between two readings of the same domain, so every one carries the date it was seen and both sides of the change. Nothing here is inferred and nothing states a motive: we read a file, not a decision.
2026-08-2021 – 3,0792026-10-19
A day at zero is a day the corpus was read and nothing moved, which is a different fact from a day with no reading.
Changes, 60 days18,363across published pages
Days recorded61with a reading
Busiest day3,07931 August 2026
Newest change17 August 20262026-08-17
The feed, page 139
last 90 days- 17 August 2026 (2026-08-17)
- discover.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- maven.orgrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- fastly.comThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 4 September 2026.
- zus.plrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- crossthegame.comThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 13 November 2026.
- crossthegame.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- batch.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- paramount.techThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 18 February 2027.
- paramount.techrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- whatismyipaddress.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- eksisozluk.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- nationalgeographic.orgrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- rferl.orgThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 4 November 2026.
- pngtree.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- jet2holidays.comThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 6 February 2027.
- jet2holidays.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- fakt.plrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- warframe.comThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 5 February 2027.
- warframe.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- streamlabs.comThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 9 November 2026.
- about.googleThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 28 October 2026.
- gitbook.comThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 3 November 2026.
- gitbook.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- crisp.chatrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- programmaticx.airobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- oecd-ilibrary.orgThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 3 November 2026.
- bartleby.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- learnings.airobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- epicentrk.uaThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 11 November 2026.
- hk01.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- googlemail.comThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 28 October 2026.
- alberta.carobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- hanime.tvrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- poynter.orgrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- control4.comThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 3 November 2026.
- windguru.czrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- copart.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- ok.xxxrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- suse.comA catch-all Disallow appeared between 3 August 2026 and 17 August 2026. It blocks every agent the file does not name.
- suse.comPerplexityBot was blocked between 3 August 2026 and 17 August 2026.
- suse.comOAI-SearchBot was blocked between 3 August 2026 and 17 August 2026.
- suse.comGPTBot was blocked between 3 August 2026 and 17 August 2026.
- suse.comGoogle-Extended was blocked between 3 August 2026 and 17 August 2026.
- suse.comClaudeBot was blocked between 3 August 2026 and 17 August 2026.
- suse.comCCBot was blocked between 3 August 2026 and 17 August 2026.
- suse.comBytespider was blocked between 3 August 2026 and 17 August 2026.
- suse.comApplebot-Extended was blocked between 3 August 2026 and 17 August 2026.
- cafepress.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- grow.googleThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 28 October 2026.
- knigavuhe.orgThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 6 November 2026.
- thairath.co.throbots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- 91mobiles.comThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 10 November 2026.
- gtranslate.netrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- telecomsvc.comThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 17 February 2027.
- mywot.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- catchthemes.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- icims.comThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 10 November 2026.
- dyz9.toprobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- thesimsresource.comThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 13 November 2026.
- instant-gaming.comThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 13 November 2026.
- ecer.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- cifraclub.com.brThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 30 October 2026.
- worldwildlife.orgThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 14 November 2026.
- wordstream.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- pricecharting.comThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 12 November 2026.
- nme.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- bandlab.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- sonyliv.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- tripo3d.aiGPTBot was blocked between 3 August 2026 and 17 August 2026.
- tripo3d.aiGoogle-Extended was blocked between 3 August 2026 and 17 August 2026.
- tripo3d.aiClaudeBot was blocked between 3 August 2026 and 17 August 2026.
- tripo3d.aiCCBot was blocked between 3 August 2026 and 17 August 2026.
- tripo3d.aiBytespider was blocked between 3 August 2026 and 17 August 2026.
- tripo3d.aiApplebot-Extended was blocked between 3 August 2026 and 17 August 2026.
- ixigo.comThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 24 December 2026.
- ixigo.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- pornzog.comThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 12 November 2026.
- pornzog.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- fsf.orgrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- icm.edu.plThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 6 February 2027.
- icm.edu.plrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- newsnow.co.ukrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- make.comThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 14 November 2026.
- hextom.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- pornoxo.comThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 10 November 2026.
- ksyun.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- adelaide.edu.aurobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- shaw.carobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- monopolygo.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- zalando.nlrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- myisolved.comThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 10 November 2026.
- dlink.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- mullvad.netThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 8 November 2026.
- famousbirthdays.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- scotsman.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- sportinglife.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- discovermagazine.comThe certificate was renewed between 3 August 2026 and 17 August 2026 and is valid to 10 November 2026.
- ingentaconnect.comrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- albumoftheyear.orgrobots.txt started returning HTTP 304 between 3 August 2026 and 17 August 2026, from 200.
- albumoftheyear.orgThe DMARC policy changed from (none) to none between 3 August 2026 and 17 August 2026.