mad_asshatter@lemmy.world to Technology@lemmy.worldEnglish · 7 months agoThe Internet Archive is under a DDoS attackbsky.appexternal-linkmessage-square214fedilinkarrow-up11.36Karrow-down17file-text
arrow-up11.35Karrow-down1external-linkThe Internet Archive is under a DDoS attackbsky.appmad_asshatter@lemmy.world to Technology@lemmy.worldEnglish · 7 months agomessage-square214fedilinkfile-text
minus-squareGeneral_Effort@lemmy.worldlinkfedilinkEnglisharrow-up1·7 months agohttps://blog.archive.org/2017/04/17/robots-txt-meant-for-search-engines-dont-work-well-for-web-archives/ Does this answer the question?
https://blog.archive.org/2017/04/17/robots-txt-meant-for-search-engines-dont-work-well-for-web-archives/
Does this answer the question?