Enabling historical revisionism. (Linked elsewhere in the blogverse, natch.)
There's nothing stopping me, personally, physically, from writing a search that ignores the robots.txt file. I might just do it. Or maybe some kind of WH site-mirror that strips the HTML header off the file (might be copyright issues with that) so that copy would then get indexed normally. Hm. Hm. HM. A possible new project.