There’s some discussion over at Google Blogoscoped about why all of del.icio.us is now requesting not to be indexed, and whether or not it’s on purpose (to keep Googlebot away) or a bug that was accidentally applied to the top-level site.
I’m going with the bug theory. Why would Yahoo want to keep traffic away? Sure, it would be an incremental value-add for their own search engine if they index the site and others don’t, but I doubt users are going to dump Google in droves because of del.icio.us. With del.icio.us being more of a power-user tool, I suspect folks do like I do: click the icon in my firefox toolbar that takes me straight to the site.
I doubt that Yahoo is so short-sighted or arrogant that they would willingly block Google visitors from coming to a Yahoo property.
