Some people who run wikis that are publicly accessible have asked us for the ability to set a robots.txt that tells polite search engines like Google, MSN, Alexa, and Yahoo to please not index the content of the public PBwiki. We listened and added this functionality. If you’d like to enable this on your PBwiki, just go to the “Public / Private” settings tab and click Disable indexing. This sets your robots.txt, updates your page headers, and even puts a setting in your RSS & Atom feeds to discourage feed readers & scrapers from republishing your public feed.
4 Responses for "Quickie Tidbit: Disallow Search Indexing"
Just to clarify for my nervous wiki-newbies: a private wiki is NOT indexed?
That’s correct, private wikis are inaccessible to search engines AND are marked to disallow indexing. (But without the password, the search engines could not even access the page, much less index it!)
is there a way to make part of a wiki indexed, but not other bits?
Wow — thanks for listening and putting this in. You folks are great!
Leave a reply