Have you ever made an update to your website, and had removed old pages where the information is no longer relevant?  Later on, when you look for your website through a search engine, do you see your old pages still listed there? To some, it may seem like the old pages hadn’t been removed properly off the server, but actually, that may not be the case.

Search engines actually keep their own database of website results, so even if you updated your website, the search engine results don’t update immediately, and you will have to wait until search engines crawl your website (collect website content) again before updating their own databases. Some search engines, such as Google, actually have tools that assist webmasters manage the web search results, so that older pages no longer show up in the searches. For more information on Google’s webmaster tools, please take a look at:

http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=164734&from=35301&rd=1

http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449

Another thing you can do is setup a robots.txt file on your server to block search engines from crawling content you no longer want to show up in search engine results.  For more information on how a robots.txt file works, please take a look at this link:

http://lab.redmallorca.com/how-to-remove-a-web-page-from-google-index-and-other-search-engines/

Comments

comments