Be careful when you are launching a new site. Remember the search engines will find you if they can find links pointing to the domain. We ran across a humorous example of a site in development while researching the term “runaway teens” for a client.
The site http://www.projectrunaway.net/ came up during our research. While it has nothing to do with our subject matter search the term runaway was the connection. Our researcher, read the first paragraph and got a chuckle. I’m sure they will finish the site soon and this post will make no sense so I have attached a screen shot for future generations.
So if you are building on a live server, be sure to block the search engines until you are ready for them. Here are two methods to employ. Both work well.
A simple addition of a robots.txt file with the following command will do the trick:
User-agent: * Disallow: /
You can also add this meta tag to head of your the page you don’t want indexed:
<META NAME=”ROBOTS” CONTENT=”NOINDEX, NOFOLLOW”>