8.2. Google's Importance to Webmasters

But isn't Google just a search engine web site like any other? Actually, its reach is far greater than that. Google partners with other sites to use the Google index results, including the likes of heavyweight properties AOL and Yahoo!, not to mention the multitude of sites out there making use of the Google API. So when you think about potential visitors from Google search results, you have to think beyond traditional search site borders.

Google's perception of your site has become increasingly more important, which means that you're going to have to be sure that your site abides by the Google rules or risks not being picked up. If you're very concerned about search-engine traffic, you're going to have to make sure that your site is optimized for luring in Google spiders and being indexed effectively. And if you're concerned that Google should not index some parts of your site, you need to understand the ins and outs of configuring your robots.txt file to reflect your preferences.