Back to the Web Design Home Page Buy Now - available at Amazon.com
Overview Chapters Examples Resources
Chapters
Complete Chapters
Chapter 1

Chapter 5

Chapter 9

Chapter 13

Appendix B



Check Out These Other Books!

Chapter 9: Search
Robot Control with <meta>
An alternative method to the robots.txt file that is useful particularly for those users who have no access to the root directory of their domain is to use a <meta> tag to control indexing. To disallow indexing of a particular page, use this <meta> tag in the <head> section of the HTML document:

<meta name="robots" content="noindex" />
You can also inform a spider not to follow any links coming out of the page:

<meta name="robots" content="noindex, nofollow" />
When using this type of exclusion, just make sure not to confuse the robot with contradictory information like

<meta name="robots" content="index, noindex" />
or

<meta name="robots" content="index, nofollow, follow " />
as the spider may either ignore the information entirely or maybe even index anyway. The other downside to the <meta> tag approach is that fewer search engines support it than do robots.txt.


Next: Optimizing for Search Engines


Overview | Chapters | Examples | Resources | Buy the Book!
Available at Amazon.com Available at Amazon.com