|
|
| ID: | 2783 | |
| Title: | Robots txt - http://www.robotstxt.org/ | |
| Pagerank: | 8 | |
| Description: | Information on the robots.txt Robots Exclusion Standard and other articles about writing well-behaved Web robots. | |
| Category: | Internet: Searching the Web: Robots | |
| Link Owner: | ||
| Date Added: | September 14, 2006 09:39:58 AM | |
| Number Hits: | 0 | |
| Archives: | Use the WayBack Machine to see how Robots txt looked like, and what it contained in the past. | |
| Submit your sites to those directories: Webmaster Directory | Irka Web Directory | Isamu Directory | The Squat Free phpLD Templates: PHPLD your Site |