puts the site map link in the robots.txt file, the advantage is that the webmaster does not need to submit the sitemap file to each search engine’s webmaster tools or similar part of the webmaster. This method can help the webmaster to ease the work.
‘s major search engines have released new ways of supporting web maps, including links to sitemap files directly in robots.txt files. At present, the search engine company that supports this has Google, Yahoo, and Chinese search engine company is apparently not in this circle.
the benefits of doing so is that owners not to each search engine webmaster tools or similar webmaster part, go to submit your sitemap files, search engine spiders will grab the robots.txt file, read the sitemap path, then grab one link ".
Here is a
based on the assumption that robots.txt is already a standard, all sites have at least one, while sitemap is still being established in the standard, this approach can let the webmaster reduce the work, and if they don’t need more detailed indexing and search information, then you only need to include the sitemap to robots.txt it.
let’s start now and modify the robots.txt file.
# robots.txt for xxx.com Version 2009.04.04 # # # User-agent: /function/ Disallow: /FCKeditor/ Disallow: * Disallow: /CACHE/ Disallow: /SCRIPT/ Disallow: /ADMIN/ Disallow: /wap.asp Disallow: /cmd.asp Sitemap: http://s.www.xxx.com/sitemap.xml
Author: love game network http://s.www.iyx.cc, if reproduced, please be sure to retain the author link and this statement! Thank you for your cooperation!