Results 1 to 3 of 3

Thread: about robots.txt
      
   

  1. #1
    José is offline Private First Class
    Join Date
    Mar 2006
    Location
    Portugal
    Posts
    6

    Question about robots.txt

    Hi!
    I'm José and my site is www.vaccontrol.com
    Looking at statistics on cPanel, and I saw frequently that "robots" made some visits...and "request" robots.txt. I don't have any file called robots.txt, and an error page 404 is the result.
    1- Is this a problem? or it's just the normal?
    2 - can (must) I do something t improve my site? . (anyway it is "listed" in importants searching sites)
    3 - What changes i need to do (tag's... metatags) when i change my pages, for exemple adding pages to my site?

    Thank you,
    José

  2. #2
    racefan20's Avatar
    racefan20 is offline Major General
    Join Date
    Jul 2005
    Location
    Concord, NC
    Posts
    2,339

    Default Re: about robots.txt

    The robots look for robots.txt to find if you've set any "rules" for what they can and cannot access on your server. In most instances, you don't really need to worry about any of this, but you can read more here:

    http://www.robotstxt.org/wc/norobots.html

  3. #3
    José is offline Private First Class
    Join Date
    Mar 2006
    Location
    Portugal
    Posts
    6

    Smile Re: about robots.txt

    Thank you for help.
    José

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

     

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49