Announcement

Collapse
No announcement yet.

/robots.txt ?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • /robots.txt ?

    Hi Everybody,

    In my awstats there seems to be quite a large number of 404 errors indicating " /robots.txt - not found ".

    What does this mean and how can I fix it?

    (I never created a robots.txt file and it's not on my cPanel either).

    Thanks,

    Gabor
    www.gabor-music.com

  • #2
    Re: /robots.txt ?

    You will find the answer in this post: http://www.vodahost.com/vodatalk/cpa...tml#post213987

    Good Luck
    Last edited by karenwms63; 08-05-2008, 03:07 AM. Reason: spelling
    Karen Williams
    Your Belief is Your Reality

    Comment


    • #3
      Re: /robots.txt ?

      P.S. For most sites, the robot.txt is just a blank page that is uploaded. If you want your site to be spidered and indexed, then just use a blank one.

      A robot.txt file is just to let the spider bots know what your preferences are. And doesn't mean they will do what you want them to do...just lets them know what you prefer.
      Karen Williams
      Your Belief is Your Reality

      Comment


      • #4
        Re: /robots.txt ?

        Thank you Karen,

        But, I believe, this is a case of "*****s and oranges"...

        The point of my question goes somewhere else; in my case, it's apparently the SEs robots encountering some sort of "missing files" (or Text?) and indicate it as an error.

        I want to know why that is happening.

        Right now I'm not interested in "no follow"s. I'm OK with the robots crawlings, etc, as is.

        I'm interested in why the robots (/robot.txt) are hitting misses.

        Thanks,

        Gabor
        www.gabor-music.com

        Comment


        • #5
          Re: /robots.txt ?

          I believe you missed the point of my second post...just open notepad, leave it blank, name it robot and save...then upload...then your robot.txt file won't be missing anymore...
          Karen Williams
          Your Belief is Your Reality

          Comment


          • #6
            Re: /robots.txt ?

            Right on.
            I can see clearly now...
            Thanks.
            www.gabor-music.com

            Comment


            • #7
              Re: /robots.txt ?

              Your Welcome!
              Karen Williams
              Your Belief is Your Reality

              Comment


              • #8
                Re: /robots.txt ?

                Oh..
                I believe I was looking at the darn thing inside-out...
                lol
                Gabor
                www.gabor-music.com

                Comment


                • #9
                  Re: /robots.txt ?

                  LOL...it's one of those simple things, that we need to know, but takes longer to understand than to actually do...
                  Karen Williams
                  Your Belief is Your Reality

                  Comment

                  Working...
                  X