 Re: AWstats | |
Do you have a 'robots.txt' file? I believe this is basically used to tell bots which pages NOT to read/cache. The bot looks for it and if it doesn't find robots.txt it knows it can read any/all the pages.
BTW: I am far from a pro! Perhaps if I am incorrect, someone will enlighten both of us. |