Robot.txt - Question

**Nite Producer** · 04-02-2007, 02:44 PM

Hi People,

I'm still reading through the SE sections etc.

I just have a question about the file Robot.txt and why I always find in my awstat an IP address got an error trying to find the Robot.txt file?

Thanks in advance.

**Bethers** · 04-02-2007, 08:44 PM

Because you don't have one and Google always checks for one. Not having it won't hurt you unless there is something you don't want the se's to spider. They look for it for any specific directions you might have for them - especially things like no follows.

**Nite Producer** · 04-03-2007, 06:10 AM

Thank you bethers, I knew it will be you who will answer.

Thank you again.

**ragamuffin** · 04-03-2007, 09:20 PM

Hello. When you go into the page html, and enter the meta tags for the robots, does it matter what position it enters the lines? Just wondering.

And did I read somewhere that the Character lines in the tags should be below the other tags?

I may have this wrong, but mine appear at the top of my meta tags lists. Yes, using BV website bldr.
Thanks
ragamuffin
http://www.learnaboutwebhosting.com

**Bethers** · 04-04-2007, 05:10 AM

I don't follow the question - what character lines?

**JoeHollinger** · 04-18-2007, 06:00 PM

I have a question concerning Robot.txt and meta tags. Where and how do I go to edit page html in the <head></head> section and the <body></body> section of a web page?

Is there a tool that I can use to check my robot.txt and meta tags to be sure I haven't made errors and/or ommissions in typing, format, etc.?

Thanks,
Joe Hollinger
Webland Security

**Karen Mac** · 04-18-2007, 06:12 PM

Joe..

Try sitereportcard.com and have it check your site for meta tags. Go to google webmaster tools and if you dont have an account open one and it will tell you rather or not its reading your robot.txt file

Karen

**JoeHollinger** · 04-21-2007, 05:11 AM

Karen Mac Thank you for your help. I went to Google Webmaster Tools and got a report. Just what I wanted to see. Now I have a sort of road map that shows me where some additions and corrections are needed in my robots META tags. Speaking of META tags, are there tools that will check the META tags on a page and provide a report of what was found? Thanks again! Joe Hollinger Webland Security

Thread: Robot.txt - Question

LinkBack

Thread Tools

Robot.txt - Question

Re: Robot.txt - Question

Re: Robot.txt - Question

Re: Robot.txt - Question

Re: Robot.txt - Question

Re: Robot.txt and meta tags - Question

Re: Robot.txt - Question

Re: Robot.txt - Question

Thread Information

Users Browsing this Thread

Posting Permissions