Results 1 to 22 of 22

Thread: help with robots
      
   

  1. #1
    Join Date
    May 2005
    Location
    The High Hills
    Posts
    257

    Question help with robots

    For quite sometime I have been getting this error message

    [Wed Aug 31 10:43:58 2005] [error] [client 213.180.216.253] File does not exist: /home/onlineg/public_html/robots.txt

    I do have robot text in my source but robots, crawls whatever you call it, are having a hard time finding my page. Does it have something to do with me uploading my files to public_html? please help I am begging. pet121

  2. #2
    Join Date
    May 2005
    Location
    UK
    Posts
    514

    Default

    Search engines will look in your root domain for a special file named "robots.txt" (http://www.mydomain.com/robots.txt). The file tells the robot (spider) which files it may spider (download). This system is called, The Robots Exclusion Standard.
    www.bluevodaexchangelink.com
    BluevodaExchangeLink Help Center
    Bluevodaexchangelink Forum

    The only place for bluevoda site to be linked with!

  3. #3
    Join Date
    May 2005
    Location
    UK
    Posts
    514

    Default

    www.bluevodaexchangelink.com
    BluevodaExchangeLink Help Center
    Bluevodaexchangelink Forum

    The only place for bluevoda site to be linked with!

  4. #4
    Join Date
    May 2005
    Location
    The High Hills
    Posts
    257

    Thumbs up thanks

    I am getting there slowly. This is turnning out to be no hobby thats for sure. My site well its not even a quarter finished but put it on the net anyway. I keep on getting sidetracked on learnning how to promote your site. I appreciate your response and will look into it. Some robots grab some don't. I am doing very well on google and yahoo. 35000 hits last month not to shabby for a quarter of a site. Thanks again pet121

  5. #5
    Join Date
    Aug 2005
    Posts
    12

    Default

    what is robot text and how do you put it on your website? i want to optimise my searches on search engines for my site www.davidjoffediamonds.com and i dont know why everyone keeps mentioning this robot text!! PLEASE HELP

  6. #6
    Join Date
    May 2005
    Location
    UK
    Posts
    514

    Default

    Quote Originally Posted by djdiamonds
    what is robot text and how do you put it on your website? i want to optimise my searches on search engines for my site www.davidjoffediamonds.com and i dont know why everyone keeps mentioning this robot text!! PLEASE HELP
    david

    go here

    http://www.webtoolcentral.com/webmaster/tools/

    this site will help you make your "robot.txt" and explain all about it, also there some other great tools for helping to get your site listed/ranked

    any problem just ask and I will try and help
    www.bluevodaexchangelink.com
    BluevodaExchangeLink Help Center
    Bluevodaexchangelink Forum

    The only place for bluevoda site to be linked with!

  7. #7
    Join Date
    Jul 2005
    Posts
    500

    Default

    Quote Originally Posted by Maddog
    david

    go here

    http://www.webtoolcentral.com/webmaster/tools/

    this site will help you make your "robot.txt" and explain all about it, also there some other great tools for helping to get your site listed/ranked

    any problem just ask and I will try and help
    Thanks for the website.
    ::Steven.H::

  8. #8
    Join Date
    Aug 2005
    Posts
    12

    Default

    thanks very much for you help - i didnt quite understand but i think i did it right :~

  9. #9
    Join Date
    May 2005
    Location
    UK
    Posts
    514

    Default

    Quote Originally Posted by djdiamonds
    thanks very much for you help - i didnt quite understand but i think i did it right :~
    Not much to do other than copy to note pad save as robot.txt and the upload to you web site using FTP,

    Also have a look at your meta tags, get this forum on changes to make.

    The other part to help search bots is to have a sitemap, go to

    http://www.blocklayer.com/googlesitemap/Default.aspx

    will help you make what you need
    www.bluevodaexchangelink.com
    BluevodaExchangeLink Help Center
    Bluevodaexchangelink Forum

    The only place for bluevoda site to be linked with!

  10. #10
    Join Date
    Aug 2005
    Posts
    12

    Default

    thanks so much - you make it all seem so easy!! im almost certain i created a sitemap and i managed to create meta tags!! (hooray)

  11. #11
    Join Date
    Aug 2005
    Location
    UK
    Posts
    68

    Default

    Hi Maddog,

    Just used the free tool for robots file, this is what came back, it doesn't seem a lot, I expected a lot of code,

    Paul

    Here is what your robots.txt file should look like;

    # Robots.txt file created by http://www.webtoolcentral.com
    # For domain: http://www.superfastincome.com

    # All robots will spider the domain
    User-agent: *
    Disallow:

  12. #12
    Join Date
    May 2005
    Location
    UK
    Posts
    514

    Default

    Know what you mean Paul not alot to it, looks ok to me.
    www.bluevodaexchangelink.com
    BluevodaExchangeLink Help Center
    Bluevodaexchangelink Forum

    The only place for bluevoda site to be linked with!

  13. #13
    Join Date
    May 2005
    Location
    The High Hills
    Posts
    257

    Post maddog

    wouldn't the below robot text disallow robot to enter?



    # Robots.txt file created by http://www.webtoolcentral.com
    # For domain: http://www.superfastincome.com

    # All robots will spider the domain
    User-agent: *
    Disallow

    I don't want to get it wrong, big penalties. My ranking are going well without it. I am pr3, but futher optimization is needed. thanks pet121

  14. #14
    Join Date
    May 2005
    Location
    UK
    Posts
    514

    Default

    the * is not a wild card to disallow

    User-agent: *

    here you would put the bots disallow, my understanding is blank means allow all
    www.bluevodaexchangelink.com
    BluevodaExchangeLink Help Center
    Bluevodaexchangelink Forum

    The only place for bluevoda site to be linked with!

  15. #15
    Join Date
    May 2005
    Location
    The High Hills
    Posts
    257

    Thumbs up thanks maddog

    Okay I understand but I am still getting these error messages. Out of 72 robots that visited my site only 17% actually found it. Its just weird. Thanks anyway pet121

  16. #16
    Join Date
    May 2005
    Location
    UK
    Posts
    514

    Default

    who do you know 72 visits and only 17% found it?
    www.bluevodaexchangelink.com
    BluevodaExchangeLink Help Center
    Bluevodaexchangelink Forum

    The only place for bluevoda site to be linked with!

  17. #17
    Join Date
    May 2005
    Location
    UK
    Posts
    514

    Default

    Quote Originally Posted by pet121
    Okay I understand but I am still getting these error messages. Out of 72 robots that visited my site only 17% actually found it. Its just weird. Thanks anyway pet121
    just been looking at your site pet,and code, you need to name each page and page the meta tag in each page, remeber can take upto 60 day for search companys to update there sites
    www.bluevodaexchangelink.com
    BluevodaExchangeLink Help Center
    Bluevodaexchangelink Forum

    The only place for bluevoda site to be linked with!

  18. #18
    Join Date
    May 2005
    Location
    The High Hills
    Posts
    257

    Thumbs up thanks

    I will double check the source again and make sure each page is named. thanks pet121

  19. #19
    Join Date
    Sep 2005
    Posts
    20

    Default

    Maddog,

    I have visited webtoolscentral and tried their metatag analyser thingy; and, horror of horrors, it tells me 'error, your site cannot be found.' I am completely perplexed by the whole search engine, meta-tag, description thing... I can access my site by typing the name followed by .com, but on google, aol and msn (all I have available), if I simply type the name of my site, it is also not found. In desperation I have paid 3.49 to eBiz1000 to submit my site to some search engines for a month - was this a sensible thing to do? I found a link to them on webtoolscentral and it didn't seem like a lot to lose but now I'm beginning to wonder. Earlier in the day I subscribed to a free service which offers subscription to 40 engines for nothing and almost immediately I had a couple of spiders on my site, whereas there has been no sign of a spider in the three hours since I used the paid service. Crikey I'm confused. Why should people have to pay to get their site listed on search engines? It seems implicitly wrong to me. (On the Vodahost tutorial there is a heading related to 'submit to search engines,' but that cannot be found in the control panel as shown in the video tutorial - I wonder why... Anyway, anything related to search engines / getting listed would be greatly appreciated, as I'm becoming fed up of constant spamming to attract visitors to my site. Extremely jealous of the person whose site is only one quarter finished and attracting thousands of hits - luck?

  20. #20
    Join Date
    May 2005
    Location
    The High Hills
    Posts
    257

    Thumbs up Maddog

    Its funny you asking for help? Currently I am listed on all the major search engines. On yahoo page3, msn page1, alta vista page1 plus I rank high all over the world. How did I do it no idea? I noticed the your sites are in flash I heard could be a problem for robots, look into it. I will probably get into trouble for this but you need a good seo program, voda hits is one of them, haven't had a chance to try it out yet. But in the mean time down load the free trial version of web ceo but be sure you know what you are doing. Web ceo will automatically submit your site once. But on a ongoing basis you can see were you rank on the internet. If fact there are alot of freetrial seo programs out there. Market your site its not easy but keep plugging away at it. For instance web ceo reported that i had 298 links on yahoo. Another way is creating back links. Some forums are crawled by robots and if you leave your website address on a thread it will pick it up and post on say google. There is so much more information. The bottom line it takes some hard work. pet121



    p.s do not pay anything to get good rankings or listed!

  21. #21
    Join Date
    May 2005
    Location
    UK
    Posts
    514

    Default

    Steve W whats your web address (url to your site)

    As pet121 said you do have to keep pluging at it, are you new to BV? if you are VH will submit free your site, just open a support ticket, you do have to rememebr it can take wait for search companys a while to get around to your site and index it. as for paying for it, well some people are good some are not.
    www.bluevodaexchangelink.com
    BluevodaExchangeLink Help Center
    Bluevodaexchangelink Forum

    The only place for bluevoda site to be linked with!

  22. #22
    Join Date
    May 2005
    Location
    The High Hills
    Posts
    257

    Smile Maddog

    Bv as far as I know only submits to a limited amount of sites. As for paying to have your site listed and get good links and ranking forget it. The search engines who require cash to submit are not really worth it. Eventually the paid search engines who charge will include your site anyway. This will be do to the preasure of being every where else on the net. I have proof of this happenning. For example, ask jeeves search engine is a paid for inclusion, but I am listed. The funny thing is I just can't figure out how I did it? I am still learnning from other sources however. In less than three months I am listed on yahoo page 3 and have 198 links on yahoo, how, I don't know. thanks pet121.


    P.S maddog my prevous post on this thread was meant for Steve. My wife insisted last night we have a few drinks.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

     

Similar Threads

  1. Link Building - The Waiting Game
    By VodaBot in forum eBusiness and Concept Development Talk
    Replies: 3
    Last Post: 06-30-2006, 09:06 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •