Author Topic: Wow, most online users at one time ever today  (Read 14623 times)

0 Members and 1 Guest are viewing this topic.

Offline classmate

  • Sens Fan Hall of Fame
  • ******
  • Posts: 6525
  • Gender: Female
    • View Profile
Re: Wow, most online users at one time ever today
« Reply #20 on: March 29, 2008, 11:35:56 PM »
^^^ Will ya look at that.  I learn something new everyday.  Thanks BD.  Technology sure is interesting.   :thumbsup: 
Amazing all the things people my age have seen come and go in our life time.... so far.    :popcorn:

Offline pulp

  • FRIGGING GENIUS
  • Sens Fan Hall of Fame
  • ******
  • Posts: 3345
  • Gender: Male
    • View Profile
Re: Wow, most online users at one time ever today
« Reply #21 on: March 30, 2008, 12:09:24 AM »
I had a website for a year and within 2 minutes of putting it up (hosted it on my PC itself) Google spiders were my first hits. LOL!

Offline work2play

  • Sens Fan Hall of Fame
  • ******
  • Posts: 2194
  • Gender: Male
    • View Profile
    • Petition Spot - CBC is for all Canadians
Re: Wow, most online users at one time ever today
« Reply #22 on: April 01, 2008, 02:26:05 PM »
These spiders/bots (whatever you want to call them) will always walk the "Print" links because really they want the stripped down version of the site.  Most are from search engines, but there are some that are out there crawling the web to steal content and crap too.

I'm sure BD is probably aware of this, but for those that this is a new thing to, you can control what part of your site these bots can crawl through by placing a file called robots.txt at the root of your web site.

I have robots.txt files for all my sites, since there are some areas that I don't want the bots to crawl. 

For example placing the text below in a robots.txt file would keep all bots out of your entire site (which would certainly hurt your search rankings).

User-agent: *
Disallow: /

If anyone is interested in more specific examples I could dig some more detailed samples out, but they are pretty easy to construct yourself (in fact Google has a help page that gives pretty good instructions and syntax examples).
Lost your job yet?  Keep buying foreign!