Eve version 1.2 makes it much easier for search engines like Google to "spider" your site. The term "spidering" means a search engine will read every linked page on your site for its index.

While this is happening you will notice more bandwidth used and page views statistics will increase as the spiders do their work.

Most people want their communities searchable by Google, MSN, Yahoo and the like. If so, you need do nothing and the search engines soon will find your site.

Some people do NOT want their community "spidered" for reasons of privacy or bandwidth. If you do NOT want your community spidered, you can take any one of these steps.

1) Make your site completely private, requiring a login. No search spider will enter.
2) Leave you site "public" with no login required for reading but block the spider using a robots.txt

In the /public_html or "web root" of your site, upload a file called robots.txt with the following code in it.

User-agent: *
Disallow: /

This code will tell all responsible search engines to stay away.

3) Insert in the HEAD section of the style used everywhere by your site

<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">

For more information on "spidering" and what it means to your site see the Google FAQ or the Robots FAQ

Just did a few search...look like Google spider Infopop Nation the best..with over 27000 results.

After that is MSN Search...2000+

Yahoo Search is only 360+ (Doesnt seems spider the new URL format yet)

Cool!

If only members have permission to "View Discussions" will the spiders still be able to spider the site? I assume not

You are correct, if your forums require a login in order to see forum content, then spiders will not be able to see and spider the content, since they are recgonized the same as a guest on your site.

Okay, for 90% of our forums, which are readable by anyone, this is great news. VERY Excited, been waiting for this for some time.

For the 10% of our forums that are limited to small groups, and would be pretty devastating to some posters should they be spidered, what should we do? Is it an all or nothing thing?

If you have a forum that only allows a certain group in, spiders can't see those forums. Even if you just have to be logged in to view them, they won't be spidered. Spiders can't create a groupee account, and then login to your site to spider the content.

quote:

Originally posted by Chi Chi Valenti:
For the 10% of our forums that are limited to small groups, and would be pretty devastating to some posters should they be spidered, what should we do? Is it an all or nothing thing?

The robot.txt file can be very detailed as to what you want to allow to be spidered - or not in your case.

How long does it take for them to get spidered. Doesnt seem like the conversations can be searched. The youthhockeyforum.com has been online for over a year and on just about every search engine I can think of. I assume my site would get ALOT more hits from search engines if the topics are getting picked up but it doesnt seem like they are.

Commish,

Google has visited my site twice since it was upgraded to version 1.2 on 2/07/05.

Consequently, there are many more pages for my site listed on Google now.

Overall, I'm VERY happy with the upgrade to version 1.2.

quote:

Commish,

Google has visited my site twice since it was upgraded to version 1.2 on 2/07/05.

Consequently, there are many more pages for my site listed on Google now.

Overall, I'm VERY happy with the upgrade to version 1.2.

Great...
But that really didnt anwser my question. How long does it take for them to spider the sites. 1 week? 2? 5 months? How can you check how many links are listed on any given search engine.
Thanks

Pretty hard to say ... We can't speak for google or the otehr search engines. You could submit your site rather than wait for spiders.

Submitting your site to Google -- http://www.google.com/intl/en/addurl.html

Google has a paged cached for our Web site that's about six months old, before we switched our domain to Infopop. Does that mean they haven't tried spidering our site in months? I used that addurl feature a couple of weeks ago but the cache is still not updated.

They say it can take 4-6 weeks.

quote:

Some people do NOT want their community "spidered" for reasons of privacy or bandwidth. If you do NOT want your community spidered, you can take any one of these steps.

1) Make your site completely private, requiring a login. No search spider will enter.
2) Leave you site "public" with no login required for reading but block the spider using a robots.txt

In the /public_html or "web root" of your site, upload a file called robots.txt with the following code in it.

User-agent: *
Disallow: /

This code will tell all responsible search engines to stay away.

3) Insert in the HEAD section of the style used everywhere by your site

<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">

......


Okay. I put the what where and fiddle with which doodad again? I"m not sure what a web root is. And, as just a moderator, not a site owner, do I even have access to it?

My board members just discovered the google thing last night, and are most distraught about it.

aitapata,

The webroot is essentially the public_html folder on the webserver. If you are a moderator and do not have access to the server itself then you won't be able to add the robots.txt file. The Site Admin will need to look into it.

Hi aitapata,

Another alternative is to set your site to be viewable only by registered users. The search engine spiders like GoogleBot are effectively guests when they come to your site, so if you don't allow guest access, then you won't have to worry about your site being spidered.

MichaelF et al,

The newer versions of EVE have been a blessing when it comes to spidering our building code discussions group (BCDG) site. I track our visitors daily, and since March 1st, the traffic almost tripled. I can also see that 90%+ of the referals are coming from Google.

The new versions 1.2+ appear to have opened a flood gate with search engines. From my prespective....Im loving it

Thank you.

Same here!

my site also see much more traffic since the release of 1.2!!!

Thanks Infopop!