What's up?

[leafspring]

I gotta disagree.  All it takes is opening a page up to the Internet once and Google can nab it.

And how is that supposed to work? Where should the crawler get the URL from?
And how comes there are websites that aren't listed in the google index?

[Mohawk20]

Just wondering, people who visit a site while having the google toolbar with pagerank and feedback and everything... will they be used as source for adding new urls?

[leafspring]

Well, it is suspected by some people but Matt Cutts, a google technician, denies it. (as do other google employees)

@calico: An addition to my answer to your post: It is possible that a simple visit brings an URL into the google index but only under certain conditions. For example as a referrer-link in an open webstat of a server. But an URL is not per se added to index if it just was visited by someone.

[Mandrake]

Just wondering, people who visit a site while having the google toolbar with pagerank and feedback and everything... will they be used as source for adding new urls?

I doubt they need that Mohawk, go out to the homepage and check the number of guests, 10 to 1 that most of them are bots, sifting through the server. There are codes to turn them off, and higher security for private static sites, but as Ashundar is down right now, nothings foolproof. Burns me up when I see an obvious hacker, on the board with me...

[rcallicotte]

I'm not sure how it works, since I'm not inside Google.  But, I know it has worked this way on a page that was made public that shouldn't have and the incident was only one time.  The site kept the information, even after we locked the same page down.

[dwilson]

It definitely can happen.  I had a site that only had one page with "this site is under construction" text on it and google picked it up within a few days.  There weren't any external links (or any links for that matter) on the site and nobody knew about the site.