On Wed, Jun 25, 2008 at 09:59:47PM +0100, David Cantrell wrote:
Christopher Schmidt wrote:
On Wed, Jun 25, 2008 at 04:06:28PM +0100, David Cantrell wrote:
Yes, what you're missing is that google don't pay attention to robots.txt or the meta thingy. I expect that they cache it and then ignore changes for some time. Yahoo do the same.
Er, you seem to be misunderstanding how meta tags work: they have to *Crawl* the page to see the tags... and there is no tag that says "never crawl this page again."
You seem to be misunderstanding the concept of a cache. If they read the meta tag once, they should remember what it said for a while, AND OBEY IT without asking for that page again. Likewise robots.txt.
Sure. And they do, in my experience. The definition of 'a while' may not be what you want it to be: In general, this can be helped by accurately describing how long 'a while' should be.
the key thing to point to would be aninstance of Google search results containing a piece of HTML that is blocked by noindex. If you can find one of those, I bet that Google would be interested in seeing it. (Cheap tricks like modifying the HTML after Google crawls by don't count.)
I have no interest in helping google.
Then allow me to put it differently: if you were to present that, instead of half-baked vitriol filled with hateful comments, I would be much less likely to mentally dismiss you as nothing more than a troll. It seems clear that this doesn't bother you one way or another, so I'm sorry that you have found your experience with running a webserver so painful, and hope that one day, you find a teddy bear that you can hug and make yourself feel better.
Regards,