This morning I got a worrying e-mail from my hosting provider: "This message is to advise you of a temporary block placed on your account. This account was found to be consuming an inordinate amount of processor time, to the point of degrading overall system performance."
Looking into the matter I found the culprit being the Microsoft Bing search engine crawler... That was indeed firing away thousands of website (and connected database) calls a minute... OK, Microsoft software is generally seriously flawed, however that MS is spawning "Denial of Service attacks" is new to me.
Proper dealing with web crawlers involves a universally recognized and respected robots.txt file. So I added a 'crawl-delay' for the 'Bingbot'. Within minutes after the change, my website was blocked again with a prompt informative and helpful support team e-mail saying: "The site is being swarmed by Bing crawlers. Bing crawlers do not respond to a robots.txt file". Another serious misconduct by Microsoft.
The hosting provider support team directs me to "Bing Webmaster Tools". Not to be confused with "Google Webmaster Tools". Google Webmaster Tools is THE BEST (free) webmaster support tool EVER. Probably something that Microsoft now tries to copy. But in the very best of Microsoft traditions they just created a big mess of theirs.
Instead of allowing Bing to find my site, I just now made it extremely hard for Microsoft to index my site in Bing, at least to a point that it NEVER EVER AGAIN comes anything close to launching more 'MS DoS attacks'. What use it is to have ones site on the #1 spot on a search results page, only to be arriving at a blocked site? Microsoft now surely made it to Junk status (with a negative outlook), at least with me. More than April fools!
Anyway, part of this beautiful Easter monday was spent with a short but rewarding family outing, kinda reunion. What happened to my kayaking? Is Spring finally out there? Paddling in due course!
Looking into the matter I found the culprit being the Microsoft Bing search engine crawler... That was indeed firing away thousands of website (and connected database) calls a minute... OK, Microsoft software is generally seriously flawed, however that MS is spawning "Denial of Service attacks" is new to me.
Proper dealing with web crawlers involves a universally recognized and respected robots.txt file. So I added a 'crawl-delay' for the 'Bingbot'. Within minutes after the change, my website was blocked again with a prompt informative and helpful support team e-mail saying: "The site is being swarmed by Bing crawlers. Bing crawlers do not respond to a robots.txt file". Another serious misconduct by Microsoft.
The hosting provider support team directs me to "Bing Webmaster Tools". Not to be confused with "Google Webmaster Tools". Google Webmaster Tools is THE BEST (free) webmaster support tool EVER. Probably something that Microsoft now tries to copy. But in the very best of Microsoft traditions they just created a big mess of theirs.
Instead of allowing Bing to find my site, I just now made it extremely hard for Microsoft to index my site in Bing, at least to a point that it NEVER EVER AGAIN comes anything close to launching more 'MS DoS attacks'. What use it is to have ones site on the #1 spot on a search results page, only to be arriving at a blocked site? Microsoft now surely made it to Junk status (with a negative outlook), at least with me. More than April fools!
Anyway, part of this beautiful Easter monday was spent with a short but rewarding family outing, kinda reunion. What happened to my kayaking? Is Spring finally out there? Paddling in due course!
No comments:
Post a Comment