06-03-2008, 11:10 PM
|
#1 |
| Administrator
Join Date: Sep 2006 Location: clw Age: 50
Posts: 55
| Invasion of the Slurps and Spiders
Well it seems we found the problem causing the frustrating slowness on this site. 2000 yahoo slurps searching and indexing 1.5 million posts will indeed bog down the server every time. The yahoo slurps started showing up in numbers about a month ago. Slowly, they overwelmed us with numbers some days of over 2000. We've now blocked most of their IP addresses and done some other clever behind the scenes kind of stuff. We may not get as good of placement on yahoo, but who searches with them anyway. Unfortunately, just when we thought we had it under control, we discovered the fool slurps had also invaded ShopSBT which is on the same server as PWCToday and was again slowing it down. Needless to say, we smashed those pesky slurps too and all seems to be good now.
We still plan to upgrade the server, but will probably do so after the summer season just in case it does not go as planned. As for now, you should notice a nice improvement in response time now that the yahoo slurps have been exterminated.
Thanks,
Greg Pickren |
|  |
Linear Mode
