It was very naughty today. I kept getting 500 internal server messages too. Sometimes it wouldn’t load, then it would. This evening it seems better, although quite slow.
Still a bit slow for me this morning but SO much better than last night. Thanks for contacting Ben, it was over my head.
Let’s not immediately jump to doomsaying whenever there’s an issue. There are still people taking care of this site who love it, and tech problems happen to all websites.
Yeah the 500’s came back for me. When I first got up it seemed like the site was slow but not erroring out anymore. Probably Ben hasn’t had time to take a look yet. If I knew where the error logs were stored on the server I might be able to figure out what’s causing this but there are so many damn files and I can’t find the error logs.
Well from the logs I can see that we have high traffic volume because we’re currently being crawled by Baidu Spider and Bingbot. I don’t know if that’s unusual or the cause of this though.
You might want to try blocking them in http://www.fluther.com/robots.txt and see if that helps at all. Note that might take about 24 hours for the change to get picked up.
Or just block them in .htaccess since that would kick in immediately.
There’s another, more current version of this question asked by @janbb, too.
FYI
My desktop access seems to be marginally better the past ½ hour or so, BTW, unless it’s just the consequence of traffic here, generally.
I’ve disabled indexing of Fluther by all search engines/crawlers/robots for now. Like John says it’ll be some 24 hours before we see the effects. I hope it helps, but I don’t know if this is the root cause of our problems.
I am fairly convinced the problem is related to server load. A CPU constraint and not RAM. If it was RAM apache generally just gives up. But since it intermittently fails my hunch is CPU. So sudo htop (sudo apt-get install htop) should provide some insight.
I think this is a function of load. As folks are giving up checking the site it loads better. So things aren’t fixed. People are just using it less so you get the illusion of things being better.
Thank you all that answered and commented on this thread.I’ve been on the road and away from my computer since Tuesday the 14th so have not “thanked” all.
Another update: Things are still far from perfect. We may actually be completely down again soon, the techies are tweaking some things. However, some good news for a change: Ben is looking into some real changes, so when things get better, they should get much better.