As I posted before in ‘Mysterious problems with my VPS’, I recently got an increasingly unstable VPS system hosting a lot of my and my customers sites. After a lot of digging I initially presumed that Dovecot (the mail server) was responsible for the issues, as you can read in ‘Dovecot causing memory issues’.
Last week I did a lot of debugging on the Debian server to try and find out what was the issue. And initially it was Dovecots memory usage. After disabling this tool for a couple of days the server was still running fine. However the day after I posted the article on Dovecot the server crashed again. So I had to restart my investigation.
First off I had to had to get a better memory management tool, so I installed Htop on the server (apt-get htop). This shows the current memory usage of each running application. After installing this I enabled all services and applications again and started running stress tests. And though Dovecot was causing some peeks in memory usage it did not keep the high memory usage after the requests where done.
As it turns out for some reason Apache 2.2 was using a lot of memory during peak loads. But even more frustrating it didn’t seem to release any memory any more. Which was causing issues for services that only spawn when they are being accessed like Dovecot and Postfix, which explained why both of these services crashed when the server halted.
After tweaking the maximum amount of servers Apache is allowed to start and the maximum amount of client threads to handle the memory usage dropped dramatically. And I am very happy to report that the server has been running again for more then a week, without any glitches.
Still it doesn’t explain why all of this only happened after updating my server with the latest versions and patches. But I’m glad it’s solved for now.
In my previous post I mentioned that I had and still am having some serious issues with the stability of the VPS I’m running all of my websites on. Though I’m still a long way away from solving the issue, I have figured out that it is very likely caused by an upgrade of Dovecot.
Why I believe that dovecot is slowely over time eating up memory, well after I disabled it the VPS continued running without any issues. I already knew Apache 2, subversion and MySQL weren’t causing it. So I only had postfix and dovecot left to test.
What is truly amazing is that the website for Dovecot indicates it is low in memory consumption, hence it has no settings to limit the amount of memory allocated for Dovecot. So I still need to figure that part out, or alternatively change to a different IMAP server.
I’ve heard a lot of people always say that you can’t crash/hang or break linux easily. Well then I guess I’m one of the lucky bastards that gets it done. Since a few days the VPS system I’m operating keeps hanging itself up.
Now off course it is not the fault of Linux
, but with my very lacking skill. I recently did an apt-get upgrade command on the VPS. To my suprise it started updating with Lenny packages, whilst I could have sworn I have Etch installed. Mystery on its own, but worse yet the server now keeps hanging and breaking.
I figured it was a memory issue at first cause some weird service was installed that was eating it alive. So I de-installed that and it ran fine for a few days. But allas, after 3 days of running the VPS hangs itself up again. When I say hangs-up I mean that I cannot connect using SSH, the mailserver goes down. MySQL and Apache keep running though, as does postfix. So I’m still guessing it is memory, but I have no idea why my configuration is no longer working after running an apt-get upgrade command. Something is really wrong
.