SlimTimer turbulence
Posted by Richard White Wed, 11 Oct 2006 12:50:00 GMT
I wanted to take a moment apologize to anyone who used SlimTimer in the last 12 hours and encountered slow reponse times and the 500 errors. The good news is that it seems to be fixed now (you never want to anger the gremlins by proclaiming that you’ve conquered them) and the site should actually be faster because of the solution.
I attempted to roll out a silent release so people could beta test the API, but the server was none too pleased and started thrashing about. No biggie I thought, I’ll just do a quick rollback and debug these issues elsewhere. That too did not go so well and left me with a server that would last an average of 5 minutes before I’d have to restart it (the fastcgi dispatchers, not the whole actual box) and no way to fix other than on the live site.
I did my best to try and fix the problem while also keeping the server up at least every other 5 minutes (to pick up SlimTimer submissions). The solution? Swapping out Lighttpd/fcgi for Lighttpd/Mongrel.
Update 10.12.2006:
So it turns out my Mongrel solution turned out to be a false positive as the server went back down a couple hours after this post. I did finally manage to track down the real culprit: bad data in some user sessions.
The positive upshot of all this is I became acutely aware of some performance issues in the system. I’ve sped up most of the views and reports by 150% or more on my local development version. I’ll push those improvements out in the next release, which will be whenever I get up the nerve to touch the server again. :)
Update 10.12.2006
Seems that performance issues were also part of the problem mix as they reared their ugly heads again during peak hour (11-1 MDT). I went ahead and hot fixed some of the SQL queries (referenced previously) which should help. I’m also investigating whether it’s a configuration issue or a lack of server resources. I’ve temporarily changed the client update rate to 10 minutes from 3 (this is how often the client updates the server).
Update 10.17.2006
Pushed out a new version on Sunday(10.15) that optimized some more SQL queries and fixed two memory leaks. Things seem to be running well now as I haven’t seen a server load over 1.0 which is good considering Monday is the busiest day. Thanks for your patience during all of this.
First of all, I can’t live without SlimTimer. It’s just an amazing tool. I’m doing freelance work and it’s all I use to keep track of my hours. I just couln’t say how much I missed it when I figured out slimtimer was getting the 500 error. I had to use notepad and F5 to punch in/ out manually.
Please have the server fixed because I really depend on SlimTimer now. I know SlimTimer is not your fulltime project but if you need donation/ subscription, I would totally go for that. $5/ month is very reasonable. And yeah, the new report looks awesome.
Keep up the good work,
Alex.
Hi Richard, I have been using SlimTimer for the past one month. It’s really great time tracking apps. I was at lost when I could not access it yesterday. However, kudos for resolving it – I’ll definitely stick to SlimTimer!
Slimtimer, I’m glad you’re back and alright.
Cheers, SlimTimer is awesome. However, I still can’t create a new entry manually in Edit Entries.
Hm, nevermind, appears to be working now magically. :)
@Alex Le: Actually, SlimTimer is my full time job and #1 priority (shhh don’t tell Lindsay) so rest assured when it goes down it’s the only thing I’m focused on.
All that said, I’d love to get 5$/mo from everyone :) I’ve been struggling with how to monetize the service from an individual user account perspective (ie I don’t know what new features would be worth paying for). Suggestions are appreciated.
I guess it’s a good thing (for you, Richard) that so many people feel the loss! I suffered through it by taking a day off from client projects and working on my own (coming soon) business card design blog. I’ll keep ‘suffering’ for now, but look forward to getting full service back soon.
Richard - Are you telling me you’re not living and breathing my project! Oh for shame :-) hehe…slimtimmer is looking great. I haven’t had a ton of time to look at it recently but keep up the good work!!!!