Today I was with a client and initially everything went well until we suddenly got strange errors building a project using Jenkins. Looking at the log file it sure looked like Subversion was corrupt and we started to investigate. Of course, this happened at the last part of the project before going life and so time is precious.
After some back and forth it turned out that we just ran out of disk space because we created a few more jobs on Jenkins depleting all remaining disk space on an 8GB partition. Yes, right, a 8GB partition for /var which is so bad. For sure we could blame the installer of Jenkins just to use the Jenkins RPM and not configure Jenkins to use a more spacious partition but at the end of the day creating a /var partition on a linux server with 8GB is way too low. At the end of the day it took 3 people 3 hours (or more) to fix this mess which means we could have bought a Drobo drive with enough disk space to given every virtual machine at least 1 TB.
At the end of the day I don't understand how companies can underpower and underequip their development servers. Naturally things go wrong when we use the equipment the most and that is right before going live where you can afford a downtime the least. Beside the costs of the fixing it it also puts another stressful event on the development team leading to more errors, frustrations and eventually to further delays.
Powerful and reliable hardware and software isn't a treat but an essential component of a successful project. Having developers that cost between 10 to 20k per month excellent hardware is a drop into the bucket to the overall costs. 2 to 3k for hard- and sofware is less than one week of employment. And believe me a week delay in a project going live is on-time in my book.