“If you can’t measure it you can’t manage it” said someone – and everyone repeats ever since. Part of that is a mania for metrics, which affects also many in the software industry. Those affected by it uphold that without measuring KLOCs, defects per KLOC and all other metrics it’s not possible to deliver good software. There are some very scientific discussions going on LinkedIn Answers and elsewhere about which of the cryptic sounding metrics is best and should be meticulously applied, usually using some quite expensive tools.
I would rather say that you can’t manage anything until you first know what you want to get and what is it that you manage. People trying to apply metrics, especially automated metrics, as a tool for managing software development do not realize that programmers are not machines and software is not made, it is created. Hence all the management methods , theories and measurements right for the assembly line just don’t apply.
Managing programmers is quite a different challenge – someone said it’s like herding cats. There is a lot a good manager can do, but metrics of any kind can be only of minor help. The biggest challenge is to get bright people who care for the code – and get them to care for the code they create for you. If they do they’ll apply all the right engineering practices. If they don’t no metrics will help – the result will be crap.
It’s not that metrics are useless. They do show something. Bugs should be tracked and maybe counted. Percentages of all kinds can be counted as well. However, they are just an indicator of something – and not the most important one. And they are just a side tool, not the product. They should never be blown of their right place or context. And never ever should any of these metrics be be linked with a reward/bonus system.
Why? Because good software is a bit like a good meal or good architecture. It’s something that can’t be strictly measured. Good code is, for example, usually a smaller code. So measuring programmers output in lines of code makes no sense whatsoever. A programmer that introduces many bugs might be worth his weight in gold because he might be good in introducing creative solutions to the software. Just metrics won’t tell you that.
“Not everything that counts can be counted, and not everything that can be counted counts.” read on a plaque in Albert Einstein’s office. Metrics-maniacs should keep that in mind when they get too carried away with their bar charts.
Some people go around trying to get “an accurate price to do the project” when they want to develop some software system. Then they collect the bids, choose the best contender (usually taking the lowest bid), order the project and go away for the time fixed in the contract hoping to get a working system in the end.
We don’t work like this. We don’t do fixed bids, and I get frequently asked why. So here is my explanation why we think fixed bids are bad and our clients will be better off without them. It is based on an e-mail to a prospective client, whom I couldn’t call to explain all that over the phone.
First of all, you’ll never get an “accurate price”. Developing software, especially new systems built to individual needs is a creative, inventive process. It is not possible to accurately say how long developing feature A might take. It is possible to make a prediction, an estimate, but it’s just this – an estimate. Especially with the amount of information about each feature that is typically available at the project start.
Every software developer knows they can’t accurately predict how long each feature will take so they take a guess. Nothing wrong with that, the problem starts when this guess is being presented as solid reality. Which is exactly what happens in a typical bidding process. Since each company knows there is a risk in the estimates they compensate for it by overestimating. Industry standard for security margin on estimates is somewhere around 25% which shows how inaccurate they typically are. In any case what you get as a bid is a guess bloated by some margin so that the developer feels safe they won’t loose money if things go wrong.
Now, let’s say someone will tell you completing a job will cost $25000. But we all know it won’t. In the end it might cost $24998 or $25231 but it won’t be exactly $25000. In fact it’s more likely that it will cost either $2000 or $30000. In the first case you are ripped off $5000 in money or features that might have been developed. In the second case the quality will suffer and it’s easy to prove that: if the project is over budget but a contract is on a fixed price every sane company will do all they can to limit their loses. That means finish the software as quickly as possible. Just kick it out the door, the sooner the better. In software that means develop poor, undocumented code, solve problems via quick hacks not elegant solutions, reduce testing to absolute minimum. And lower the cost – replace senior developers with interns, working overtime – anything. If – heavens forbid – not only the price but also the deadline was fixed and penalties apply the software company has an even bigger incentive to lower the quality to limit losses.
All that explains why so many IT projects (be it software development or deployment) end up with failure or at least client dissatisfaction.
But there is something else to fixed bids that is bad for clients. It’s the fact that the bidding process by its very nature requires the scope to be fixed as well. That means clients have to provide all the features they might ever want from the development team upfront, before any development starts. So many people start by writing a list of everything they want from the envisioned software system.
There is nothing bad with writing that list, it is a very helpful effort. What is bad is fixing it, making it carved in contractual
stone. Why? First, because if you write this document thinking that if you don’t put something there you will never get it you end up with putting many things there which are not needed. But also, you won’t put there many things that will be needed only because you haven’t thought of them. And you haven’t thought of them because those ideas will occur to you only after you will start using the software, even in an early version.
In other word writing a fixed scope document and then freezing it by making it a part of a contract means trading many useful
but yet unknown features for many nice-to-haves or things that no one will use, but they were squeezed out of your mind by the fear of omitting something.
To sum it all up: we think long-term planning is a good thing, but we don’t sell the illusion that a plan is anything more than it is. We fix quality and release dates (every iteration, usually 2 weeks), we fix the price of each iteration beforehand. We don’t fix scope and we don’t fix the total price for the project. If you have a great new idea two months into the project – that’s fine, we’ll just implement it. If by fourth month you decide what was developed is enough – that’s fine, you can end the engagement at any time. We give you the flexibility traditional development based around static documents can’t give you. And we are totally honest and open about what we offer: we offer our services, at a set, negotiated price. We’ll build as much as possible at best quality for the amount you will be willing to spend on the project.
And one more thing: with an agile team you’ll never be left with a heap of code that can’t be used for anything except showing it around in hope someone will finish it. Why? Because at the end of each iteration you get a new, complete product release which is 100% tested and 100% functional. It might not contain all of the features (in fact, first release after first iteration won’t probably do much) but those that were finished are indeed done – fully working. It means that from the very beginning you have a working software system, that is upgraded every two weeks with new features. You’re never held hostage by the development team: you own the code, you have something that is useful. The best part is: in fact you decide which features are most important *now* and should be delivered in the next iteration.
Thanks to Paul Klipp whose podcasts on selling agile software development remain a source of inspiration. Many ideas for presenting the merits of the agile approach used in this article are based on those podcasts and discussions during our meetings with Paul.