I may not win many friends by commenting on the emperor's nudity, but let's charge ahead anyway. I am not a clever man.
Possibly you've not heard of Douglas Hubbard. If you're interested at all in estimating the value of IT projects, his works should be required reading. In fact, his books are required reading for the Society of Actuaries if you want to pass their exams and become an actuary. Douglas Hubbard is a clever man.
But we'll come back to him in a moment.
If there's one key thing I've learned in consulting, it's that potential clients who start discussions by talking about building value for their customers have usually been better clients than those who start conversations with "what's your rate?" Why? Because the "what's your rate" clients are focused on controlling costs, not building value. I've found that their approach problems is different and those who are paying more attention to minimizing risk than maximizing value simply have different ideas about how to approach a project.
And you know what? I get it. If you want to go out to dinner and you only have £15 in your pocket, you're probably not going to a fancy restaurant. You just might grab a pint at a pub and hit a kebab shop after. Budget constraints are real.
But curiously, while development costs on a project are one of the first things we look at, they're also one of the worst metrics you can use to judge the value of a project. It's part of a problem that Hubbard refers to as the IT Measurement Inversion. Specifically:
The variables having the greatest impact on project success are the variables we measure the least.
So what sort of variables are there? There are the ongoing development and maintenance costs when the project is delivered. There are marketing costs. There are costs to have your people learn the new system. And so on. The costs are myriad and complex, but if we're going to judge the value of a new system, which variables matter the most? Those are the ones we should be focusing on.
Hubbard's work will guide you here. In particular, you might want to consider his book How to Measure Anything (that's not an affiliate link, by the way), where he teaches you how to break problems down to things that can be measured and how to measure them. You'll learn quite a bit about statistical analysis and running Monte Carlo simulations. In fact, one of the things you'll learn is that of the various techniques which have been developed to estimate project success, Monte Carlo simulations win hands down.
In fact, Monte Carlo simulations are so powerful that Hubbard talks about research at NASA showing that, for over 100 projects, the accountants outperformed the subject matter experts (scientists and engineers) on project estimates because the accountants used Monte Carlo simulations rather than other methodologies.
But for the purposes of our article, the key takeaway is that Hubbard has done a lot of this work for you and has nailed down what your real risks are. And it's almost intuitive once you think of it. Let's start with the most important.
The greatest risk to project success is whether or not it will be canceled.
That seems so brain-dead obvious that it's almost insulting to say, but when was the last time you quantified that risk? When was the last time you looked at the potential value of a large project and said "there's a 25% that this project will be canceled" and then factored that into your risk estimates? It's the single most important variable, but most people ignore it! Can you imagine running a company and discovering your managers are repeatedly ignoring the most valuable information at their disposal?
The next most important variable is system utilization. This is actually complex and includes how fast your product will roll out, whether people will use it, their rate of adoption, and so on. Again, this seems so obvious given 20/20 hindsight. Sure, your project took six months to develop instead of three, but customers will be using it for many years. When it's put that way, it's obvious that estimating system utilization is more valuable than estimating development costs. But people still ignore it.
So let's look at development costs.
Of course you have to factor in development costs; I'd be insane to suggest otherwise. You have a budget and when you're looking at backing your project with Oracle or PostgreSQL, most of the time you find that Oracle isn't a terribly cost-effective solution. So yes, costs are important, but consider this: if the first version of Amazon's code cost twice as much to develop, Jeff Bezos would probably still be insanely rich. Uber would still be dominating their space. Facebook would still be selling your pesonal data to dodgy companies. I can confidently say that because these companies are providing real value to people (you might disagree about Facebook, but I can offer you 2.32 billion rebuttals). The fact that these projects weren't canceled and are being used far, far outweighs the initial project costs. Or to restate all of this:
Initial development costs are incurred once. Your revenue stream is continuous. That's why development costs are less important.
"But Curtis, I don't know how to do Monte Carlo simulations or estimate projected system utilization!"
Well ... that's actually a fair point. These articles are intended, if nothing else, to be practical.
Since the two most importance variables in project success are the odds of it being canceled and system utilization, we need to reduce the risk of each of those. How do we do that?
Simply put, you go lean/agile. Figure out the smallest possible work you can do which would add value and build that. Instead of a year-long project, you have a crazy one month race to get a simple prototype in front of your customers. If it turns out your customers have zero interest in your "Recycled Food" project, it's better to find out after a month instead of a year.
But let's say there's some interest. What now? Well, you've greatly reduced the chance of project cancelation because it looks viable, so you need to focus on system utilization. Amongst the key factors, getting customers to actually use the damned thing is incredibly important. You will be amazed at the ideas they come up with and the feedback they give you. As you push forward, you address customer concerns and have a lower risk of building features on top of things they say aren't working. You're constantly pushing new versions of the product in front of your customers and iterating based on their feedback. Every step of the way, you're improving your project based on customer behavior, not your hunches or market research.
This, incidentally, is why Lean Startups are such a powerful movement in business today. They assume they're not canceling their entire business, so they need to maximize system utilization. That means rolling the product out quickly and immediately responding to customer feedback. By doing this, assuming your product isn't canceled, you've now reduced the single greatest risk your project faces.
If you have any thoughts about this, please leave a comment below.