Why do Programmes Fail?

What Constitutes Failure?

Of course there are many definitions of failure, failure to deliver all of the intended benefits, failure to win public approval, but what I am talking about here is what we read so often in the press. Programmes overrunning in time and cost, IT programmes being ineffectual, big programmes generally coming off of the rails.

Is it always big programmes or are these the ones that make the news? Our perception of likelihood is distorted through what the media decides makes news in all manner of things.

In this post I am going to concentrate on big programmes that go off of the rails.

FAILED!

FAILED!

Big Programme Experience is Limited

One of the issues is experience, big programmes take several years to deliver, those managing them may not stay at the helm for the whole duration and then they also move off into more senior positions running businesses instead of programmes and interestingly there is quite a difference. The end result of this is that there are not too many people with the experience of what can go wrong and what approaches and tools can be used to fix things and help keep programmes on track.

If a thing is worth doing its worth doing well

A lot of businesses put significant effort and resource into winning new work, see Writing a Winning Proposal and then when the work is done the bid budget is closed off, the delivery team is switched in and all of the people that wrote the proposal, negotiated the contract go back to their day jobs, or other jobs. The programme can’t keep them because they have other jobs and their costs are too high since the delivery was priced with lower grade people. The problem is the lower grade people are not there yet, they are still being recruited, inducted or extracted from other roles. The Programme Manager is therefore looking at work that needs to start with less than the full complement of people. The fundamental planning error is assuming that the required resource pool will be there at the outset. So the initial progress reports show that the programme is behind schedule but for the tangible reason that people are not all on board yet, of course the programme spend is also under budget and so there is an obvious cause and effect, the stakeholders, managers all breathe a sigh of understanding and the PM looks forward to getting his resource, in fact he puts all of his energy into it. The thing is even if all of the planned resource arrived the following day is the programme ever going to recover?

Fail Early

There is an expression in vogue at the moment “fail early” which is effectively saying if you are going to potentially fail, do it early, minimise the cost, fix the problem, reset the playing field and start again. The thing is most programmes that fail do in fact fail early but nobody either notices it or believes that such an early failure is possible or that it will not right itself given more time.

A Programme is a Journey

Let us imagine that we are preparing to go on a journey from Park Lane in London to Edinburgh Castle, we have to reach Edinburgh at 14.00 hours on a Saturday because we are going to a wedding, being later than 14.00 hours constitutes a failure and a wasted journey. We need to plan when we are going to leave, we are driving because we have 2 other people in the car and luggage with wedding clothes to change into and we have determined driving is cheaper than the train. Oh yes one other thing we have, is the ring, one of the occupants is the Best Man.

Get Me To The Church On Time!

Get Me To The Church On Time!

Get me to the Church on Time!

So our rudimentary plan consists of three distinct activities, pack, drive, change. We consult a online route planner, years ago we would have perhaps marked a route out on a map of Great Britain and broken down the journey into stages, we would also have marked Key Milestones, we would then have worked out the distance between the milestones and using an average speed, from experience, we would have made a table of the stages of the journey and added up the times. Let us assume that our online route planner makes all of this unnecessary, it tells us it is 403 miles if we use the M6 or 385 miles if we use the M1. The M6 route will take 6hrs 30mins, the M1 will take 6hrs 50 mins. So already we have a decision to make. We opt for the slightly longer but faster route using the M6. I should add that neither the driver or any of the passengers has driven this journey before.

We allow 30 mins to pack and a further 30 mins to change into wedding clothes when we get there, so that simply adds an hour. The total duration is therefore 7hrs 30 mins. Working back from 14.00 this means we need to start loading the car at 06.30 on Saturday morning.

One of our passengers has done a bit of project management, they say we will need to allow time to get a coffee and use the bathroom on the  way so that’s another 30 mins to add, so we had better leave at 06.00 for the plan to work.  Another passenger, who knows about risk management always looks on the black side of things and asks what happens if something unexpected happens like a breakdown or a flat tyre. The others have no idea how long this would take so they say ok we will just leave 30 mins earlier to allow some contingency. So that’s a whole hour of contingency. They go to bed on Friday feeling confident and looking forward to the journey.

Have they got a good plan? They have broken the programme down to activities, they have evaluated options on the route, they have thought about some milestones, they have done a rudimentary risk assessment and they have added some management contingency. The programme or journey is set to start at 05:30 Saturday.

Execution – Driving the Route

At 05:30 the driver and one passenger are packing the car, the second passenger has just called saying that they have been unwell overnight and they will be about 15 mins late getting there. The car gets packed and they wait and sure enough the second passenger arrives at 05:45, they get in the car and start to drive, not bad but they have used 25% of their contingency before they have started. After about 30 mins they hit some roadworks and find that they are stationary for about 15 mins then the traffic is very slow. They carry on and feel that they are making good time.

By the time they reach the M6 Toll Road they have covered 130 miles and the time is 09.45, the phone rings, it’s the groom, he asks how they are doing and where they are, this is their first progress report. They say they were a little late leaving and they encountered traffic in London but they have made good progress and have just reached the M6 Toll Road. They have 273 miles to go and just over 4 hours before the wedding, one of them works out that they need to average 68.25 mph and since thats below the national speed limit they should make it.

They keep driving, the next few miles are slow moving, there has been an accident and the normally clear motorway is down to one lane, local traffic is adding to the volume of cars on the road as people start their Saturday morning. It takes them an hour to cover 20 miles. The phone rings again, they say they are on the M6 still, the traffic has been heavy but its starting to move, they tell the groom not to worry, they will make it to the ceremony. The passenger that does the calculations works out that they have 253 miles to go and 3 hours before the wedding, they can only get there if they average 84.33mph.

The M6 speeds up and they are now rattling along way above the legal limit. They see the Entering Scotland sign, they have covered 315 miles. They only have 88 miles to go but the time is now 13.15, they have 45 mins to get there forgetting about changing, calculator man says that they need to average 117mph for the rest of the journey, the phone rings, its time for their progress report. They say they are in Scotland now, that they have something less than 88 miles to go, and that the passengers are changing in the car and that they might be a tiny bit late. They can tell that the groom is now stressed but they do their best to reassure him. They are now out of integrity, they have distorted the real report, they have offered some trivial mitigation and they have calmed the client but are they going to get there with the ring?

No, but when they don’t make it and the wedding is ruined they will insist that they did everything they could, they planned, they considered options, they did a risk assessment, they built in contingency and they reported progress correctly (that is until the last few reports where they had to manually override the metrics because the tool was giving counter intuitive outputs)! Sound familiar in the context of failed programmes?

9 Causes of Programme Failure

  1. Not agreeing the criteria for success with the customer up-front
  2. Lack of breaking down the work into manageable chunks (a Work Breakdown Structure)
  3. Lack of identifying tangible and meaningful intermediate milestones
  4. Performing risk assessments under process to get a tick in the box, considering some template risks, assess and forget.
  5. Not estimating the effort and duration of activities based upon experience, (lines of correct code/day)
  6. Creating a bottom up plan from lots of detail that becomes impossible to communicate and control, the plan needs to be modular following the WBS
  7. Reporting progress in terms of a cost baseline not an earned value baseline
  8. Unrealistic reporting to the customer, withholding issues because these are not palatable will cause eventual failure when being open early may lead to changes and success.
  9. Poor or late decision making without working through the consequences.

Earned Value – We tried that back in the 90’s and it was too difficult

Many of you will have worked in organisations that have implemented or tried to implement Earned Value Analysis (EVA) in the past. Several things normally cause people to give up on it including:

  • Difficulty getting cost information from the finance part of the business into the system because it is typically delayed wrt project reporting periods.
  • Difficulty understanding the outputs of reports, uneducated senior managers reject it as theoretical
  • Difficulty making the planning tool work with EVA
  • Difficulty knowing what to report

Most people wouldn’t reject the idea of EVA, it is both compelling and logical, in fact it feels like one of those blinding flashes of the obvious when you first get it. The common practice of tracking budget or project costs in isolation to progress in terms of milestone achievement is bizarre because its of no real value at all.

What is EVA?

  • It was first used seriously in the 60’s by the US Military to control projects.
  • In the UK you might be taught it on a PM course
  • It is very rarely used in practice in the UK and if used it is rarely correctly or at least with true data.
  • Its not rocket science, nor does it stand for Extra Vehicular Activity!

To run EVA you need to be able to track actual cost of doing work, you need to be able to convert timesheets to money for the period in which work is performed. Sounds easy but for lots of companies this is difficult. You also need to have planned your programme out properly, to have used a sensible work breakdown structure (WBS) planned your activity dependencies and broken tasks down to a level where you can reliably from experience estimate the durations. To get the cost baseline over time associated with the planned work you have to apply resources to your plan, you need to ensure that resource conflicts are resolved and that the plan is workable. Finally you need to understand how your resources completed activities generate or earn value. So like everything in life, preparation is key. You need three things:

  1. A planned cost profile, if everything went according to your baselined plan, tasks completed with planned effort and resources. This is the Budgeted Cost of the Work Scheduled (BCWS) or in current terminology simply PLANNED COST.
  2. You need to be able to track on each reporting period what your resources (including material used if this is in the Planned Cost) actually cost. This is simply the Actual Cost of Work Performed (ACWP) or in current terms ACTUAL COST
  3. Finally you need to know what the completed work is worth according to the original PLANNED COST for a tangible deliverable and yes you can prorate for partially completed work if this is something you agree up front or you could say a 1/2 completed something has no value. This is the Budgeted Cost of Work Performed (BCWP) or simply EARNED VALUE.

That’s it, thats all you need to build into your plan and have mechanisms to track. Its not complicated but the insight it can give on reporting is very powerful indeed if made simple enough!

EVA Basics

EVA Basics

So now you can see we can plot over time the 3 cost variable as shown. From these it is possible to calculate the variances in cost and schedule and by the ratios between them we can also calculate schedule and cost performance indices. The Cost Performance Index (CPI) is probably the easiest to understand. CPI is BCWP/ACWP and what this tell us is the efficiency of our work against the original costs. For every £1 actually spent how much of the budgeted work is achieved. If the CPI is below unity say 0.85 then every £1 actually spent is achieving £0.85 value. If its over unity say 1.15 then every £1 spent is achieving £1.15 of value. The Schedule Performance Index (SPI) is BCWP/BCWS, if it is below unity we are behind schedule, if it is above we are ahead of schedule.

The beauty of EVA is in the preparation because now we can calculate these indices for the entire project or programme but also if we have done our prep for each WBS element, then for each activity or task and even for each resource. I have done this for resources years ago and the peer pressure of understanding who is the most effective in the team is powerful but should be used with care. The point is it allows a system view of progress, imagine a portfolio of programmes all with this implemented, it becomes easy to see which programme needs attention but better yet it is easy to drill down into that programme and understand where the problem lies.

Its all in the Presentation

The above graphical presentation is common, it’s in all of the text books but for some reason turns managers and business directors off. They feel useless, the engineers and project managers seem to have taken away their intuitive decision making, they are presented with a fait accompli and subconsciously they reject it, they find every reason under the sun why this is not worth doing. So I put it to you that EVA is not difficult to implement or to understand  however it threatens managers who are used to traditional business reviews.

How about this as an alternative presentation of metrics, I have used this to great effect over the years.

CPI/SPI QUADRANTS

CPI/SPI QUADRANTS

This is really simple to process in MS Excel with an X-Y Scatter Chart. What we plot is simply SPI and CPI per reporting period. Everyone quickly understands the target zone is (1,1) and labelling the quadrants gives everyone an intuitive overview. In a report or a meeting it promotes discussions and decision making. Managers and Directors like it because it gives them status that requires a decision, do they leave as is or take action. Again you can arrange the data such that you can drill down to problem areas and plot multiple WBS elements on one chart in different colours.

There are all manner of other calculations that can be performed, the estimate to complete, the estimated budget at completion, efficiency variances etc. There is plenty of reference material available from the PMI or the APM.

Programmes fail because….

Programmes fail because people forget:

  • To do proper planning
  • To manage risk throughout not just a quick appraisal at initiation
  • To include contingency and release it carefully
  • To structure work into a workable WBS
  • To breakdown tasks to a level that they can estimate duration from experience
  • To allocate resources correctly
  • To tell the finance department they need actual cost for the reporting period not the previous period
  • To do Earned Value Management
  • That the Managers and Directors that they report into have the power to help but they won’t if you don’t ask or demonstrate to them that it is necessary
  • Communication, of the plan, the issues. Remember that it is the customer’s programme too, issues should be openly reported with integrity, remember the groom!
Advertisements

11 responses to “Why do Programmes Fail?

  1. Hi George, the vertical (y-axis) is SPI, the horizontal (x-axis) is CPI. A CPI (BCWP/ACWP) below 1.0 is bad if it was 0.5 then for every $1.00 you spend of actual cost you only achieve $0.50 of the budgeted work. Similarly an SPI (BCWP/BCWS) below 1.0 is bad again if was 0.5 then you are only performing work at half the rate you scheduled it at. So the bottom left is “double bad”, behind schedule and over budgeted cost. The top right is “double good”, ahead of schedule and under cost. A point at (1,1) is where you want to end up. Each reporting period will give you a point on the x-y scatter graph, the locus of the reporting points will tell you whether the data is saying you should be able to sleep at night. Of course it’s just an indication but it’s a pretty good one. Hope that helps.

  2. Hi Keith, I’ve also known programmes to fail because there is no clear concept within the business team of what it’s trying to deliver. The problem here is not that you fail to deliver anything but that what you deliver cannot be mapped onto expectations so it will always be perceived as failing.

  3. Great article Keith. But who at the highest levels will read it. Pushing discipline into culture and ensuing that it becomes a behavior is the key. Richard

  4. Excellent points, Keith, and well made (I’ve worked as a professional evaluator in the United Nations system – as project evaluation officer for the Food and Agricultural Organization, it’s headquartered in Rome where I live today – I’m retired now and writing novels…)

    All this to say that while I appreciate the EVA concept, I believe it’s hard to apply because the “value” earned is something that is often intangible and beyond the control of the project team: it depends on other factors. For example, in development work, it depends on the rate of appropriation of the project results by beneficiaries: if you promote a development model with associated techniques that are not culturally in line with the degree of preparation of your beneficiary group, they won’t take it on board, your project will fail – as soon as you’re gone with your project management team and have left the country, whatever activities the project has launched will simply come to a standstill…

  5. Great comments, thanks to all. It is interesting to note that the difficulty of enumerating the “value earned” seems to be the biggest obstacle to people adopting EVA. The ‘earned value’ (BCWP) is simply the cost that your baseline plan (BCWS) stated it would take to deliver the sum of your project or programme’s “tangible” achievements across the WBS on a certain reporting datum. Sure there are principles that are up to you on the accumulation of value for partially completed tasks. You could argue that a 50% finished element is worth nothing until its finished or adopt a convention that attributes 50%. The biggest leap in understanding is that whatever you end up recording this is not the actual cost to date, which of course is meaningless other than for short term accounting and yet I could walk into a dozen companies and find that this is all they report along with achievements and issues next period. The power of EVA lies in comparing the ‘earned value’ with the ‘actual cost’and making predictions on the health of the project, forecasts of EAC and ETC, plus an insight on the accuracy of task estimating to improve the next project. To leverage the power of EVA it needs to be made obvious and compelling to those unfamiliar with the concept. Hence I would recommend using the quadrant plot every time since this put EVA metrics in simple visual terms for your stakeholders. If they don’t understand this you have bigger problems.

  6. Everyone makes mistakes, or takes the wrong route; but, if they don’t learn from from past experience, then they are doomed to repeat them.
    Experience has a value, if used appropriately.

    Many employees change roles, or even companies, out of frustration…because the same problems exhibit themselves, over and over; and no one appears to address them.

    Ignore the warning signs at your peril!

  7. I think from our experience, I would say a distinct lack of trust both within the programme team and between them and the client organisation. A lot of programmes go off track because the actual initial reason for starting it is forgotten and as such it drifts and fails to deliver the final/ desired outcome. A lot of this can be managed from having good governance and maintianing the business case/ benefits plan etc. It is surprising the number of times which these are just put to one side and ignored/forgotten.

    Government is now trying to tackle some of the issues with its programmes by initiating an accadamy for the people managing it. I hope that the increase in knowledge will continue to improve on the delivery of key projects.

  8. I recognise those charts . . . It’s the passion and drive behind the principles and realisation of why programmes fail that needs pushing every day. That lifts the passion I the delivery teams. Then build on modified EVA that’s right for your organisation to overcome the difficulties of its implementation.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s