Why IT projects still fail

IT organizations have worked hard to get away from the problems that had plagued their past project delivery processes.

They have replaced expansive scopes, the waterfall methodology, and long timelines with iterative development, the agile approach, and multiweek sprints, hoping to avert the big failures that have littered IT’s history.

Those changes have indeed helped, but many IT projects still fail.

True, project failure no longer brings down the whole IT environment — as it might have done a few decades ago. Nor does it typically mean a new system doesn’t work at all and needs to be completely scrapped, another scenario from IT’s checkered project-delivery history.

Instead, failure today means an IT project doesn’t deliver some or all its expected benefits, according to CIOs, project leaders, researchers, and IT consultants. Or failure may mean a project doesn’t produce returns, runs so late as to be obsolete when completed, or doesn’t engage users who then shun it in response.

“Today, with the advances in agile, the definition of ‘failure’ has morphed,” says Te Wu, CEO and chief project officer of PMO Advisory, an associate professor at Montclair State University, and a Project Management Professional (PMP) who also serves as co-chair of the Project Management Institute’s development team on the portfolio management standard.

In other words, IT project failure now is more about missed marks than technological disasters.

Some studies bear that out. According to KPMG’s 2023 Technology Survey, 51% of the responding 400 US technology executives said they have seen no increase in performance or profitability from their digital transformation investments in the past two years.

Surveys and studies are inconsistent on the rate of IT project failures, but researchers confirm that the percentage of IT projects falling short continues to be higher than hoped.

So what’s going on? Why do IT projects continue to fail? There are some common culprits.

1. Lack of project management expertise

Workers are often tapped to take on extra work, and for IT workers that sometimes means being tasked to lead projects.

But those workers don’t necessarily have the expertise, training, or experience necessary to succeed in the project manager role, nor are they given enough time to learn what it takes to manage a project or to complete the extra project management tasks.

“We expect people who have no training to run projects. Or we expect managers who have a full-time job doing something else to also manage a project,” says Zach Rossmiller, CIO for the University of Montana.

Rossmiller knows the consequences of that approach as his IT department has had a history of doing as much.

“We’d have a couple hundred different projects going on and were expecting our managers to run them and run them efficiently,” he says. “If we had a project management office dedicated to running projects, I think we would have been a lot better off.”

Rossmiller is addressing this situation, with IT recently adding two project managers to its team. The addition of trained project management professionals is already delivering improvements, Rossmiller says, particularly in streamlining efforts and making project delivery more efficient.

That’s because trained project managers are better able to corral and schedule resources, coordinate staff schedules, and get everyone moving in the same direction — and do so across multiple projects, he says. They’re also more capable of implementing the governance needed to keep projects on target to deliver what’s expected and not let scope creep run up costs and schedules without adding additional value.

2. Little or no executive support

Another leadership problem that can tank an IT project: top-level indifference.

That scenario happens more than it should, veteran project leaders say.

“We have had meetings and project status reports where executives were asking, ‘Why are we doing this?’ They truly didn’t grasp the importance of where we were going,” says Karla Eidem, a Project Management Professional and PMI’s North America regional operations manager.

Eidem says a lack of executive support can happen even when the project has a business unit sponsor — a situation which indicates that the project, while maybe a localized priority, isn’t aligned to organizational goals.

In such cases, there’s a need to realign.

“It may mean asking the [executive team]: Given all the facts that are laid in front of you, is this project a go or no-go? Sometimes it is a no-go for some reason, but that decision has to be made,” Eidem says, explaining that executive support is crucial for success because it ensures that the resources — money, people power, attention, and so on — will be made available.

3. No business sponsor accountability

Similarly, releasing the project’s business sponsor from accountability for the tech project’s success — and putting it all on IT — can doom the project.

“You really want to have good project leadership, and that starts with the person who wrote the business case being the sponsor, making sure the project is done realistically and being a champion,” Wu says.

Business sponsors, like executives, play an important role in articulating the business reasons for the project, establishing the expected benefits, and rallying resources to the cause. Their support also signals to staff the project’s importance, which helps drive adoption of new technologies and any associated process changes, Wu adds.

4. Lack of business sponsor engagement

A supportive and informed business sponsor can remove roadblocks and bottlenecks that arise, they can escalate problems to help get them solved quickly, and they can champion successes to build momentum and excitement for the changes ahead, Eidem says.

Those benefits go away when a business sponsor is MIA.

Eidem has seen that happen.

She was working on a complex multilocation software implementation that was steadily progressing until she needed that business sponsor to bring together regional divisions for the next step in the process. The sponsor dropped the ball on that, leaving some operational teams in the dark about what was happening and others questioning why they had to participate.

“We were seeing that information wasn’t cascading, and there was some resistance in some areas,” Eidem says, adding that the situation jeopardized the project’s forward momentum.

Eidem says she discovered that the business sponsor had been pulled away from the project by another big issue and as a result wasn’t as attentive as needed in delivering information about the project to the impacted regions.

“I needed the sponsor to be focused on this project, but he had too many other priorities. So I had to be clear: As an organization, we said this project was the No. 1 priority,” she says.

To prevent the project from collapsing, Eidem scheduled more one-on-one meetings with the sponsor to build a stronger relationship and to allow for more direct communication about the project’s needs, which helped refocus the sponsor and get the project rolling again.

5. Not involving all stakeholders

IT project manager Krista Phillips recounts one case in which a large multinational corporation implemented a new technology across its companies but caught one division completely unaware of the ongoing implementation work.

Turns out that specific division had been left out of all the planning and project processes.

“I don’t know how the [project leaders] missed that, but they missed a whole unit. So when the system went live, they were like, ‘What is this?’ That caused the project team to miss scope,” says Phillips, a PMP holder serving as president of PMI’s Pikes Peak Regional Chapter in Colorado.

Phillips acknowledges that project teams don’t usually overlook entire divisions, but they sometimes fail to identify and include all the stakeholders they should in the project process. Consequently they miss key requirements to include, regulations to consider, and opportunities to capitalize on.

6. Not enough resources or not the right ones

Some project leaders list the prevailing do-more-with-less expectation as another reason for failed IT projects today. They say this mentality generally leads to project teams lacking the resources that they need to get the desired work done on time.

“Everybody is very concerned with that bottom line, and they should be concerned about that, but the other side of that is they’re expecting a few people to do a lot of things,” Phillips says.

For example, she says workers are frequently assigned to multiple projects simultaneously, and many are assigned to that project work on top of their existing duties. As a result, these workers are pulled in too many different directions.

Others say enterprise leaders underestimate costs and the time required to complete the work or they fail to allocate the right talent to the team (thinking untrained or inexperienced workers will develop the required skills on the job), even as project managers surface the consequences of under-allocating the money, talent, and time needed for success.

Experienced project leaders say it’s crucial for IT project managers and CIOs themselves to ensure that the business sponsors and C-suite executives get the information they need to be realistic about the required resources, support, and schedules.

Those project leaders, however, also acknowledge that’s a continuously tough task.

To help, they advise implementing a portfolio management program to bring visibility into all the work happening and to break down project siloes that can sometimes contribute to that under-allocation of resources.

“An emphasis on portfolio management can close a lot of the productivity pain,” Wu says. “Do fewer things, too, and do them well; that’s a hallmark of portfolio management.”

7. Lack of in-person collaboration

Wu acknowledges the benefits that the shift to widescale remote work has brought, such as the ability to scale project work globally. But he also sees how the virtual work environment can create pain points in project delivery.

To start, Wu says he has found virtual teams have a harder time coming together to solve the harder problems. “I think when it comes to tackling module tasks, virtual is good. But solving big, complex problems, there’s only so much you can do over Zoom,” he says.

Wu says he also sees some teams struggle to bond in virtual work environments. “You sacrifice the human connection,” he adds.

Consequently, Wu says work can take longer; solving a problem that would take an hour working together face to face might take 90 minutes in a virtual environment. And handoffs that went quickly and smoothly when teams were physically side by side now often require more time to coordinate across computer screens.

“People are spending more time to get to the same net effectiveness, and I think human relationships are starting to fray,” he says, noting that newer employees in particular may be missing out on the informal mentoring and learning that happens in person when they can readily see good role models and good work patterns to emulate.

Wu says these dynamics in turn put more work onto project managers who must “put in far more work and time to be collaborative and to communicate,” adding that he hasn’t yet seen any remedies to these challenges. “I know the problem, but I don’t have a solution,” he says.

8. Disjointed handoffs

At some point a project moves into production, project managers move on, and the IT initiative is then expected to show its value.

Those steps in many organizations mean that projects — and any remaining problems with them — are still being tossed over the fence, leading to disjointed accountability that does not lead to success, Wu says.

“If the business case is one party, and the project manager is another party, and then people managing the results of the project, who are even further removed from the business case for the project, are another party, then you can see the disjointed connections,” Wu says.

That disjointedness can mean missed opportunities, wrong turns, and miscommunications that lead to final products that are suboptimal.

Wu says that disjointedness can happen even in organizations using Agile and DevOps methodologies, saying that those “are just different ways of carrying the water from one side to another.”

He adds: “DevOps can address a change very quickly, but whether that change is the right change or if it delivers ROI is a very different question.”

To counteract that, Wu and others point to the need for good management and strong project governance that continually link the project’s business case to its deliverables throughout the project work as well as through the implementation and adoption phases.