7 IT metrics that matter most

Metrics are essential tools that help IT leaders focus their teams and resources on important core business areas, such as transformation and operations. Yet while highly useful, metrics can also be dangerous when improperly used. It’s easy, for instance, to rely on the wrong metric to track a specific operation, a mistake that can lead to inaccurate or misleading results. Complicating things is the constant flow of new metrics, many of which have yet to prove their long-term value.

For the CIO assembling an analysis package that enterprise leaders can use to glean IT’s impact on transformations and operations, metrics minimalism is key. Avoid clutter and confusion to instead focus on a core set of fundamental metrics that provide clear insight into critical issues and challenges. While additional analysis tools can be added over time to augment and illuminate important trends, it’s best to begin with just a few basic tools.

Here’s a look at the seven fundamental metrics that should have a home in every CIO’s analysis toolkit.

1. Return on investment

As it has for years, return on investment (ROI) continues to be the central metric associated with transformation. “Transformation is all about doing business differently and generating value,” says Lou DiLorenzo, managing director and US CIO program and technology strategy lead at Deloitte Consulting. Besides providing insight into value, ROI, when performed correctly, will highlight the joint ownership between business and IT that’s necessary to bring transformation to life.

Project ROI should never be defined by a single IT project manager working in a vacuum, DiLorenzo cautions. Planning should always be a joint effort. Under this arrangement, business users own the numerator — essentially the reason why the transformation project launched — to increase sales volume, optimize margin through pricing, and so on. IT, meanwhile, owns the denominator — the cost to deliver the work on time, on budget, and with high quality.

“Each side needs to deliver on their obligation for the project to be successful, and piling up a number of successful, value-creating projects will help transform a business,” DiLorenzo says.

2. Business value delivered

Closely related to ROI is business value delivered. “This is the most relevant metric for the business, and it helps to justify the investment needed by the IT organization to drive the transformation initiative,” says Sameer Bhagwat, vice president and head of IT consulting firm Capgemini Americas’ Applications Managed Services Center of Excellence.

Specific IT transformation projects are frequently linked to overall enterprise digital transformation initiatives. But the success of these projects can still be measured by the traditional metrics of project costs — implementation timelines, functionality delivered, and so on. “The true measure of success is based on the business value delivered,” Bhagwat says.

According to Bhagwat, the ability to directly link IT transformation to business benefits will foster greater collaboration between business and IT leaders. “It also enables better governance on transformation initiatives, since initiatives that drive higher value for the business get prioritized.”

Business benefits can be quantified in three dimensions, Bhagwat says — revenue improvement, cost reduction, or working capital improvement. “This allows for easy comparisons across initiatives to identify those that deliver the maximum transformation benefits,” he states. “Quantifying the expected business value of an IT initiative forces the IT teams to better understand the implications and impact of the transformation initiative they are implementing.”

3. Online application availability

Yan Huang, an assistant professor of business technologies at Carnegie Mellon University’s Tepper School of Business, says that measuring online application availability provides a simple, objective way of assessing IT operational performance. “It can be measured by, for example, tracking the percentage of time the application is functioning properly and the average/variance of the time it takes to deliver a requested service,” she says.

Huang explains that the availability and efficiency of internal-facing applications significantly affects workforce productivity, while the availability and efficiency of external-facing applications greatly impacts external stakeholders, particularly customer experience, satisfaction, and retention. “All … will have a major impact on the organization’s bottom line,” she notes. “This metric is very helpful to organizations in detecting and solving operational issues and identifying areas for improvement.”

4. IT and business team engagement

Matt Mead, CTO of tech consulting firm SPR, believes that IT and business team engagement is a powerful metric for measuring transformation success. “We know teams experience psychological development through forming, storming, norming, and performing,” he says. “We also know it takes time for joint business and technical teams, typically created for an organizational transformation, to gel and get into a groove.”

Mead notes that he frequently sees clients failing to allot sufficient time for transformative changes to settle into place. Instead, poorly engaged IT and business project leaders declare success and move on, prematurely, to the next transformation stage.

“The best metric, especially during the first year of a transformation, is to measure the engagement level from all team business and IT team members,” he advises. “If engagement is high, you have the foundation upon which to be successful,” Mead says. “If engagement is low, your transformation is flawed, and failure is likely.”

5. Customer experience quality

While a digital transformation initiative may include a variety of goals, such as enhanced productivity, larger market share, or optimized operational costs, success is ultimately defined by a single thing: how do customers feel about the enterprise’s brand, product, or service, says Milind Damle, director of customer success at Infostretch, a digital engineering professional services firm. “If your digital transformation initiatives don’t improve the quality of the customer experience, are they really meaningful in achieving consistent top-line growth?” he asks. 

Damle notes that numerous reports over the years have discovered that customers place experience over price when making a brand decision. “This makes customer experience the most important metric in measuring the success of any transformation initiative,” he states.

6. Adoption and quality

By providing deep visibility into real-world product or service acceptance, adoption is an essential transformation metric, claims Adam Landau, CIO at rideshare and delivery vehicle insurance provider Buckle. “When you measure the adoption rates of individual features, you can begin to see what functionality is the most valuable,” he says. “Using this data makes it clear where additional investment should be made.”

Landau notes that it’s important to pinpoint and understand specific areas of high adoption. “This could be functionality provided, ease of use, or even something else,” he says. “This information should then be applied to features that have lower adoption so they can be refactored to increase adoption.”

Landau views quality as the most important IT metric for measuring operations. Organizations suffering from poor quality tend to have much lower employee and customer engagement and suffer decreased productivity, he explains. When organizations deliver lower quality products, they often waste valuable time and resources addressing the issue.

7. Technical debt index

Excessive technical debt can fatally damage even the most promising transformation initiative. When a project is developed and deployed in a rush, quality often suffers and the venture must inevitably be revisited to repair compatibility problems, security gaps, performance issues, and various other budget-draining headaches, says Shafqat Azim, partner, digital strategy and solutions at technology research and advisory firm ISG. Azim believes that technical debt is the biggest inhibitor to successful agility and transformation. “Managing technical debt actively as part of operations ensures an organization can transform as needed, when needed,” he states.

Azim suggests using a technical debt index to measure and track spending. “At a strategic level, the most important metric is the percentage of the overall technology budget being allocated to transformation,” he says. “It’s imperative to measure the percentage of the technology budget being allocated toward three areas: running the business, enabling incremental innovation, and enabling disruptive innovation.” At a tactical level, measuring transformation throughput and velocity of transformation is critical, he adds.