IT analytics: The hidden advantage
Transformational IT projects often fail because project management offices focus on lagging metrics, not predictive IT analytics. As a result they can’t spot problems early on before they bloom into crises that often derail the program’s budget, schedule, and delivery of new capabilities. The right IT analytics create an early warning system to keep projects on time and under budget.
IT analytics The hidden advantage
Beirut Fadi Majdalani Partner +961-1-985-655 fadi.majdalani @strategyand.pwc.com Chicago Mike Cooke Partner +1-312-578-4639 mike.cooke @strategyand.pwc.com Kumar Krishnamurthy Partner +1-312-578-4613 kumar.krishnamurthy @strategyand.pwc.com Boris Abezgauz Principal +1-312-578-4608 boris.abezgauz @strategyand.pwc.com Dan Holland Principal +1-248-680-3105 dan.holland @strategyand.pwc.com
Dallas Donald Dawson Partner +1-214-746-6503 don.dawson @strategyand.pwc.com Carter Utzig Executive Advisor +1-214-746-6551 carter.utzig @strategyand.pwc.com Muthu Manohar Principal +1-214-746-6569 muthukumar.manohar @strategyand.pwc.com
DC Nathaniel A.F. Clark Partner +1-410-274-8882 nathaniel.clark @strategyand.pwc.com Düsseldorf Jens Niebuhr Partner +49-211-3890-195 jens.niebuhr @strategyand.pwc.com Houston Kevin Heard Principal +1-832-474-9524 kevin.heard @strategyand.pwc.com
Los Angeles Dan Priest Partner +1-424-294-3749 dan.priest @strategyand.pwc.com Munich Nicolai Bieber Principal +49-89-54525-545 nicolai.bieber @strategyand.pwc.com San Francisco Danielle Phaneuf Principal +1-415-653-3518 danielle.phaneuf @strategyand.pwc.com
About the authors
Dan Priest is a partner with Strategy&’s digital business and technology practice and is based in Los Angeles. He works with clients across industries including auto, media, high technology, and financial services. He leads the firm’s IT strategy and effectiveness team and specializes in IT strategy and execution, operating model design, strategic sourcing and outsourcing solutions, and lean IT and operational excellence. Kumar Krishnamurthy is a partner with Strategy&’s digital business and technology practice and is based in Chicago. He has led high-impact engagements at a number of global enterprises across multiple IT dimensions — including business intelligence, technology strategy, and operating model design. Boris Abezgauz is a principal with Strategy&’s digital business and technology practice and is based in Chicago. He specializes in IT program optimization through levers such as analytics and sourcing to maximize the value of a program to the broader organization. Samar Sarma is an associate with Strategy& based in Chicago. A member of the ﬁrm’s digital business and technology practice, he focuses on IT strategy and program value realization from large ERP-driven business transformation across industries.
Strategy& associate Michael Neff also contributed to this report.
Digitizing the business value chain is critical for competitiveness, but the large transformational IT programs necessary to achieve this digital state often run over budget and well behind schedule. In fact, 80 percent fail to achieve full success. It’s not a new problem, but based on our extensive client work we believe we know the reason: Most project management offices focus on lagging metrics, not predictive IT analytics. As a result they can’t spot problems early on before they bloom into crises that often derail the program’s budget, schedule, and delivery of new capabilities. Ironically, most companies already have the data that signals problems early in the project when corrective action is easier and less expensive. To remedy this broken IT analytics process, we propose a three-step approach: Identify and track the right data, empower the team, and turn insights into business decisions. Companies that take these steps can create an early warning system to complete projects on time and within budget, accelerate benefit capture, and increase return on investment — not to mention become more competitive in their industry.
IT programs continue to fail
Digitizing the business value chain is critical for competitiveness in today’s global interconnected economy, but getting to that digital state is not easy. Most companies require a large, transformational IT program. Such projects are expensive, time-consuming, and sweeping in scope. What’s more, they often fail. Based on our experience and several studies, more than 80 percent of projects are deemed unsuccessful because they either don’t deliver the expected functionality or exceed their budget. These astoundingly poor results cost companies hundreds of millions of dollars, produce years of frustration, and sometimes cause serious reputational damage and noncompliance liability. A few recent examples include a global healthcare organization’s program to restructure IT and CRM systems that failed for a net loss of US$400 million; a personal care company that halted its global rollout of an SAP-based order management system and had to write down $125 million in software costs; and a public-sector program in the U.K. that involved a new monitoring and routing infrastructure that ran six years late and exceeded its budget by 50 percent. The list goes on and on. Granted, these types of programs are never going to be easy to execute. There’s a natural complexity given their enterprise-wide scope and the number of stakeholders involved. There are hidden program risks to operational integrity, exacerbated by the tendency of executives to underestimate potential complications. But even these factors cannot fully explain a mere 20 percent success rate when so much of the organization is committed to the program’s success. What exactly is happening, or not happening, at these companies to create such woeful outcomes? Based on our extensive client work, we believe we know the reason: Most companies rely on traditional project management office (PMO) practices, which focus on lagging indicators, not predictive IT analytics. In other words, they have only a backward view of the program’s progress, and are not set up to spot problems before they become crises that can derail the program’s budget and schedule. For example, our experience shows that the strength and maturity of a program’s change management process will affect the expected volume
of defects. Yet PMOs typically track only defect rates, which are an after-the-fact indicator that things have gone wrong, rather than tracking change-control processes, which are known to signal potential quality problems ahead of time (see Exhibit 1, next page). Today, most companies track massive amounts of data (e.g., time data, budget data, requirements data, fault-density data), which they present in hundreds of dashboards. However, these metrics usually measure discrete processes and do not identify any of the interdependencies present within an IT department or a program. Too often, IT departments focus on maximizing the dashboard technology itself and not on analyzing the findings. As a result, the IT analytics process is effectively broken at most companies (see Exhibit 2, page 8), suffering from data paralysis (programs swimming in large volumes of data with no clear insights), lagging indicators (hindering preventive action), and leadership apathy (program leaders hesitating to use analytics to tune the program strategy and delivery approach). Here’s the kicker: Most companies already have the data that reveals problems early in the project, when corrective action is easier and less expensive. Companies could create early warning systems, but they fail to track the right data, interpret the data, and turn conclusions into actionable business decisions to guide the project and keep it on track. If they did, they would have a much better chance of bringing projects in on time and within budget, accelerating benefit capture, and increasing return on investment. Not to mention, they could become more competitive within their industry by accelerating time-to-market and lowering the cost to achieve critical new business capabilities.
Companies already have the data that reveals problems early in the project.
Exhibit 1 Ability to influence program success across development life cycle
Establish objectives Deﬁne requirements Program planning and managing Speciﬁcation and design
In uence over success Low Planning and analysis
Ability to inﬂuence program success
Coding and testing
Source: Strategy& analysis
Exhibit 2 Causes of broken IT analytics process
Limited leadership foresight leads to business management apathy and overreliance on structured standardized PMO processes
Leadership blind to impending issues
Ocean of data ﬂoating in disparate systems
Wrong KPIs lead to collection of wrong data; with no business owner, centralized IT analytics teams fail to provide insight that leadership seeks
Wrong metrics fail to act as leading indicator of impending risks; with no incentive to speak up, people silently follow suboptimal processes
Disengaged stakeholders, no incentive to speak up
Wrong people running and presenting analysis
Wrong people with no appetite for value-driven decision derail the program or the team by giving inefﬁcient directions
Source: Strategy& analysis
Three corrective steps
We have identified three steps that companies can follow to fix a broken IT analytics process and create a well-geared process: Identify and track the right data, empower the team, and turn insights into business decisions. Identify and track the right data To move the organization’s focus from lagging metrics to predictive analytics, organizations must identify, collect, and use the right level of data throughout the life cycle of the program. In the broadest sense, this involves data concerning quality, speed, and predictability. More specifically, the company needs to dig below the qualitative hunches of managers monitoring dashboards to quantitative insights based on analytics. Take the issue of “requirements.” When building a new human resources system, for example, there is the initial phase when the business units define all the functional requirements that must be built — how to track hours, record vacation time, count taxes, and so on. Once that identification phase is completed, the company moves to build those requirements. Trouble comes when the requirements start changing during the build phase. This creates duplicate work, which raises costs and delays the project. Most PMO programs track “changing requirements” but don’t delve much deeper, leaving managers to make qualitative guesses about what is driving the changes. But by applying analytics to the data behind the dashboard, the company can know with more certainty what’s going on and create an early warning system to catch and correct developing problems. For example, a company can use analytics to do more than just count requirement changes. It can measure requirement volatility, and see at what stage of the program the requirement changes are occurring, as well as the quantitative impact of those changes on cost, schedule, and scope. It’s also possible to see where the requirement changes are
The company needs to dig below the qualititative hunches of managers.
coming from. Is there a specific business unit responsible? If so, business architects can begin to tease out the real reason behind the changing requirements. Is there a lack of clarity about the real business need the requirements are intending to articulate? Do the teams know which requirements drive the majority of the business value, and which could or should be pushed to future releases? Are the new requirements critical to the business or “nice to have”? Does the value of enabling the business functionality associated with a change request justify the additional cost and risk it brings to the program? By assessing the requirements and management practices and assigning quantitative scores, teams can spot risks early by estimating downstream implications. One of the best ways to help stakeholders make requirement trade-offs is to frame those decisions in clearly understood business terms — such as the degraded probability of the program’s success or the lowered expected return on investment. Another popular dashboard metric is progress toward a major milestone. But at most companies, this is a very two-dimensional metric. Once again it’s necessary to dig down a level. For instance, how does the team’s actual productivity compare with the original assumptions? Comparative analytics can allow a company to see that 80 percent of the team is generating certain deliverables in an average of eight hours, while 20 percent of the team is taking 36 hours to produce the same deliverables. Armed with that knowledge, the program managers can take corrective actions, such as reevaluating the project’s scope, addressing any skill gaps, or identifying additional resources that are required to keep the project on schedule. Indeed, time and again we have seen the importance of identifying the right data and drilling down for insights. We once analyzed more than 1,500 critical errors in an ERP transformation program and found that 35 percent were related to software problems that should have been caught in development. To find the root cause of the poor software quality, we began interviewing employees and were told by 75 percent of them that unit testing was a major weakness in their development organization. Lack of adequate tools and a poor testing environment were key factors (29 percent mentioned them), but the fundamental issue was the compressed time frame and the pressure to meet the cut-off date for Level 2 testing (65 percent). Absent such analytics, teams will rely on more qualitative judgments, which often lead them to focus on the wrong problems.
Time and again we have seen the importance of identifying the right data.
Empower the team Make sure you have the right people looking at the data, which includes training and encouraging project managers responsible for creating, presenting, and monitoring executive scorecards. It’s critical to remember that these analyses are not intuitive and most project managers have not been trained in IT analytics. Their expertise is often in being well-organized and driving things to conclusion, but they do not have the functional or technical knowledge (and often the patience) to get to the root cause of a metric, or to understand the interdependencies with other metrics. Companies need to move away from a “check the box” mentality to a higher, more sophisticated skill level to tap the value of these IT analytics (see Exhibit 3, next page). Aside from the daily involvement of project managers, the company needs to make sure that functional experts and solutions architects are engaged with the data and working alongside the project managers. These relatively senior people must understand the business, technology, and IT processes, and have the curiosity and persistence to identify and track these issues. As noted earlier, most companies have lots of data on time lines, sequences, risk, vendor spend, and capacity management, and other relevant matters; what they lack are useful insights and the ability to present complex analysis in simple, actionable business language. By working with the PMO, these functional experts and solutions architects can unearth drivers of current or potential problems by asking questions such as these: What is the probability that I am going to overrun my budget for the year? What are the implications of sequencing and resource constraints on my investments? What are the bottleneck constraints in my portfolio? What is the impact on potential costbenefit value proposition? For example, one Strategy& client undertaking a multiyear transformation program needed to predict costs and time lines for many subprojects and put contingency plans in place for any potential delays. We looked across all the programs and used statistical analyses to develop various project completion scenarios and the likelihood of each. Then, for each scenario, we calculated the risk impact on schedule and cost. With a clearer understanding of the underlying risk, the company was able to develop cost contingencies and create a plan suited to the organization’s risk appetite.
Exhibit 3 A well-geared IT analytics process
Goal to gain key program insights drives data-collection agenda
Right people Principles of root-cause analysis are ingrained at every stage of the program
Functional leaders with deep content knowledge lead the analysis and draw actionable insights
Root cause analysis
Simpliﬁed synthesis and foresight
Focus is on presenting complex analysis in simple, actionable business language
“Smart” analysis eventually results in concrete decisions
Integration into decision making Preventive intervention
Finally, maturity level is reached at which problems are solved even before they occur
Source: Strategy& analysis
Case study: A CPG conglomerate struggles with global ERP implementation
A major global consumer packaged goods (CPG) company undergoing an ERP implementation found itself significantly behind schedule and over budget, with the majority of the company’s business units still needing to deploy the solution. Everyone involved — the client, the software vendor consultants, and the systems integrator (SI) — knew the project was off track, but the momentum of the project kept pushing them forward and deeper into trouble. The team was continually fighting fires on the previous deployment, while simultaneously scrambling to make the next deployment go live. Strategy& applied analytics to IT tickets, time analysis, and change requests. All three revealed flaws contributing to the delays and overruns. For example, a simple analysis of the company’s existing IT ticket data showed a continuous increase in tickets per user, evidence that the SI was struggling with problem management. The SI was able to take these insights and shift to a more proactive delivery model that promised to lower ticket volume. In other words, even basic analysis of existing data that is readily available can be very helpful.
Turn insights into business decisions Once the right data is being tracked and interpreted through a business lens, those insights must be passed to the right people to make actionable business decisions. This requires clear lines of communication, decision rights, and accountability within and across businesses to coordinate responses and prevent concerns from being siloed until they become intractable enterprise-wide problems. Team members responsible for the daily rollout of the transformational program must speak up when they spot trouble, and share observations with managers. Unfortunately, the culture and governance structure in many of today’s organizations discourage people from speaking out, especially with bad news. People get ahead through relationships and political alliances instead of by presenting executives with fact-driven data analyses to prove or disprove hypotheses related to program performance challenges. The net result is that problems often remain hidden until they blow up and become huge, expensive, timeconsuming distractions. A well-defined governance structure, when sustained through the entire duration of the program, can ensure that insights are passed to the right people to make actionable business decisions. In our experience, even when programs begin with a well-defined governance structure, governance often starts giving way to “bad behavior” as the program progresses. Over time, decisions are made based on mutual understanding and individual working relationships. Either information is not allowed to move smoothly through the hierarchy, or facts are obscured and only selective information flows through. Eventually, the fate of the program falls into the hands of a few select, engaged individuals. In other words, good governance depends not just on setting up the structure, but also ensuring that it is sustained throughout the length of the program to support the IT analytics process. In an ideal world, a dedicated project management team that is trained in analytics and IT processes can glean insights — including danger signs — from data and present options for executives.
Driving the program
Ultimately, a company should construct a well-geared IT analytics process to recognize early signs of trouble for proactive intervention, enable continuous monitoring and improvement, identify unknown constraints and bottlenecks, and provide a negotiating lever with vendors during contract renewal. With these capabilities, the company has a much better chance of avoiding budget overruns and finishing the transformational IT project on time, accelerating value capture and boosting ROI.
Strategy& is a global team of practical strategists committed to helping you seize essential advantage. We do that by working alongside you to solve your toughest problems and helping you capture your greatest opportunities.
These are complex and high-stakes undertakings — often game-changing transformations. We bring 100 years of strategy consulting experience and the unrivaled industry and functional capabilities of the PwC network to the task. Whether you’re
charting your corporate strategy, transforming a function or business unit, or building critical capabilities, we’ll help you create the value you’re looking for with speed, confidence, and impact.
We are part of the PwC network of firms in 157 countries with more than 195,000 people committed to delivering quality in assurance, tax, and advisory services. Tell us what matters to you and find out more by visiting us at strategyand.pwc.com.
© 2015 PwC. All rights reserved. PwC refers to the PwC network and/or one or more of its member firms, each of which is a separate legal entity. Please see www.pwc.com/structure for further details. Disclaimer: This content is for general information purposes only, and should not be used as a substitute for consultation with professional advisors.