Not your father's oil and gas business

Not your father's oil and gas business: Reshaping the future with upstream digitization

Executive summary

Digitization is the new lubricant for the future of the oil and gas industry’s upstream sector.

With supply and demand mismatched and prices likely to remain low until that imbalance starts to ease, the need for new efficiencies is urgent. The industry is going through a wrenching period of change: The name of the game is no longer big production but better margins. Digitization — the collection, analysis, and utilization of huge amounts of data — can bring substantial operational improvements to the field through the use of sensors, analytics, robotics, and control systems.

To be sure, digitization is nothing new, but the oil and gas industry has yet to broadly embrace its implementation on an end-to-end basis. The exploration and production, or upstream, sector has been particularly slow to adapt compared with both midstream transportation and storage operations and downstream refineries and marketing functions. During the industry’s recent boom years, there didn’t seem to be a need to invest in radical new upstream technology, as high energy prices swept away much of the cost of inefficiency. Then, too, information technology has traditionally not been seen as an essential element of operations. Although the expense of these programs is moderate, the need to commit significant personnel, time, and leadership has been off-putting, especially in the face of the cultural changes required. For too many companies, digital has been relegated to a “nice to have” function, implemented in a piecemeal way with off-the-shelf systems. As a result, the technology has been installed on a siloed basis, limiting its full value.

But more companies are giving digitization another look now that the industry is struggling with a particularly painful downturn. Their future, and the industry’s, will depend in large part on how quickly they can rewire their operations to create a complete ecosystem that will set digital’s power free.

Not only does digitization bring much-needed efficiencies, but it also makes it possible for upstream companies to develop powerful new capabilities in order to benefit from smarter exploration, easier capture, and safer operations, all with much better utilized labor. Done well, digitization can also transform the planning process with its predictive analytics, giving companies a better chance to anticipate, and respond to, continuing market shifts.

And as free cash flow and the ability to generate returns become ever more important to an upstream company’s valuation, digitization can help here as well. By accelerating the industry’s very slow payments process, technology can free up cash for exploration, lower a company’s operating expenses, and contribute to higher margins per barrel.

This report examines how digitization can benefit the upstream sector and what companies can do to get around the obstacles to change. In future reports we will examine in detail similar benefits for midstream and downstream players. (For a general overview, see “The impact on other sectors.”)

The impact on other sectors

The downstream sector, long plagued by periods of low margins, embraced digitization earlier than its upstream counterpart. Modern refineries are now some of the most automated facilities in any industry.

Embedded smart sensors in vessels, tanks, compressors, and turbines send real-time data to control rooms where a handful of experts can monitor processes and provide diagnostics. Every piece of equipment is under constant surveillance, control loops adjust themselves, and whole-process analytics and simulations enable operators to predict the effects of changes in the system.

But the sector could still do more. For example, additional opportunities exist in connecting biometric data to improve operator safety and in enabling intelligent materials movement within facilities.

Much like their downstream brethren, midstream companies leverage vast digital components — in their case, to help them oversee gathering systems, pipelines, and storage facilities. Advanced measurement devices such as electronic flow metering, pipeline inspection gauges, and digital sensors are prevalent across the sector.

Despite all of that effort, only a few midstream organizations have captured meaningful value from their digital investments. Future advances in the midstream sector include integrating field, compliance, and maintenance data to enable analytics and simulations.

Time to double down on digital

At points in the last couple of years, crude oil and gas prices have fallen significantly. In the wake of the disastrous spill in the Gulf of Mexico, there is no ceiling on potential environmental liabilities. And with production substantially outstripping demand, excess stock keeps building, exerting pressure on storage options and forcing laden tankers to wait their turn to unload.

Senior oil and gas executives have always faced any number of strategic challenges. But the energy market’s current turmoil feels more perilous, longer-lasting, and more difficult to navigate.

For two years, production has sharply exceeded demand, flooding the world with an oversupply that reached as much as 1.8 million barrels of oil equivalent (BOE) a day in 2015 (see Exhibit 1). Predictably, prices experienced something of a free fall. Although the second quarter of 2016 saw a slight rebound, low prices are expected for some time to come, with a vigorous recovery unlikely until supply grows at a slower pace, allowing demand to catch up. Wild cards such as embargoes, political unrest, and fiscal requirements might further influence the scenario.

Exhibit 1: Estimated incremental production requirements

Estimated incremental production requirements

In the meantime, upstream oil companies must deal with far more fluid conditions than in the past, requiring much more flexible responses. Which drilling operations should be slowed down, and which sped up? Which assets should be self-operated, and which operated by others? Which markets need more stock, and which less?

To move ahead through market turbulence and shrinking margins, while avoiding a catastrophic environmental event, companies will need to make significant operational improvements. In their rugged physical world of massive drilling platforms, tankers, and pipe systems, the unlikely change agent turns out to be the largely invisible technology of digitization.

Digitization involves physical devices communicating directly with one another — machine-to-machine — with little or no human intervention.

In the oil and gas business, that means smart elements such as sensors, measuring devices, and actuators embedded in drills, wellheads, vessels, and other field equipment that constantly exchange data.

Wireless networking connects these elements and sends their data to servers for processing, storage, and analytics. Millions of smart elements “working” around the clock produce almost unimaginable flows of information. An oil field might send one petabyte or more of data per day — picture a library crammed with the books from 50 million trees.

Countless industrial operations can be improved with the much-discussed Internet of Things. Production rigs can be largely, if not fully, automated; the flow of crude and its composition can be constantly monitored; and the process of finding new reserves can become far less hit-or-miss.

In a control room, personnel see and interact with virtual representations of the field and all of its components. The data is processed continuously, with applications automating decision making, performing predictive analyses, reacting to alarms, and monitoring and controlling production processes — with or without human intervention.

Digital analytics, robotics, and control systems need to be integrated into the core of the business.

Digital technology has long been employed by the industry as a cost-cutting tool, but it can be much more. Now, with the industry’s focus shifting from production growth to bottom-line growth, digital analytics, robotics, and control systems need to be integrated into the core of the business.

Indeed, digitization has to start flowing through all upstream operations — delivering new efficiencies but also powerful new capabilities. Software will be the lubricant for smarter exploration, easier capture, and safer operations. Everything will be geared to real-time realignment with real-world changes, and all will be managed by a leaner staff.

Over the next decade or so, digitization will radically transform the industry. Companies will still be producing oil and gas, but in ways that will be virtually unrecognizable.

A long tradition of obstacles

Almost all major and independent oil and gas players already have some form of digital program in place. Some launched theirs more than a decade ago and are making great strides — but few, if any, companies have captured the big prize: an end-to-end “rewiring” that taps into digital’s full transformative power. In fact, many companies have barely scratched the digital surface, jeopardizing their competitive position.

The barriers to digitization include the usual suspects: high investments of time and personnel and slow-to-appear returns. Until recently, an even bigger barrier was organizational inertia.

There was little incentive to go through the effort and expense of a digital transformation when prices were high and profits were flowing. Value in the industry was mostly about production: the size of a company’s proven reserves and how much oil it was able to bring to market. The bigger the numbers, the higher the value. So instead of investing in digital, capital was allocated to “drill a few more holes.” Even if some of the production was of lower margin, it didn’t really matter because of the high prices.

Ironically, it was those high energy prices that in part sent two other industries, the airlines and the automakers, into the arms of digitization. The soaring cost of gasoline cut sharply into airline profits and helped depress sales of new cars. For both the airlines and the car companies, going big with digital produced savings through the bad times and added greatly to the bottom line when fortunes improved.

But as the beneficiaries of high prices, upstream companies remained very hesitant about committing to digital technology in an equally ambitious way.

Even with the recent downward shifts in commodities prices, significant opportunities existed in the oil and gas industry to improve the cost of production, or lease operating expenses, in more “traditional” ways. These included improving supplier contracts, simplifying work processes, using improved drilling techniques, and shutting down lower-value assets. But these approaches lead only to incremental improvements — not to fundamental change. Even taken together, these marginal improvements are simply not enough when the price of the industry’s core product has fallen by 50 to 75 percent, as it has since the beginning of 2014.

By contrast, digital allows for a complete reconsideration of how things operate — including a reexamination of such basic assumptions as the needed level of labor and inventory. And that helps to explain why the industry’s indifference to digital is fast disappearing.

For the short term, the pressure on margins continues to lead to reductions in capital expenditures (capex) for exploration. Spending is expected to hold steady for U.S. shale deposits over the next four years, but continue to decline for offshore development (see Exhibit 2).

Exhibit 2: Annual and average capex1

Annual and average capex

For the long term, another piece of calculus is coming into play involving lease operating expenses (LOE). Oil companies are going to be formally valued on the basis of their free cash flows and their ability to generate returns — i.e., on their margin per barrel. And that margin turns on a company’s LOE. Even a small producer that has high margins, courtesy of a low LOE, is going to be seen as inherently more valuable than a bigger company with a high LOE.

A long list of benefits

Whether applied to the short-term or long-term picture, digitization has the center stage. Profitability increases on every front as costs are taken out through intelligent maintenance, workflow automation, better labor utilization, and increased standardization and simplification of designs, processes, and equipment.

More automation also leads to better health, safety, and environmental performance, lowering the odds of that one bad accident that can prove ruinous. Sensors that monitor pressures and other stresses on drilling equipment and transfer pipes are just one of the safeguards. Having fewer people, sometimes none, on dangerous drilling platforms obviously keeps more employees out of harm’s way. And the analytics applied to seismic probes for new reserves can foster more effective, less expensive exploration, with drones and “wearable” technology (sophisticated data entry and display devices) keeping a watchful eye out to avert, or at least minimize, work errors and other accidents.

Digitization can also help accelerate cash recovery and lower operating costs. Years of limited technology innovation have left upstream companies with a long cash recovery cycle. In many cases, the time from production to payment can be as long as four months. In other industries, that sort of delay would be a death knell. Putting digital to work in this area can speed up the process enormously while ensuring accuracy. Freeing up cash flow will be critical to keeping the capex spigot flowing to develop U.S. shale, since it is now so difficult to gain access to capital (see Exhibit 3).

Exhibit 3: Projected industry capital

Projected industry capital

For all of these reasons, digital has become a critical capability for upstream companies.

Because they can’t sell or store oil as easily as before, upstream companies need to ramp up and ramp down output in sync with demand.

Companies become more agile with digital. That means they move more quickly, of course. But speed isn’t the only dividend. Because they can’t sell or store oil as easily as before, upstream companies need to ramp up and ramp down output in sync with the ebbs and flows of demand, whether driven by market, seasonal, or geopolitical factors. Digitization gives them the tools to do that. Similarly, annual planning exercises can now be replaced by continuous updates, making it easier to prepare for major projects through simulations.

Even marginal improvements in operational efficiency might have a big impact, given the the huge production volumes. In a tech industry–sponsored study last year, Oxford Economics estimated that the oil and gas sector’s increased adoption of the Internet of Things could produce inflation-adjusted gains of up to US$816 billion in global GDP by the end of 2025.

A lot has to be done to hit that number, however. At one end of the digital spectrum are oil fields in which sensors barely exist and operations are still paper-based. At the other end are companies that are already using unmanned robotic drilling platforms. Few, if any, players, however, have made it all the way to installing a complete digital ecosystem.

Capturing the prize

Even though digitization is still in its early days at many oil and gas companies, it is propelling five major trends that are or will be hugely beneficial to the business:

  1. Internet of Things. As the cost of sensors and actuators falls and the amount of data collected from connected devices grows, the increasing availability of predictive analytics is driving a higher return on investment (ROI).
  2. Building alliances. With mergers and acquisitions down, companies are forming alliances to create and deploy end-to-end digital technologies.
  3. Simplification and standardization. The search for end-to-end technologies is fostering the simplification and standardization of designs, processes, and equipment across the full industry life cycle.
  4. Solution-based buying. Companies are shifting from purchasing discrete services to solution-based buying, eliminating disparate interfaces and data (along with the risks they bring).
  5. Knowledge transfer. Consolidation of expertise among independent and national oil companies, as well as oil-field services companies, means costs are cut, risks are mitigated, and ROI is accelerated.

Those trends are playing out in all parts of the upstream operation. Value creation through digitization is being pursued in everything from data management, operational analytics, and field surveillance to operations automation, integrated field planning and delivery, and asset optimization (see Exhibit 4).

Exhibit 4: Capability imperatives in upstream

Capability imperatives in upstream

Yet at too many companies, the power of digitization has been diluted or largely lost because of fragmented implementation, often in silos, and decisions to buy commercial off-the-shelf systems instead of developing in-house systems.

Traditionally, most oil and gas companies have viewed information technology and data/digital activities as unrelated to the core business. As a result, each of their departments has typically used its own suppliers to deliver expertise and solutions that deal only with its immediate individual needs.

But digitization confined to silos does not provide the cross-functional, cross-asset insights needed to drive efficiency at the enterprise level. What’s more, financial, process, and organizational information is rarely integrated into any of the digital solutions. Making matters worse, each initiative has to “start from zero” in terms of building the business case, gaining stakeholder buy-in, mobilizing a team, setting up infrastructure, and addressing quality issues.

Although there is much sporadic innovation across the industry sectors, there are no end-to-end digital solutions common to the business and no clear leaders.

That’s the glass half empty analysis. To see the glass half full is to see the untapped potential of digitization. Capturing that potential requires a fundamental shift in strategy and operating model:

  • Shared standards. The melding of operational technology with information technology calls for a common set of standards that meets stringent operational rules but also allows for information sharing across the organization.
  • Collaboration and cross-training. The seamless integration of digitization mandates multidisciplinary domain knowledge that encompasses information and operational technology, but also data management, process design, and cybersecurity.
  • Self-funding projects and change management. Identifying potential brownfield projects that are prime candidates for digitization drives organizational momentum and reduces resistance to new technologies.
  • Security and compliance. As assets become increasingly connected to the data network, it is important to protect critical digital infrastructure against cyber-attacks — by monitoring threats, identifying vulnerabilities, installing robust controls, and promoting a culture of security awareness.

Above all, what is needed is a strategy that works. Winning companies focus on building integrated capabilities systems that differentiate them in the marketplace, and then scale those systems across their entire organization. They also stop fighting the imperfections in their culture and instead identify and leverage the parts that work in their favor.

It’s time to get that strategy working. Right now, in upstream oil and gas, end-to-end digitization is the prize, and it is up for grabs.


Digitization hasn’t quite yet ripened into a full-fledged revolution in upstream oil and gas. But it will. Digital has the power to help the sector reap the most from its boom periods and avoid the worst damage when things go bust. The sector is starting to fully recognize that power, and more and more upstream companies are scrambling to seize it.

As with any revolution, those in the vanguard tend to do better than those on the sidelines. In this revolution, the oil and gas companies that wring out new efficiencies by building sustainable digital capabilities will be tomorrow’s winners.