Oil and Gas Slow to Embrace Unstructured Data Management

“I’m pretty frustrated by the progress the industry is making.”

That’s Alec Walker, who has an MBA from Stanford Graduate School of Business, talking about professionals in the oil and gas sector and their commitment to machine knowledge, generally, and their understanding, or lack thereof, of unstructured data, specifically.

The industry, according to Walker, is stuck in its sepia-toned past, especially during the down times, and he thinks it’s past time to move on.

“It seems to me,” said Walker, “that many decision makers are still hoping to wait out the storm, as if the good old days of easy money will come back.”

Walker, CEO of the artificial intelligence company DelfinSia, said the embrace of such unstructured data (and more about that in a moment) is vital to the success of the oil and gas sector.

“Operational efficiency is software-driven and remaining ignorant about software is not a viable option,” he said.

Walker believes industry has been putting off this new paradigm of efficiency and productivity for so long, it’s actually getting quite good at the denial; he said professionals in the field have been hiding behind buzzwords and jerking from one decision to another.

To that end, he suggests industry should test what it knows, what it doesn’t, and how to bridge the information gap.

“I want the industry to organize hackathons so we can compare our tech openly to our competitors,’ for everyone to objectively determine the differences. That’s not a radical idea in the software world; why is that so hard in this industry?” he said.

Structured versus Unstructured Data

The knee-jerk reaction of which he speaks manifests itself when oil companies throw their weight behind the first potential solution they see without proper technical or business vetting.

Image Caption

As subject matter experts retire, less experienced employees need a way to find answers from historical documents.

Please log in to read the full article

“I’m pretty frustrated by the progress the industry is making.”

That’s Alec Walker, who has an MBA from Stanford Graduate School of Business, talking about professionals in the oil and gas sector and their commitment to machine knowledge, generally, and their understanding, or lack thereof, of unstructured data, specifically.

The industry, according to Walker, is stuck in its sepia-toned past, especially during the down times, and he thinks it’s past time to move on.

“It seems to me,” said Walker, “that many decision makers are still hoping to wait out the storm, as if the good old days of easy money will come back.”

Walker, CEO of the artificial intelligence company DelfinSia, said the embrace of such unstructured data (and more about that in a moment) is vital to the success of the oil and gas sector.

“Operational efficiency is software-driven and remaining ignorant about software is not a viable option,” he said.

Walker believes industry has been putting off this new paradigm of efficiency and productivity for so long, it’s actually getting quite good at the denial; he said professionals in the field have been hiding behind buzzwords and jerking from one decision to another.

To that end, he suggests industry should test what it knows, what it doesn’t, and how to bridge the information gap.

“I want the industry to organize hackathons so we can compare our tech openly to our competitors,’ for everyone to objectively determine the differences. That’s not a radical idea in the software world; why is that so hard in this industry?” he said.

Structured versus Unstructured Data

The knee-jerk reaction of which he speaks manifests itself when oil companies throw their weight behind the first potential solution they see without proper technical or business vetting.

“For example,” he said, “one leading oil company invested significantly in an AI vendor that had not yet built a solution, without bothering to explore whether that vendor startup had competitors.”

When a better vendor comes along, Walker said that oil company turns them away because of time and money already spent. The reason for this is that many in oil and gas don’t see or appreciate the sophistication of those who work on such intelligence or create programs or products that can help the sector.

“An oil major takes a quick look at an AI vendor and decides to build the solution internally with a ‘How hard could this be?’-mentality,” said Walker.

At the heart of the discussion about machine learning is the distinction between structured data, which is data with a home, in a manner of speaking, for it is usually numerical and often created by machines. Unstructured data, by contrast, tends to be textual and tends to be created by humans.

Therein lies the untapped reservoir of success.

“Let’s say that somewhere down the road, someone wants to see an example of unstructured data, perhaps having forgotten what it is,” he said.

He uses as an example this very article.

“How would they find the above example sentence buried in this paragraph, in this article, in this publication? Typically, the incumbent solution is key-word search. Most enterprise search falls into this category: a search tool lets users type in queries about what they’re looking for and then comes back with documents that contain the words used in the query,” he explained

But that approach only takes you part of the way there.

If, for instance, you ask, “Where is the most mature source rock in the Delaware Basin?” you will find tons of documents containing the words “Delaware Basin” or “source rock,” but Walker said the search engine will not understand the context or intent of your question.

It is that limitation that has existed in software since text was first digital, and it’s the reason why companies spend millions on data organization and tabulation projects.

“It’s also the reason why the energy industry spends 80 percent of their time looking for their own data, despite having enterprise search,” he said.

Preventing ‘Staggering Inefficiencies’

Walker – who calls himself “a big nerd,” has been a program development consultant at companies like Intel, General Motors, Swarovski and the Starwood Hotels and Resorts Worldwide Group – wants to help industry bridge this gap. To that end, he organized and hosted a course in Houston last month called “Unstructured Data and Machine Learning in Oil and Gas,” which was designed for those professionals without a background in machine learning, software engineering or data science. The purpose was for those in the industry to engage in an interactive simulation of a typical oil and gas workflow involving structured and unstructured data.

It was not, he emphasized, a “How To” seminar, nor even very technical.

“My goal with the course was to get practitioners in the energy industry up to speed on the issues and tradeoffs. Everyone is eager to create value, and their decisions about how they go about getting their work done need to be informed by the basics of contemporary data management,” he said.

This new horizon is important, not just for those in the industry, but for those who might want to join it.

“The inefficiencies created by mismanagement of this unstructured data are staggering,” he said.

This is because those inefficiencies created by mismanagement of unstructured data are given context by the growth in private equity backing, especially as it pertains to a lower hydrocarbon price environment, the proliferation of data sources and the aftermath of the big crew change.

“This prevents oil and gas engineers with software talent from wanting to join the industry. This prevents Wall Street investors from believing in the future of companies in this industry. This prevents current employees from feeling that their talent, skills and ideas are fully utilized on the job,” said Walker.

How serious a problem is this inefficiency?

“The number oil operators are throwing around is ‘80 percent,’ which means 80 percent of time is spent looking for data already contained in the organization,” he said.

Walker, who has earned a certificate in machine learning from Stanford and contributed to research projects in places like Mongolia, Japan, Singapore and North Korea, said an investment in machine learning will more than offset the cost of the new technology.

“For example, if an oil operator has two weeks to place a bid on a lease and needs to know whether and how much to bid, and they only have time at their current data analysis rate to explore 30 percent of their related data, and the last time they were in this situation they overbid by $1 million, there’s a pretty compelling argument to use a tool to help them navigate their unstructured documents at the speed of thought,” he said.

Walker first offered the workshop to his clients so all would be on the same page about the nature of the problems and the solutions that were out there, but he knew there were other audiences.

“I’ve since tailored it to the oil and gas industry,” he said.

It was a good fit.

“Thanks to AAPG for letting me try it out in San Antonio a few months ago,” he added.

As for the nuts and bolts of the course, as well as the overall philosophy of unstructured data and machine learning, Walker said it starts with familiarity.

“It is about raising awareness of how different types of data create different types of bottlenecks and challenges within organizations,” he said.

The goal is to make it happen. And quickly.

Comments (4)

More Followup from the Author
So how does programmatization of data management augment human performance? Currently, people spend large percentages of their company time finding the information that would otherwise be used on creative endeavors. Whenever someone wonders about something and doesn't act, creativity is stifled, and the chance for innovation is stifled with it. Currently, many wonderings are left unexplored because of the high time cost of researching the issue. "I wonder what would happen if we cut this part out of the process entirely," or "I wonder whether we could install heat exchangers on the waste water lines to power our surface equipment," etc. Our best practices and knowledge are buried in our own unstructured files, and our data is scattered across them. Finding them and intelligence gathering within them is a huge burden. Since the major risk of most creative endeavors is wasted time, a high upfront time cost of intelligence retrieval creates a bias toward inaction. If you lower the barrier to exploring information within the company, people are less limited by this barrier when attempting to be creative. Information access is therefore a catalyst to human-driven innovation. It also catalyzes proliferation of successful innovation within the company. I'm thinking about a time when I got creative and wrote a new model for a process, only to learn when I published it that someone else had written a better model a while ago. My initial search for such a tool had been fruitless, and yet it existed and could have been used by my team all along. Automating data management boosts creativity, but it also reduces lost time, lost opportunities, and redundant work efforts.
Show more
12/16/2019 1:35:28 PM
So, what about Geoscience?
Read the article, looked at the Delfin webpage, and watched the video. Interesting. Is Sia appropriate for cataloging, structuring and accessing large quantities of geologic and geophysical information and reports as well?
12/16/2019 12:10:24 PM
Followup from the Author
Thanks for publishing about my course, Barry. A follow up to the comment, hoping to stimulate more dialogue generally here: I agree that whether or not there is a felt sense of urgency when it comes to exploring new ways to generate ROI, individual creativity within the firm will continue to be a crucial component to the firm's competitive advantage. This will become all the more true as the pace of technological change continues to accelerate. I disagree that such individual creativity is generally coveted by leaders, however, as it often spells creative destruction of the comfortable practices that are nonetheless obsolete within the firm. It can feel threatening to have the foundation shaken of the pyramid one sits atop. I have seen this time and again from middle managers up to the C-suite, but that's a separate point. The main clarification I want to make is that my proposal is not to automate away the people, whose creativity is so important. I think automating a workforce like the oil and gas industry's is not only too hard for current technology but is also ill-advised. Instead, I am proposing to empower those people to spend their company time using their creativity to create value.
Show more
12/16/2019 9:24:15 AM
Interesting Perspective
Barry, thanks for providing this article with a different set of optics on the Oil & Gas industry. The comments made by Alec Walker emanate a sense of urgency that many of us also had in our careers at various times. I do not want to make a case for or against that sentiment. However, in our particular business, individual creativity is essentially "the secret sauce" needed for success and future discoveries. That in itself will remain elusive, very personal to most Oil & Gas explorers and something coveted by organisations that plan on building trust within their ranks. Personally, recognition of contribution by individuals if what that spells. That aspect of our business may not be well suited for machinisation and process efficiencies in what ever way one chooses. Feelings of frustrations by intrepid individuals are understandable but may not be the answer to advance the knowhow of successful explorers.
Show more
12/13/2019 10:23:26 AM

You may also be interested in ...