And now for something completely different.
Everyone who thinks the best approach to prospect evaluation is to get a collection of your smartest people to come up with the best group prediction, stand to the right.
While there, start thinking about a recent hit movie. Like "Star Wars III: Revenge of the Sith." Or "Wedding Crashers," or even "The Alamo."
Why? Because you might learn something about prospect evaluation — and about the perils of group mentality.
That's the theory of at least one geoscientist who loves his work and loves movies — and who has concluded that when it comes to predictions of success, there's definitely a correlation between the two.
Comparing movies to the oil industry might sound trivial — but when was the last time your reputation depended on a prediction that would be absolutely either a hit or a miss?
"Both are based on knowledge and intuition. I realize the limits of the analogy, but…the similarities are enough to make the study valid. Or at least interesting."
The geoscientist who loves the reel thing is Pierre Delfiner, E&P scientific adviser for Total in Paris who first focused on the connection several years ago thanks to two sources of inspiration.
One was Ed Capen, an expert well-known throughout the industry for his studies of prospect evaluations and, most famously, for his theory of "the winner's curse" (see December 2004 EXPLORER).
The other was his son-in-law, a French film distributor named Boris Pugnet.
Delfiner's interest specifically started about seven years ago, when he heard Capen give a paper in Paris about the winner's curse. He then read a Capen paper on the theory of competitive bidding and was "really impressed" with the insight.
The winner's curse theory says: In competitive bidding, the winner tends to be the player who most overestimates the true tract value.
"The proof of the winner's curse is based on the assumption that the mean of all estimates is equal, or close, to the true value," Delfiner said.
"This notion is very intuitive and very common," he continued. "It is the idea that the errors of the different estimates cancel out by averaging, and the larger the group of people that make an estimate, the more accurate the average."
But Delfiner wondered: Is this an established fact?
Should we tap into the collective wisdom of all our geologists? Should we trust our best experts?
Is the winner's curse inevitable?
"The problem with prospect evaluation is that we never really know the answer," he said. "We could work with simulations, but they lack the flavor of reality.
"I was intrigued by that, but I was trying to find a data set to check it."
That's when his son-in-law came into the picture by referring him to a game played by a group of French film experts who comprise "the Bet Club."
It was an offer he couldn't refuse.
The Crying Game
The Bet Club refers to a weekly public competition involving Parisian film experts.
It works like this: In France new movies are released every week on Wednesday.
On Tuesdays, the film experts log on to a Web site and record their forecast of the number of first-day tickets that will be sold in Paris for the new films.
When the actual box office totals are known, the "experts" receive a public ranking of their expertise. No financial stakes are involved. This game is all about reputation and ego.
Delfiner began to study the group, the predictions, the results and the trends — and quickly found analogs that translated to the oil industry.
For example, like the oil industry:
- The film industry is extremely risky. (Just ask Sylvester Stallone.)
- Each film is different. ("Jaws" was a hit. "Jaws 2"? You get the idea.)
- There is a large dispersion of estimates. (Remember how many studios passed on "My Big Fat Greek Wedding"?)
- There are some solid "nuggets." (You have "Star Wars" and "Spiderman," and you have Kashagan, the Caspian Sea discovery touted as the largest in 40 years.)
- Levels of information varies among players. (Indeed, some did their research and properly concluded that Brad Pitt and Angelina Jolie in "Mr. and Mrs. Smith" would be hot.)
- Estimation process is both analytic and intuitive. ("March of the Penguins": Who knew?)
For his data set, Delfiner tracked the Bet Club for six months (November 2004 to May 2005), and his sample included 42 players offering their predictions for the first day attendance figures of 125 movies in Paris.
He charted the mean estimates and the actual values, and to his surprise discovered that most of the time they didn't match, even when allowing for statistical fluctuations. On a case-by-case basis the experts as a group seemed to generate a bias.
On a global basis, however, the bias disappeared. The mean of all mean estimates came within 2 percent of the mean of all actual values.
And that's good, right?
(Careful. It might be a trick question.)
The Group
Delfiner concluded that the experts all had a similar bias in their estimates that "can be explained by the lack of diversity among players who use the same reference."
They were all from the same age group. They all had similar socioeconomic backgrounds. They all had watched the same movies all of their lives.
"If they all use the same reference, that reference will pull all the estimates in the same directions and create a bias," Delfiner said.
"Even though they were picking as individuals, their estimates were all based on the same information," he said. "That means their estimates are not independent.
"The same phenomenon may well be at work with geologists," he added, tying the study back to oil. "If they share the same cultural background, (and) the same exploration experience, they may all be driven by the same analog."
And that creates the misleading if not financially dangerous risk for a company of bias in prospect evaluation.
Over a large enough portfolio of prospects, however, there is much less danger of bias: "If we promise the mean," he said, "we will deliver the mean."
Delfiner also said he learned several lessons from his study that could be interesting if not beneficial to those involved in prospect evaluation — including a few new insights into the winner's curse ("you can predict it," he said) — but the universal lesson may lie in the cautionary look at group think.
"When recommendations are made by committees there is a risk of group behavior," he said. "For example, members may align with the most senior person."
And that may mean missing several good and valid ideas and estimates for the sake of unity.
"How to implement a decision process within a company needs further consideration," he said.
One idea he suggests: Ask committee members to make their assessments independently before they meet. After all, consider the success of joint ventures.
"Besides mitigating risk, joint ventures generally improve the quality of the work by providing a diversity of viewpoints," he said.
In other words, in evaluating prospects as well as movies, perspectives matter. Here's looking at you, kid.