Offered here are two class exercises that demonstrate how estimating through a consistent, systematic and calibrated process may enhance the profitability of your enterprise.
Intuition vs. Systematic Estimating
First, half a class views a slide showing a large number of beans (far too many to count!) for 15 seconds, while the other half looks away. At the end of this viewing period, this group of students (called the intuition group) each write down their individual best-guesses for the number of beans they think are on the screen, i.e. deterministic estimating from an intuitive bent.
Now, for the other half of the class - they also get 15 seconds, but they don't get to view the bean slide. Instead they view a distribution of estimates made by a previous class from the same slide of beans (during this viewing period the intuition group looks away). The distribution from the previous class is arrayed on a cumulative log probability graph, with the P10, P90, median and mean values clearly labeled.
Not surprisingly, the graph shows a nearly straight, sloping line on this coordinate system.
Does this seem strange? It shouldn't. It indicates that the distribution of estimates is lognormal, implying that individual estimates depend on multiplication of constituent factors, in this case, height, width and density of beans on the slide. According to the Central Limit Theorem, multiplication of constituent factors yields a lognormal distribution.
We call this second group the systematic group. Everyone in the systematic group must now write down their individual estimates of the number of beans.
Deterministic? Sure, but they are providing their best guess from a distribution, and thus benefiting from a broadened perspective.
At the start of this exercise, we collect 25 cents from each student, and pay back four times that amount ($1) if their estimate turns out to be plus or minus 10 percent of the actual number of beans.
Which group tends to make money in this exercise -- the intuition group or the systematic group? Well, take a look at our data (Figure 1). About 60 percent of the systematic group make a profit, but only about 12 percent of the intuitive group do so.
What's the message?
A systematic estimating approach almost always outperforms intuition, and group wisdom is better than individual best guesses!
Competitive Sealed Bid Sale: Antelope Ranch
Antelope Ranch is our fictitious-but-realistic township in the western United States that was recently made available for oil and gas exploration via the most common method for acreage acquisition - the sealed bid sale.
For this exercise, multi-disciplinary teams:
- Make maps;
- Probabilistically analyze prospects for both their size and probability profile leading to a chance of geological (flowable) success;
- Select a discount rate to convert the prospect reserves to present value; and
- Select a bid strategy multiplier to protect their team from the dreaded "Winner's Curse."
From all these data and analyses, the teams then execute their exploration strategy and compete for acreage, bidding with their own pocket money!
(The acreage winners then watch eagerly as their prospects are drilled and the economic results of their exploration campaigns become apparent.)
Just as in real life, some teams are shut out by not securing acreage. Some teams secure attractive acreage, but are left with dry holes. Some teams celebrate discoveries. But the object of the exercise is to create value, to generate a profit -- so some teams discover oil but suffer a loss, while other teams discover oil and are wildly profitable.
All of these results (which in our business may take years to unfold) are revealed over a few hours in the classroom.
Rigorous post-appraisals of each team's geotechnical estimates are made, compared against the school solution of the prospect parameters, so that a "skill score" is calculated for each team. The skill scores are then broken into thirds for each class, so that if there are six teams, the best two skill scores will reside in the upper third segment of the database, and the poorest two skill scores will reside in the lower third segment of the database (Figure 2).
What separates the money-makers from the money-losers?
Most of the time it is the efficient and effective application of probabilistic risk analysis. Our database of over 400 teams' skill scores, bid expenditures and profit measures is quite revealing:
Profitability correlates with skill!
Teams with moderate or superior skill recognize tracts with inherent value, bid widely wherever they calculate positive expected value - and when unlucky, don't lose a lot of money. The value of their discoveries, when made, greatly exceeds the capital they have spent to secure them.
Teams with poor skill typically overbid dramatically (oftentimes even more than the success value of the oil potentially present in the prospect!) Sometimes teams with poor skill tend to be conservative. For their efforts, they are shut out from securing the acreage.
While they have not lost money, they have lost something almost as valuable: their time.
The exercise reinforces the value of effective application of probabilistic estimating systems in portfolio management efforts.
Skill pays!