If a bat and a ball cost $1.10 together, and the bat costs $1 more than the ball, how much does a ball cost?

This is a classic riddle. I’m assuming you’ve seen it before. If you’re like most people, your mind quickly came up with $0.10 for the ball. Even if you’ve heard the problem before, your mind still likely jumped to this conclusion. But then one of two things happened. Either you accepted the answer your intuitive mind came up with, or you stopped for a second and thought “wait, if the ball is a dime and the bat is a dollar, then the bat would only be ninety cents more than the ball.”

Thinking Fast and Slow

This is exactly what Daniel Kahneman calls “Thinking Fast and Slow.” The “fast” part of your brain is responsible for making snap decisions so that you can make quick reactions to danger. The “slow” part of your brain is responsible for deliberative thought and logic. It’s job is to test the assumptions of your fast brain and ponder alternatives.

If your slow brain is tired, or just doesn’t want to waste mental energy, it accepts the fast brain’s conclusion. That is, if your intuition seems reasonable and the consequence of being wrong is small, you don’t bother to second guess your assumptions. That is, unless you put in the effort.

Faulty Conclusions

I recently saw a post that claimed that the oil tax reform of 2013 was a failure because production rates on the North Slope are lower today than they were when the bill was passed. While technically correct, accepting this intuition as true doesn’t hold up as soon as you give it a little thought.

Before I explain why, let’s explore other examples of the same logic:

1. When I was a kid, a candy bar would cost me $0.25. Today, the same candy bar costs $1. The candy bar company is therefore making $0.75 per candy bar more profit off of the kids today than they did off me.
2. I sent my 12-year-old son to spend the summer with his grandma. When he came back, he was two inches taller than when he left. Therefore, something my mom fed him made him grow.
3. I installed a new water heater in March. Since then, my electric bill has been about $100 a month lower than before I got it installed. Therefore, the new water heater is saving me $100 per month.

Now, any of these conclusions may sound reasonable. But most of us have learned to challenge these conclusions without making adjustments for inflation, puberty, and seasonality.

Rational Adjustments

We would want to know how much the costs of making candy bars has changed before we accept the conclusion that profits have soared. We would want to know how much the kid would have grown at home, before concluding that grandma did something special. And we would want to compare our electric bill to last summer, rather than last month, before we ascribe the savings to the change.

All of these things seem intuitive to us. We reject the simple explanation (the sound bytes) and activate our slow brain to make sense out of what is true and what just sounds good.

Normalization

In my field, we call this process “normalizing the data.” That just means we want to make those adjustments for things that change naturally over time before we make comparisons. We can’t say that a medicine is effective unless we compare it to a control group. We can’t say if a business is more profitable until we adjust for inflation. And we can’t say if a policy change had an impact unless we know what would have happened in the absence of the change.

In other words, if you want to speak to whether or not the tax reform was effective, you can’t compare production in 2012 to production in 2018. You have to compare production in 2018 under the tax change to the production in 2018 without the tax change.

Drawing Conclusions

Is that something we can do? Well, no.

We cannot observe the counterfactual universe in which the tax bill did not pass. So, the best we can do is make educated guesses. And these guesses cannot be scientifically valid. They can only be logically consistent.

That said, here is the data. It is painfully obvious that something changed in 2015. If the slope followed the same path, there would have been 25 million less barrels of oil produced in 2017.

Looking at it another way, here are the historic rates of decline since production decline began (a natural consequence of removing pressure from the reservoir). We know that the flat production in the early 2000’s was due to the addition of the Alpine field. You’ll have to draw your own conclusions about what happened over the last 2 years.