“Science relies on the careful collection and analysis of facts. Science also benefits from human judgment, but that intuition isn't necessarily reliable. A study finds that scientists did a poor job forecasting whether a successful experiment would work on a second try,” writes Richard Harris in their recent NPR article entitled “Scientists Are Not So Hot At Predicting Which Cancer Studies Will Succeed.”
According to Harris’s article, “That matters, because scientists can waste a lot of time if they read the results from another lab and eagerly chase after bum leads.”
"There are lots of different candidates for drugs you might develop or different for research programs you might want to invest in. What you want is a way to discriminate between those investments that are going to pay off down the road, and those that are just going to fizzle," says Jonathan Kimmelman, an associate professor of biomedical ethics at McGill University in Montreal, in the NPR piece.
“Kimmelman has been studying scientific forecasting for that reason. He realized he had a unique opening when other researchers announced a multi-million dollar project to replicate dozens of high-profile cancer experiments. It's called the Reproducibility Project: Cancer Biology. Organizers have written down the exact protocols they would be using and promised not to deviate,” Harris continues.
Harris notes in their NPR article, "This was really an extraordinary opportunity,’ according to Kimmelman because so often scientists change their experiment as they go along, so it's hard to know whether a poor forecast was simply because the experiment had changed along the way.”
Now, while you’re not forecasting results of cancer studies, we found a lesson to be learned here.
Do you see it?
Keep reading; we’re going to explain the number one lesson you can gain from the current flaws of scientific forecasting.
How Did We Get This Result?
It’s the start of your day.
You grab some coffee around 7 AM and head to your office.
You fall into your desk chair and turn on your computer.
You log into your analytics tool and begin scanning through yesterday’s data. “What’s this?” you ask yourself.
It appears you’ve found some interesting data. Your response times for yesterday averaged at 30 seconds faster than usual.
The number of guests on your property didn’t fluctuate in any way that’s out of the normal or different from any other business day.
You’re stunned. You’re confused.
“How did we get this result?” you ask yourself.
You review your personnel roster, call up your managers to verify the figures you have in front of you, and realize there are no anomalies.
But, there’s a problem here you can’t see.
It's a problem that’s affecting your ability to understand how your team achieved these remarkable numbers.
Here’s your problem: your staff deviates from protocol – and no documentation exists to show the deviation.
You need to fix this.
Only Occasional Success Is Present Without Discipline
Kudos to your employees for knocking their response times out of the park. But, shame on them for deviating from your process.
As a leader, you must realize the importance of long-term optimization over short-term successes.
One win without the discipline to protocol puts your operation in the position of just that – one win.
Is this what you want for your team? Of course not.
Yes, your operation is different than scientific research, but its lesson shows the importance of sticking to your established protocols.
When you remain disciplined, you can make formulated adjustments over time to get reliable results day after day.
Your success is determined by your ability to be consistent when analyzing your operation.
Deviating eliminates that ability.
A win here and there will improve morale momentarily, but the adverse effects can cripple your operation.
Here’s what we recommend you do:
- Read this article here on Proactive Operations
- Create a strategy that you’re willing to modify on an established basis, by you and your team
- Build your infrastructure in way that permits the efficient flow of information for easy review of strengths and weaknesses
- Deploy a unified solution that lets you enter your new protocols, so nothing slips through the cracks and staff sticks to the plan
- Maximize your unified solution by capturing data, analyzing it with confidence, and using reliable results to optimize performance every day
- Don’t ever deviate from protocol until agreed changes are made by you and your team
Over to You
We get it; your operation is far different from scientific forecasting and cancer studies. But, the takeaway here is the importance of adhering to your protocols.
Your staff must remain disciplined with current protocols, so you have the ability to get reliable results. The only way you can maximize your performance forever is with data that supports it.
You don’t want unreliable data to cripple your operation, right? Then, don’t deviate.