Sunday, March 24, 2019

Learn to embrace uncertainty

In graduate school, it seemed like the faculty expected me to go the quantitative route, because of my background in electrical engineering. Statistical modeling and, more importantly, GIS, which was in its infancy back then.

I couldn't care about those data-based inquiries.

Because, I was convinced that the human condition that I was interested in learning and thinking about was beyond mere data.  Further, I even made fun of how the data were used in the publish-or-perish academic culture: Faculty and students were more often than not playing with data and seeing if anything would come out of it, instead of thinking through the issues and then using the data.

As with everything else, I chose the losing route.

Graduate students who specialized in data-based research techniques and GIS moved on to good jobs in academia and outside.  I struggled to find a job!

I did continue to criticize the publish-or-perish culture that encouraged fooling around with data and statistical software, which was getting to be more and more sophisticated.  All these were before "big data."

I now feel vindicated.  Phew!

Pissed off scientists have risen up in their opposition to the abuse of statistical significance:

Source

The authors--"more than 800 signatories"--want researchers to quit categorizing:
The trouble is human and cognitive more than it is statistical: bucketing results into ‘statistically significant’ and ‘statistically non-significant’ makes people think that the items assigned in that way are categorically different ...
On top of this, the rigid focus on statistical significance encourages researchers to choose data and methods that yield statistical significance for some desired (or simply publishable) result, or that yield statistical non-significance for an undesired result, such as potential side effects of drugs — thereby invalidating conclusions.
Note how they phrased it? "encourages researchers to choose data and methods that yield statistical significance for some desired (or simply publishable) result" is exactly what I did not find appealing back in graduate school!

More from the authors:
The objection we hear most against retiring statistical significance is that it is needed to make yes-or-no decisions. But for the choices often required in regulatory, policy and business environments, decisions based on the costs, benefits and likelihoods of all potential consequences always beat those made based solely on statistical significance.
Exactly!

And even more important is this point from the authors on what will happen if we got rid of our focus on statistical significance:
Decisions to interpret or to publish results will not be based on statistical thresholds. People will spend less time with statistical software, and more time thinking.
More time thinking.  What a concept!

No comments: