Ron, I agree that not much attention has been paid to classical statistical methods when big data sets are analyzed and results are presented. There was certainly an awareness of the big data problem (<100 patients with 10,000+ variables) as far back as the early 1990s [Benjamini Y and Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Statist Soc B 1995; 57:289-300]; however, it required a decade or so before this technique was frequently referenced in big data publications. Similarly, the problem of variable selection from many variables was also addressed in the 1990s [Tibshirani R. The lasso method for variable selection in the Cox model. Stat Med 1997; 16:285-395], with several attempts to generalize the technique over the years [Tibshirani R. Regression shrinkage and selection via the lasso: a retrospective. J R Statist Soc 2011; 73 (Part 3):273-292].

    Very few of my statistical colleagues (myself included) feel comfortable designing big data studies or analyzing data from these studies. Instead, we tend to refer researchers to individuals/groups that specialize in bioinformatics. There are now programs that offer advanced degrees in data analytics with a focus on big data (see, for example, Penn State’s program at http://www.worldcampus.psu.edu/degrees-and-certificates/data-analytics-base/overview). They do offer some introductory statistics modules, but nothing very advanced.

A conference like you suggest would probably be quite popular. It should include individuals with expertise in several areas, including people who set up and run the assays that produce the multidimensional data, people who pre-process the data (eg, hybridization normalization, central tendency normalization, calibration), and people who build models (using data reduction / variable selection techniques) and validate their biological and clinical utility.

Gary Clark     Sep 26, 2016

Dear Ronald,

Thanks for copying us into this. As you might expect, discussions about big data are rife amongst statistical associations at the moment, so I’m sure you are onto something.

Best,

Hetan Shah     Sep 26, 2016

Dear Ron,

Thank you for soliciting my opinion.  I have to confess first that I know little about big data (certainly heard a lot of it) as this is not my day job.  We only deal with ‘small data’ in contrast to the big data. 

If your big data problem is related to the statistical books collection for the Library of Alexandria, then the first question is what is the objective of the research and what you want to achieve.  The quote at the end of your first email:

"Big data analytics is the process of examining large datasets to uncover hidden patterns, unknown correlations, market trends, customer preferences and other useful business information"

Is this your objective?  If yes, then a relative modern discipline of data mining seems to be the right approach.  Do we need classical statistical methods?  It depends.  Many classical statistical methods evolve around hypothesis testing, mostly suitable for a modest amount of data.  When you have a lot of hypotheses to test, then the control of false discovery rate as mentioned by Gary Clark may be relevant.  Are there any hypotheses testing you have had in mind?  So it is really what is the right tool to address your question (and what is the question), it does not matter whether it is classical, Bayesian, or others.

Dick De Veaux of Williams College is one of the experts in data mining.

I think I exhausted my knowledge here.  I hope you get some practical advices from others.

Good luck with your next endeavor and kind regards,

 Kan Yee         Sep 26, 2016

Dear Ron,

Dear Dr. LaPorte,

I was fascinated by your Google-based analyses that you shared.  But something about the method seemed to invite a possible fallacy (or at least, room for confounding).   I took me a while to pin down what it was. 

The problem became clear from this search pair that behaved in a similar way to what you described:

    On Google Scholar, “Physics” gets about 5.4 million hits; “auto mechanics” gets about 16,000 hits; but “Physics AND “auto mechanics”” gets only 3,000 hits—implying only a very small overlap. 

 ….But does it follow that there “is very little linkage” between these two fields?   Surely, Forces, Torque, Acceleration, and Friction are just a few very key areas of overlap between auto mechanics and physics.   Indeed, the authors of a paper “Auto mechanics in the physics lab: Science education for all”, John W. Tillotson and Paula Kluth, lament that this overlap is not used explicitly to bridge the stereotypical gap between “academic”- and “trades”-oriented  streams in high school curricula. 

What this suggests is that absence of Google overlaps tells a lot about the “silos” from which different writers may think about and contribute on certain subjects; but it is not evidence of there being no inherent relation between the subjects.   It may just need champions to get out the message. 

As this relates to your letter, may I recommend that you clarify your objective in asking your question.  You have certainly shown that, as a historical fact, not a lot of people write about these (related) topics outside the ‘silo’ or framework of their own particular interest.   But if you are asking, is there inherent relation between the two topics “statistics” and “big data”, which people should be keeping in mind, the Google data neither confirm or refute a connection.

Does this make sense?

Bill Goodman   7 Oct 2016 

Dear Ron,

  I believe there are probably several types of differences between “Statistics” and “Big Data”.   In many cases, “Big Data” deals with the data from entire population (e.g. all users of Google say), while classical statistics typically deals with a representative sample (subset) of the population.  This can lead to differences in keys areas such as “experimental design” and “statistical inference”. 

  With experimental design we carefully plan how we are going to sample our data (to make inferences about our population). 

  With statistical inference, we often use special models that involve random variables and probability concepts can that we can compute probabilities of making an incorrect decision. 

Many practitioners of “Big Data” however, use clever algorithms to find structure (e.g. trends, clusters, etc.) in the data.  Since many such practitioners of “Big Data” are working with the entire population, they seem less concerned about the classical statistics issues of “experimental design” and “statistical inference”. 

  However, with changing technology, more and more situations are arising where scientific experiments (e.g. genetic experiments, high-dimensional measurements made with spectroscopy devices, etc.) are themselves generating “Big data”.  It is my belief that these particular “Big Data” situations still require good experimental design and statistical inference.    

You might find the review paper “Statistical Modeling: The Two Cultures” by Leo Breiman (with Discussions from several leading statisticians) interesting as it touches upon the subtle but important issue of classical statistical modeling vs. the algorithmic approaches now popular in “Big Data” practice.   See http://projecteuclid.org/euclid.ss/1009213726    Best regards,

As human it is well documented that our thinking embraces the physical world that we perceive through our senses (expanded through technology), as well as, the conceptual world that is associated with language, culture and our self-awareness about some of the mental paradigms we use (expanded through IT).

When exploring Ron LaPorte’s question about possible distinctions between “classic statistics” and “big analytics” my interpretation is that before the exponential development of IT in the last 50 years, looking at the physical world as constituted by “a collection of parts”, was making sense with “classic statistics”. Today with the “Internet of Things” and the “growing interconnections between things and humans” the resulting exploding complexity creates for our environment new attributes more linked to the conceptual world than the physical world. The study of these new attributes requires “Big Analytics” therefore I propose the below distinctions:

Francois Sauer   October 9, 2016