Years of thinking social archives >
For all of the hullabaloo about BIG DATA (so BIG it requires capital letters), the tangible examples remain ghost-like and hard to grip. Yes, I am sure that all the wonderful data-crunching recommendations I receive when logged into Amazon are valid examples. I even believe the data collection, mashing and outputs that drive Progressive’s Snapshot (collects data on how you and 1mm other people drive to build a personalized insurance rate) is a decent example of BIG DATA applied.
Still, we need more examples to better understand its potential and what are the best practices of applying BIG DATA to business. HBR has a section online dedicated to just this. It features stories on predicting movie blockbusters via data and…well, not too many more examples actually.
A story in the NY Times this past weekend highlighted the use of data to derive insights from literature.
A study from the University of Nebraska used technology to crunch 3600 19th century literary novels to reveal that, among other things, Jane Austen and Sir Walter Scott are the proverbial ‘trail head’ of influence on other writers.
“The study, which involved statistical parsing and aggregation of thousands of novels, made other striking observations. For example, Austen’s works cluster tightly together in style and theme, while those of George Eliot (a k a Mary Ann Evans) range more broadly, and more closely resemble the patterns of male writers. Using similar criteria, Harriet Beecher Stowe was 20 years ahead of her time, said Mr. Jockers, whose research will soon be published in a book, “Macroanalysis: Digital Methods and Literary History” (University of Illinois Press).”
Big Data People
My vision of BIG DATA applied is not some automated process where if you simply combine the right massive feeds of data, even crunching them via some mind-bending algorithm, that actionable meaning shoots out the other side. If I simply look at my own world of digital intelligence derived form behavior and content online, it is all about the human analysis that converts it into meaning.
“Quantitative tools in the humanities and the social sciences, as in other fields, are most powerful when they are controlled by an intelligent human. Experts with deep knowledge of a subject are needed to ask the right questions and to recognize the shortcomings of statistical models.
“You’ll always need both,” says Mr. Jockers, the literary quant. “But we’re at a moment now when there is much greater acceptance of these methods than in the past. There will come a time when this kind of analysis is just part of the tool kit in the humanities, as in every other discipline.””
We need to be growing new hybrid professionals who understand data analysis but understand other things as well. Just look at assistant professor Matthew Jockers. His qualifications span two unlikely worlds.
“Mr. Jockers, 46, personifies the digital advance in the humanities. He received a Ph.D. in English literature from Southern Illinois University, but was also fascinated by computing and became a self-taught programmer. Before he moved to the University of Nebraska last year, he spent more than a decade at Stanford, where he was a founder of the Stanford Literary Lab, which is dedicated to the digital exploration of books.“
These guys and gals won’t graduate out of some “BIG DATA meets marketing” program, to focus on my industry as an example. We need to train and grow these experts now by investing, in my industry’s case, in “Digital Intelligence Teams” who are working with data sources everyday and applying them to real work. This applied innovation is where new, valuable capabilities will be born. It is also where the BIG DATA People will come from.
(Image: Thank you Brett Shollenberger for the perfect pic.)