Unstructured Data is Dead – Now A Question

What is the answer to the ultimate question of life the universe and everything. 42 of course. Anyone who has read (or seen) Douglas Adams’ Hitchhikers Guide to Galaxy knows this. After finding this answer the ancient alien race then had to build a machine to run billions of years to determine something more important. What is the question?

I suggested before that AI will kill unstructured data as a useful distinction. AI opens the possibility that aspects of our dependence on orderly data may be obsolete with the power of machine learning and AI’s ability to analyze data in new ways.

With all of the hype you may be asking “what can I do with AI in my business.” More specifically how can AI make money. Among technologists sometimes money is an ugly word. We often replace it with words like “value.” Secretly we worry that our lofty ideas and cool tools will be held to a metric we cannot prove. Nevertheless, the output of the AI exercise should be driving a successful business venture(profit) or bettering someone’s condition in life or work.

It is natural that as technologists working large content problems, we would look to AI as a pathway to a better bottom line. We have difficulty though easily identifying those efforts that are reliably worth the effort. Content management is a decidedly mature space. Some would argue the ROI well seems to have been running dry for a decade. In past days there were simple and painfully obvious ECM opportunities that were easy to justify and guided by a single principle. Digital is better than physical. But what do you do after (most) everything is digitized, workflowed, recordized and archived. Those great pictures in paperless office presentations of an overburdened office worker buried in an avalanche of TPS reports do not have the same impact they once did. All that paper has turned to tiffs in the cloud or lays buried in a data center. So easy to ignore.

Fracking as you probably know is a controversial practice in the oil industry. The process pumps fluids into shale formations, forcing crude oil from the depths in places where wells had previously run dry. This unlocks enormous and previously unreachable oil resources. Access to these reserves has changed the dynamics of the global oil market and possibly caused a lot of earthquakes, but I digress.

Your data is the most basic natural resource available to you. For years now we have been seeking to improve the production and management of this resource but as suggested, the perpetual ROI machine of document management runs out of steam as businesses convert the largest and most troublesome physical categories into pure electronic forms. You are then left with immense content reserves in your organization and no good way to drive more value from them.

Artificial intelligence is the infrastructure that will allow us to do it , but what is a medium that we can inject into the data shale to force out value. The simplest of things really.

We talk of terms like algorithms in growing careers such as data science but we need to return to  a very basic skill that we are all capable of. Asking questions.

Questions about data. Questions about our business. Questions that surface what we don’t know by challenging what we do.

Surprisingly few people are really good at this. Managers often look for answers handed to them because that is exactly what the marketing promised.  Customers are told they need this magical thing called “insight” to make better decisions. They are promised tools will provide this fairy dust but what they don’t tell you is that before you can mine for insight you must first define interest.

It is called curiosity.

Insight about things no one cares about or that do not make a difference is a waste of time and just as expensive as meaningful information. Asking good questions about business that can be answered by bringing together enormous datasets we could never analyze before and apply algorithms to predict an outcome – THAT is the skill we need to develop. It is much more than learning R or Python.

I have known some people who were very adept at finding the holy water that forced real insight out of shale data but how does one acquire this skill?  Can it even be learned? A person that is exceptionally good at deciding what questions to ask is immeasurably valuable to this growing opportunity and they may not be who you think.

They are not necessarily the uber coder that never takes off his 2014 Comi-Con badge. They are probably not the well coiffed consultant that loses his short game every time he golfs with a director or higher.  They are possibly the business user, more interested in Instagram than SAP, that wonders why shipments to some zip codes have higher return rates. In this skill, simple curiosity is more important than knowing methodologies. The technical skills are a necessary and important component of the process but they are also much easier to acquire. Data science curriculums are rightly focused on the technology but the discipline of drawing out meaning is underrepresented.

And then there are those that don’t know the questions themselves but are skilled at working with the end users to draw out their questions. The facilitators. They may be even more rare.

If data is the new oil, unstructured content then is the data shale where enormous wealth is locked away. Extracting this value just might cause earthquakes in business too. What are you doing to learn to do it or identify those with the natural talent to ask questions essential to pulling money out of the ground? Without good questions, this very expensive AI machinery might just pump out nothing but hot air.