Data collection is an integral part of any business, and it has become even more imperative in the Internet Age. However, the reality is that most of the data that is collected and stored is never proper analyzed or utilized. In fact, Dave Picciano, who is IBM’s senior VP of analytics, estimates that 90 percent of the data that is gathered from A-D conversions and most sensors is never even looked at. When this happens, the collected information falls under the heading of dark data.
Picciano did elaborate by indicating that as much as 60 percent of this dark data becomes devalued within a few seconds, but this still leaves a lot of untapped information that could provide your startup with a big boost.
Mining Leads to Opportunities
Gartner Research is generally credited with coining the phrase “dark data,” and they define it as data that is collected, processed and stored without being used for further purposes such as enhancing business relationships and analytics. In other words, the 40 percent of data Picciano referenced that retains its value for more than a few seconds absolutely should be mined to open up additional opportunities. Imagine for a moment what you could learn about your customer base if you truly took in all of the data they freely provide by browsing your site. Examples include building a better buyer profile and tweaking your SEO to attract higher quality leads.
Retain Dark Data for Legal Protection
According to Necto Panorama Software, it is legally necessary for organizations and businesses to retain dark data even if they are not planning to do anything with it. This helps explain the legal scandals that have been caused by politicians and businesses that did purge data, ranging from emails to basic tracking information. However, it is unclear exactly how long each company needs to do this and whether or not this drastically changes based on the nature of the data.
This confusion is similar to the inconsistent data storage guidelines that are followed by mobile phone providers. Some of these companies retain phone and text records for 12 months in case law enforcement officials end up requesting them, but others can provide detailed information going back as far as seven years. With this in mind, it is probably best for even the smallest startup to store dark data on their servers for a minimum of one year.
Organized Data = Valuable Insight
Of course, an even better course of action would be to separate the dark data that becomes almost instantaneously irrelevant from the 40 percent that offers potentially vitally insight. By doing this, your startup can begin learning valuable details that could change the way you connect with consumers.
Supercomputing’s Role in Data Mining
IBM has showcased the potential future of sorting through dark data with their Watson supercomputer, even if this did happen inadvertently. Some experts believe that this type of supercomputing may be the only truly viable way to deal with dark data in the future because it will be much easier for these machines to quickly analyze the information and extract anything that may be useful. From there, it is likely that the most pertinent data would be logged in a manner that can be accessed by business intelligence leaders.
The Power of Harnessed Dark Data
In the meantime, every company is losing valuable insight by not making more of an effort to tap into the dark data that they are constantly collecting. Each startup needs to be aware of this problem so that they can look for better ways to sort this data and put it to good use. Perhaps a future startup will even be launched with the sole intention of helping others more effectively pull the most useful pieces of information out of their stored dark data.
In a marketplace that is driven by harnessing the power of tons of small and large consumer actions and buying signals, it quite simply makes good sense to tap into every possible resource that can help you attract your target audience. Even if you are able to crack the puzzle of only five percent of your currently unutilized dark data, this has the potential to give you a nice boost.