‘Databerg’ threatens to sink corporate ships, says study

Like a monster rising up against its master, the data that companies insist on collecting, measuring, collating and analyzing to the nth degree is having its revenge, according to Veritas, an international firm with roots in Israel that helps large organizations better understand and take advantage of their data.

The masses of data collected – far more than could ever be used, at least with current analytics technology – have coalesced to form what Veritas calls “the Databerg,” the virtual landfill that is the ultimate home of unneeded and unwanted data.

The good news for Israel is that, while substantial, the blue and white version of the Databerg is significantly smaller than those of many countries in Europe and the Middle East. The incidence of “Dark Data” and ROT (Redundant, Obsolete or Trivial) data – some of the technical terms associated with the Databerg – is about a third less than in the rest of the EMEA region.

As a result, said Veritas, Israeli firms have more “clean,” or usable data in their data stores – which means they have to spend less per byte to analyze information and get a logical plan of action out of their data.

It wasn’t supposed to be like this. The Big Data revolution – the collection of every scrap of data from every possible source – was supposed to make companies more efficient. By having access to metrics on every conceivable measurable piece of information, like at what times of day the company water cooler is accessed or how many employees order soup at the company cafeteria on rainy days, firms were supposed to be able to develop policies that could save them money.

For example, if X number of employees were taking water breaks at a certain time, for example, it might indicate that they were getting together for not just a drink, but a chat – so the company might get more productivity out of workers by distributing bottles of water around that time of day, keeping employees at their desks. And if the consumption of soup follows the weather patterns, cafeteria managers could hold off on one or two side dishes that are less likely to get eaten that day, saving the kitchen budget money and eliminating waste.

Time- and money-saving ideas are what Big Data does well, but the real question is how prepared is a firm that is already short-staffed (and won’t hire because it fears another recession) to track that data and implement those policies. When it comes to using data, said Veritas, companies perform a triage, using data to achieved their most important goals, while ignoring the rest.

The problem with that, said Veritas, is that with more data being collected, databases need to expand and hold all that information. But the large amount of unused data provides more opportunity for errors, more targets for hackers, and most importantly is seen as “moving corporate resources away from the direct line of sight of management teams,” with organizations adopting policies driven by data for data’s own sake, not for the sake of growth.

If that sounds like an unlikely scenario, the Veritas 2015 Databerg Report dispels the notion that more data means greater efficiency. The report is based on the results of a survey on how 1,475 respondents (including 50 in Israel) in 14 different countries across the EMEA region deal with the challenges surrounding turning data into valued business information.

Veritas classifies data in one of three categories, the most important being “clean” or Business Critical Data, vital to the ongoing operation and success of the organization. Business Critical data needs to be proactively protected and managed in real time by professionals with clear responsibility to the organization’s management team, as it used for critical policy making decisions, Veritas said.

Less useful – and far more problematic – is ROT data, the information that could be useful if anyone were to make use of it, but ends up just lying around unused, gumming up the rest of the database. Like junk accumulating in an attic – it’s there for a reason, but no one can remember why – ROT data needs to be proactively minimized and safely deleted on a regular basis, said Veritas.

Then there is the “Dark Data,” whose data value has not been identified. It may include vital business critical data, useless ROT data or even illegal or non-compliant data, leaving an unseen liability at the heart of corporate IT systems. Unlike ROT data, which is simply a nuisance, Dark Data needs to be kept off a company’s database altogether, as it can contribute to real gaffes in a business’s decision making – opening up the firm to financial, legal, or other losses and liabilities.

Taken together, the amount of useless or dangerous data in company servers is dangerously high. As little as 14% of data in EMEA companies could be considered mission critical, the report said, with a region average of 54% of ROT data and 32% of unidentified data on company servers. By 2020, the report added, companies could face losses of nearly a trillion dollars annually as a result of the flood of irrelevant or incorrect data – a flood that promises to become an ocean as even more data from more sources begins to pour in as the Internet of Things, with its insistence on recording everything there is to record, takes off.

Things in Israel were a bit better, said the report. Israeli companies scored far better than the EMEA average, with 24% of data in Israeli databases found to be useful, and Dark Data rates of 35% (EMEA avg. 54%) and ROT levels of 41% (EMEA avg. 32%).

Don’t firms realize the folly of filling up their systems with useless or dangerous data? No doubt, said Veritas – but perhaps similar to the old adage about building a baseball field, if the big data is there, it will be collected.

That’s an attitude that must change, said Gil Don, Country Manager in Israel at Veritas. “Data should deliver on its promise and work for the organization which is currently the case in Israel. However, the key findings of the Databerg Report 2015 reflect that companies still invest a significant amount of resources to maintain data totally irrelevant to their business.

“The study also reveals the majority of companies (65%) in Israel are basing their IT strategies and budgets on the volume of data stored and processed, rather than its value,” added Don. “This means they are spending time and money on managing data which is not vital for their business instead of re-investing it somewhere else.”