Anyone currently employed in any area of the IT business will be aware, however reluctantly, of the considerable amount of effort being put into marketing ‘Big Data’. During August and September of 2012 Freeform Dynamics surveyed 502 readers of The Register (for sample breakdown see here) to gauge how far organisations have advanced in their adoption of Big Data solutions and the current state of play concerning business analytics. This article looks at where organisations report they currently hold business data, how well they exploit the value locked in their data stores and their thoughts on how things may develop in the near future.
Today the words ‘Big Data’ are currently used and abused in a myriad of ways. From these it is possible to distil the term to be shorthand to refer to a number of advanced data storage, access and analytics technologies aimed at handling high volume and/or fast moving data in a variety of scenarios. These typically involve low signal-to-noise ratios, including but not limited to brand monitoring, log file analysis, high volume transaction monitoring for fraud detection, to name just a few.
Data volumes are growing
It is no news that the amount of data being generated by organisations continues to grow rapidly and this survey illustrates just how much of a challenge this is with over half of respondents reporting that they are experiencing high, or extremely high, rates of data growth and very few experiencing no growth at all (Figure 1).
Figure 1
Of considerably more importance with the vendor focus firmly directed at “Big Data”, is the fact that whilst the expansion of volumes associated with unstructured data sources such as email, office productivity documents, video files etc. is very strong, the reports of growth in structured data sources, such as those associated with relational database systems, is not that much lower.
This begs the question, just how valuable or important are the types of information held in structured sources compared with unstructured repositories?
The value of data
It is evident that the majority of organisations today still keep most of their business critical information in structured repositories, typically relational database management systems (RDBMS). Far fewer hold such data in unstructured sources (Figure 2).
Figure 2
That said, it is worth noting that nearly one in ten of the respondents indicate that they are not sure how such data is held. It is easy to understand why this should be so when the question of data storage architecture has never been an area of intense study for most IT professionals and has never even been thought about by the vast majority of business users.
When asked how things are changing, the research indicates that roughly equal numbers of respondents expect there to be a shift towards either structured or unstructured sources as the repositories for business critical data. That said, the most frequently offered opinion is that there is likely to be little change evident in the near future. This illustrates that Big Data will not be sweeping away the very strong position that relational databases and other established solutions hold in the near term.
With so much data out there just how well are organisations exploiting what they already store?
Data value is still ‘underexploited’
The Reg readers who responded to the survey tell us that only rarely is the data their organisations hold exploited as well as it could be. Even a cursory glance at Figure 3 shows very clearly that only around one in eight organisations fully exploit their structured data whilst an even smaller fraction does so for unstructured data.
Figure 3
Tony is an IT operations guru. As an ex-IT manager with an insatiable thirst for knowledge, his extensive vendor briefing agenda makes him one of the most well informed analysts in the industry, particularly on the diversity of solutions and approaches available to tackle key operational requirements. If you are a vendor talking about a new offering, be very careful about describing it to Tony as ‘unique’, because if it isn’t, he’ll probably know.
Have You Read This?
From Barcode Scanning to Smart Data Capture
Beyond the Barcode: Smart Data Capture
The Evolving Role of Converged Infrastructure in Modern IT
Evaluating the Potential of Hyper-Converged Storage
Kubernetes as an enterprise multi-cloud enabler
A CX perspective on the Contact Centre
Automation of SAP Master Data Management
Tackling the software skills crunch