Story image

The data dilemma

01 Jan 11

Every modern company relies on a wide variety of systems, from a key-card reader to enter the front door, to the application server that the company website runs on. These systems are vital, constantly changing and almost every day they introduce new dynamics that impact the business technology ecosystem.
What most executives aren’t thinking about is that each of these systems creates a unique stream of data that contains a definitive record of who’s interacting with their company, what time they stopped by, when and what they’re looking for and any obstacles that prevented them from accomplishing their desired outcome. But all this data lives in different buckets – network logs, clickstream data, firewalls, OS logs, Java garbage collection stats. The list goes on. These disparate systems reside on different servers, sometimes in different geographies and have never traditionally been thought of to provide vital business information.
This is the IT data dilemma – making sense of the volumes of data generated by systems, new devices and transient and disparate sources. Today IT faces an increasingly difficult challenge managing all of its systems, much less seeing and understanding the relation of one set of data to another.
However, once you’re able to overcome these obstacles, you can begin to gain visibility across an organisation and discover the insights needed to optimise IT and the business. This is what we call operational intelligence – harnessing unstructured, machine-generated IT data so that IT and the business can make faster and more informed decisions. As companies start to discover the value of their IT data, they reveal a wealth of new insights that were previously unattainable.
It’s time for a massive shift in how we capture and analyse IT data, and this starts with understanding the nature of the challenge.
Harnessing IT Data: The 4 main obstacles
1. Exponential growth: We’re creating 1,200 exabytes of data this year, up from 150 exabytes in 2005. New devices, applications, and trends like cloud computing and virtualisation are adding to the data deluge and continue this explosive growth curve. Gartner estimates that 80-90% of this growth is represented by unstructured data. this explosive growth curve. Gartner estimates that 80-90% of this growth is represented by unstructured data.
2. Diversity of sources: Every new mobile device, application, wireless connection, browsing session, shopping cart, creates a new stream of data to be captured.
3. Lack of a standard interface: Analysing IT data has traditionally meant buying a specialised point tool to import data into a database and analysing it with a reporting solution – this simply isn’t feasible with the diversity and volume of data found in today’s IT infrastructures.
4. No historical context: Data is constantly being created and it’s constantly changing. Specific data can unlock a trend or pattern critical to the business; sometimes it’s only revealed when it’s analysed in a historical context. The vast majority of systems that monitor or manage IT data persist the data for a short time before discarding it. Once this happens the ability to apply a historical context, discover complex patterns and identify trends, is lost forever.
The traditional approach to managing data – defining a schema and organising analysis around a fixed set of known questions – is failing us. However, technology has started to catch up and we now have the opportunity to capture IT data and create new levels of operational visibility that enable better business decision making.
For the channel, the IT data dilemma presents an opportunity to deepen relationships with organisations that struggle with managing and monitoring IT. In addition, it opens a strategic conversation about IT data and how to harness it to provide IT and the business the insight to make better decisions. To do that, channel partners need to understand the elements of successfully utilising IT data.
Keys to successfully harnessing IT data:

  • Real-time and historical data – Some data springs to life immediately as it streams across a real-time monitoring dashboard. Some data is more obscure and doesn’t reveal its value until it’s put in historical context. A solution needs to capture everything – real-time and historical IT data – to make sure valuable context isn’t lost.

  • One solution, many uses – An IT data solution needs to address use cases across IT instead of working within disparate silos. The solution needs to be usable by everyone in IT and flexible enough to provide custom dashboards or views to the rest of the company.

  • Compatibility with any IT data source – A solution can’t be limited by the ability to analyse some IT data but not others; operational visibility is about leveraging all the IT data generated across an organisation.

  • Scalability – Keeping up with the exponential growth of data means that an IT data solution has to be fast so you can search thousands or billions of events in seconds; and it has to be scalable enough to handle both departmental data volumes and global, distributed deployments with terabytes of IT data a day.

For channel partners, the IT data dilemma creates an opportunity to be the trusted partner who helps their customers move from simply managing IT to creating operational intelligence for their organisations

MulteFire announces industrial IoT network specification
The specification aims to deliver robust wireless network capabilities for Industrial IoT and enterprises.
Google Cloud, Palo Alto Networks extend partnership
Google Cloud and Palo Alto Networks have extended their partnership to include more security features and customer support for all major public clouds.
DigiCert conquers Google's distrust of Symantec certs
“This could have been an extremely disruptive event to online commerce," comments DigiCert CEO John Merrill. 
Schneider Electric's bets for the 2019 data centre industry
From IT and telco merging to the renaissance of liquid cooling, here are the company's top predictions for the year ahead.
China to usurp Europe in becoming AI research world leader
A new study has found China is outpacing Europe and the US in terms of AI research output and growth.
Google says ‘circular economy’ needed for data centres
Google's Sustainability Officer believes major changes are critical in data centres to emulate the cyclical life of nature.
52mil users affected by Google+’s second data breach
Google+ APIs will be shut down within the next 90 days, and the consumer platform will be disabled in April 2019 instead of August 2019 as originally planned.
Ramping up security with next-gen firewalls
The classic firewall lacked the ability to distinguish between different kinds of web traffic.