If there's any agreement about big data, it's that so much is coming in so quickly from many sources in many formats....
The speed at which it all needs to be processed, stored and analyzed is simply more than most corporate IT budgets, staffs and infrastructures are able or willing to handle. We are drowning in data, yet often find ourselves starved for information. For an increasing number of companies, getting a grip on the situation means making it someone else's problem. That someone is a big data as a service (BDaaS) or a data as a service (DaaS) provider.
Regardless of how a service is configured and delivered, discussion of DaaS focuses as much on analytics as it does on data collection, presenting opportunities and challenges to the development side.
"For architects and developers, cloud-based big data offerings are a way to accelerate the time to build analytics applications," said Nik Rouda, an analyst focused on big data and analytics at market research company Enterprise Strategy Group. "Without having to wait for IT infrastructure and operations teams to provision resources, developers can start immediately on prototyping and then easily roll the new tools into production when ready."
With the rise in cloud and mobility, business priorities have become crystal-clear: Grow revenue and transform the customer experience while reducing costs. Each requires architects and developers to balance traditional values, such as security and cost effectiveness, with the need for speed and agility.
"Architects must figure out how to accommodate the sky-high expectations of digital executives who have seen the capabilities of analytics in the cloud and yet do not understand why it is so difficult to integrate with enterprise systems," said Brian Hopkins, an analyst for Forrester Research. "Excuses and finger pointing won't work; those that fail will become irrelevant. This makes emerging Agile, DevOps and data science practices … a critical part of the emerging digital architecture."
Nik RoudaAnalyst, Enterprise Strategy Group
It's clear that every industry is looking to do more with its data to stay ahead of competitors. Moving data to the cloud makes access and analysis easier for everyone. "Your customers, employees and apps all live in the cloud, so that's where your data needs to be," Rouda said. "It's natural to bring the analytics to the data; you don't bring the data to the analytics. BDaaS, or DaaS, [is] particularly good at doing this."
Jim Comfort, general manager of cloud services at IBM, agreed. Analytics is a key driver for turning to DaaS. "It's one thing to simply store data in the cloud, but it's the analytics in the cloud that make data useful. If you need one, two or 20 different analytic approaches, you can easily do all of that with the flexibility and agility that a cloud services environment offers," he said.
The numbers back up Rouda's and Comfort's assertions. Data management, typified by the migration of databases from on-premises storage into the cloud, is the top IT priority for this year among 26% of organizations polled by the Enterprise Strategy Group for its 2015 IT Spending Intentions Survey. That ranks second only to security initiatives, which was cited by 34%. In that same group, 66% plan to boost spending on cloud services in 2015 compared with last year.
Forrester's Hopkins takes an alternative view. "The truth is that the data on which you do your analytics is usually reasonably sized, only a small subset." Move just that to the cloud and use DaaS to do the analytics there, he said. "Big data in the cloud is not yet affordable; it's better to keep years and years of historical data on-premises in a hybrid configuration."
Regardless of where data resides, there is little doubt that IT is finding it increasingly difficult to keep up with demand. That's not surprising, given the pace at which data is created. In 2013, Norway's The Foundation for Scientific and Industrial Research published a widely quoted study that found 90% of the world's data had been created in the last two years. In 2015, it's not unreasonable to surmise the percentage has edged higher. IBM itself says that 2.5 quintillion bytes of data are created every day.
Data warehouse providers can't live up to their former promises anymore. "Those infrastructures lacked agility, and you needed to declare everything upfront, including the [database] schema and the amount of data to be stored," Comfort said. That's no longer good enough. "An instantly scalable, quickly implemented, cost-effective DaaS is the answer," he added.
Learn more about taking control of BDaaS.
Learn about tackling big data challenges with Hadoop data lakes.