FORRESTER blog repost Advanced analytics predictions for 2010

Advanced analytics predictions for 2010

By James Kobielus

As we bid adieu to one decade and move into the next, it’s important to catch our collective breath and to take a quick look ahead. Here are some quick thoughts on the trends that will shape advanced analytics in the year to come. These trends will set the stage for thoroughgoing transformation of business intelligence (BI), data warehousing (DW), predictive analytics (PA), data mining (DM), business activity monitoring (BAM), complex event processing (CEP), and other key analytics technologies in the Teens:

* Self-service operational BI puts information workers in driver’s seat: Enterprises have begun to adopt self-service BI to cut costs, unclog the analytics development backlog, and improve the velocity of practical insights. Users are demanding tools to do interactive, deeply dimensional exploration of information pulled from enterprise data warehouses, data marts, transactional applications, and other systems. In 2010, users will flock to self-service BI offerings as the soft economy keeps pressure on IT budgets. Also fueling this trend is the increasing frustration that information workers feel in the face of long backlogs on seemingly mundane BI service requests. In the coming year, BI software as a service (SaaS) subscription offerings will be particularly popular, in a market that has already become fiercely competitive. So will the new generation of BI mashup offerings for premises-based deployment, especially mashup-oriented BI tools from IBM Cognos and Microsoft.

* User-friendly predictive modeling comes to the information workplace: Predictive analytics can play a pivotal role in day-to-day business operations. If available to information workers—not just to Ph.D. statisticians and professional data miners—predictive modeling tools can help business people continually tweak their plans based on flexible what-if analyses and forecasts that leverage both deep historical data as well as fresh streams of current event data. In 2010, user-friendly predictive modeling tools will increasingly come to market, either as stand-alone offerings or as embedded features of companies’ BI environments. Many BI vendors will add predictive modeling to their current offerings—most notably, IBM will converge its Cognos BI and new SPSS data mining offerings—with a focus on mass-market usability. By the same token, established predictive modeling vendors such as SAS, IBM SPSS, KXEN, Angoss, and Portrait Software will highlight and deepen their existing usability features, such as wizard-driven automation and interactive visualization, to speed information workers through the complex steps for building, validating, and exploring predictive models. Just as significant, in-memory BI clients—such as those from TIBCO Spotfire and QlikTech--provide an important alternative to traditional data mining tools for subject matter experts who wish to explore a multivariate data set from all angles without having to do heavy-hitting data preparation, clustering, and classification beforehand.

* Advanced analytics sinks deep roots in the data warehouse: Advanced analytics demands a high-performance data management infrastructure to handle data integration, statistical analysis, and other compute-intensive functions. In-database analytics is an emerging practice under which those and other resource-intensive processes can be parallelized and thereby accelerated across one or more data warehousing nodes. In-database analytics enables flexible deployment of a wide range of resource-intensive functions—such as data mining and predictive modeling—to a cluster, grid, or cloud of high-performance analytic databases. In 2010, in-database analytics will become a new best practice for data mining and content analytics, in which the enterprise data warehousing professionals must now collaborate closely with the subject matter experts who build and maintain predictive models. To support heterogeneous interoperability for in-database and in-cloud analytics, open development frameworks-- especially MapReduce and Hadoop—will be adopted broadly by data warehousing and analytics tools vendors. In the coming year, we’ll also see the beginning of an industry push toward an open development framework for inline predictive models that can be deployed to CEP environments. Already, IBM and TIBCO have developed interesting, albeit proprietary, support for in-CEP predictive analytics. Clearly, in-CEP predictive analytics will be a critical component of truly adaptive BAM for process analytics.

* Social network analysis bring powerful predictive analysis to the online economy: Before long, social networks will pervade all business and personal applications, including all mobile, broadband, and streaming media services. From an enterprise perspective, social networks are the buzz that can spell the difference between success and failure in a reputation-driven online economy. In 2010, enterprises will avidly adopt social network monitoring and marketing tools, while deploying advanced analytics to search for opportunities to better reach customers in these environments. Forrester sees 2010 as the year social network analysis truly emerges as the new frontier in advanced analytics, supporting mining of behavioral, attititudinal, and other affinities among individuals. Social network analysis thrives on the deepening streams of information—structured and unstructured, user-generated and automated—that emanate from Facebook, Twitter, and other new Web 2.0 communities. In the coming year, many vendors of predictive modeling tools will enhance their social network analysis features to support real-time customer segmentation, target marketing, churn analysis, and anti-fraud. The killer app for all this will become the real-time “next best offer” that your contact center makes from all this intelligence, or the marketing campaign you re-arrange on the fly to save it from near-failure.

* Low-cost data warehousing delivers fast analytics to the midmarket: Though enterprises can certainly do BI without a data warehouse, this critical infrastructure platform is essential for high-performance reporting, query, and analytics against large data sets. In one of the most important BI trends of the past several years, the price of a fully configured data warehousing appliance platform has dropped by an order of magnitude and, with the development of public SaaS DW cloud services, it will continue to decline. In 2010, many data warehousing vendors will lower the price of their basic appliance products to less than $20,000 per usable terabyte, which constitutes the new industry threshold pioneered by Oracle, Netezza, and other leading DW vendors. At the same time, enterprises will see a growing range of cost-effective solution appliances in 2010, combined DW appliances with preconfigured BI, advanced analytics, data cleansing, industry information models, and other data management applications and tools.

* Data warehousing virtualizing into the cloud: The data warehouse, like all other components of the BI and data management infrastructure, is entering the cloud. In 2010, we’ll see vendors continue to introduce cloud, SaaS, and virtualized deployments of their core analytic databases. To support flexible mixed-workload analytics, the EDW, over the coming 5-10 years, will evolve into a virtualized cloud that allows data to be transparently persisted in diverse physical and logical formats to an abstract, seamless grid of interconnected memory and disk resources that can support diverse workloads, latencies, and topologies. Massive parallelism, all-in-memory architectures, solid-state drives, and virtualized storage will increasingly revolutionize EDW environments—both cloud- and appliance-based—over this coming decade. In 2010, we will see most EDW vendors roll out pioneering offerings that offer all of these architectural innovations. However, 2010 will not be the “year of the cloud” for the DW industry as a whole, since Teradata, Oracle, IBM, Microsoft, and others will still be rolling out their initial public/private cloud platform services and partnerships. Nevertheless, the industry is moving inevitably toward cloud-based services that supplement appliances, licensed software, and other deployment options.

In developing my 2010 research agenda, I have incorporated all of these themes. I look forward to your feedback, inputs, and suggestions as we move into the new year.