FORRESTER blog repost IBM Goes Deeply Predictive, Announces Acquisition of SPSS

IBM Goes Deeply Predictive, Announces Acquisition of SPSS

By James Kobielus

IBM dropped a big bombshell at the start of any already action-packed day for the analyst community. At this moment, I’m sitting, along with several dozen of my peers from Forrester and other firms, at the IBM Smart Analytics System launch event in Hawthorne NY. I’ll blog on IBM’s other announcements in a separate items.

The bombshell was IBM’s announcement that it’s acquiring SPSS, a long-established, leading provider of predictive analytics (PA), data mining (DM), statistical analysis, and text mining tools. The acquisition, subject to the usual shareholder approvals and regulatory reviews, is expected to close later this year. But, even in advance of that near-certain consummation, IBM’s bold move has already sent shockwaves throughout the analytics market.

Most important, IBM has acquired the second largest vendor of PA/DM solutions, dwarfed only by privately held SAS Institute. In this segment, IBM’s proposed acquisition is having the same impact that its Cognos buy had on the business intelligence (BI) market two years ago. In many discussions with Forrester customers, SPSS is often mentioned as a key solution provider for predictive modeling and statistical analysis against structured, semi-structured, and unstructured content.

For IBM’s competitive standing in the data management market, this acquisition represents one of the last missing pieces of its Information On Demand (IOD) portfolio. By acquiring SPSS, IBM has acquired a substantial PA/DM brand with a very loyal set of longtime customers who have build their customer churn, supply chain optimization, and other predictive models on its best-of-breed platform. SPSS recently underlined its feature-comprehensive value proposition through re-branding around the “Predictive Analytics Software” (PASW) family name. But longtime customers didn’t need to be reminded, of course, that what used to be known as “Clementine” defines a functional high-bar in this solution segment.

IBM would probably be the first to admit that it took its focus off the PA/DM market over the past several years as it build out the BI, data warehousing (DW), and other pieces of its IOD portfolio. IBM had never really exited the PA/DM market, but casual observers might have thought otherwise. However, the vendor three years ago chose to de-emphasize its Intelligent Miner tools--which support mining of structured data--as stand-alone offerings. It essentially buried these solutions, moving them into its InfoSphere Balanced Warehouse family, where they are now offered as features of its Enterprise Edition DW software, rather than as stand-alone tools that would be enhanced and evolved independently.

IBM and SPSS’s respective customer bases should rest assured that overlaps among their respective product portfolios are not extensive. Once the acquisition closes, IBM is almost certain to build out its SPSS brand and, over the coming 1-2 years, phase out the Intelligent Miner technology within its InfoSphere portfolio. One tricky issue is which text analytics solution family--SPSS’ or IBM’s OmniFind solutions--will prevail as the parent converges these offerings in its IOD portfolio. Another issue is how IBM will integrate the SPSS offerings into its still-evolving in-database analytics roadmap for InfoSphere Balanced Warehouse. Hopefully, IBM will maintain and extend SPSS’ already extensive in-database analytics integration with a broad range of vendor DWs, including such Big Blue rivals as Oracle, Microsoft, Sybase, and Teradata.

Who loses from IBM’s acquisition of SPSS? Fundamentally, one can’t help think that SAP missed the boat by not seizing the opportunity to acquire partner SPSS, whose Clementine technology it OEMs, has integrated with its BI technology, and sells as SAP BusinessObjects Predictive Workbench. PA/DM is an increasingly key component of a full-fledged BI solution stack. However, the remaining field of vendors with stand-alone, horizontally applicable PA/DM vendors consists primarily of vendors who are large but proudly and stubbornly independent (especially, SAS Institute); high-quality but much less widely adopted (e.g., KXEN, ThinkAnalytics); or specialized on customer, financial, scientific, or other specialized analytics (e.g., Unica, Fair Isaac, Accelrys).

Some IBM rivals in the BI space already have strong PA/DM tools, most notably Oracle (Oracle Data Miner) and TIBCO/Spotfire (the Insightful tools). Among BI vendors, Microsoft, MicroStrategy, and Information Builders have PA/DM capabilities, but they are not to a SAS or SPSS level of sophistication. These and other BI vendors should also be scouting for strategic acquisitions.

What do you think? Will IBM’s acquisition of SPSS lead to further merger and acquisition activity in this space as other leading BI players strengthen their PA/DM solutions?

FORRESTER blog repost BI, Analytics, and CEP: Some Fruitful Potential Follow-Ons from Software AG’s Acquisition of IDS Scheer

BI, Analytics, and CEP: Some Fruitful Potential Follow-Ons from Software AG’s Acquisition of IDS Scheer

By James Kobielus.

Yes, of course, Software AG is buying IDS Scheer primarily for the latter’s ARIS family of business process management (BPM) tools. I’ll leave it to my Forrester colleagues who focus on BPM--on both the IT and TI sides of the house--to call out the ramifications for Software AG’s positioning in that market.

But, believe it or not, this deal will also launch Software AG into the growing markets for business intelligence (BI), analytics, and complex event processing (CEP) solutions. We bet you didn’t realize that IDS Scheer has ARIS solutions in these fast growing markets, but in fact they do--and they’re continue to evolve those offerings.

It’s no surprise that IDS Scheer’s BI, analytics, and CEP offerings supplement and extend its BPM portfolio. Its CEP solution, ARIS Process Event Monitor, supports business activity monitoring (BAM). Its analytics offerings, ARIS Process Performance Management and ARIS Performance Dashboard, support visualization, dashboarding, scorecarding, drilldown, and alerting on process key performance indicators (KPIs), both historical and real-time. And its forthcoming BI offering, ARIS MashZone, will support self-service user development of reports, dashboards, and other views of process and business metrics.

IDS Scheer has little market share in these non-core segments. And the vendor is no immediate threat, by itself or under its future corporate parent, to the leaders in the BI, analytics, and CEP segments. Indeed, its forthcoming mashup-oriented BI offering only provides a subset of the features available from market leaders such as SAP Business Objects, IBM Cognos, and MicroStrategy. But the fact that Software AG will soon be able to provide its own offerings in those segments, rather than rely wholly on partners, represents an important step in its attempt to field a full service oriented architecture (SOA) solution stack.

As noted in a blog entry a year and a half ago, BI is the crown jewel in any comprehensive SOA solution portfolio. SOA suites cannot be considered feature-complete unless they incorporate a comprehensive range of BI features. This acquisition continues the ongoing SOA solution build-out strategy that motivated Software AG to acquire webMethods in 2007.

But it’s not clear yet whether Software AG plans to flesh out its BI, analytics, and CEP strategies going forward and thereby confront SAP, Oracle, IBM, Microsoft, and other SOA full-stack vendors head-on in these segments. It is also unclear how much effort or expense Software AG would incur in extricating the IDS Scheer offerings from the larger ARIS portfolio in order to make them more general-purpose and less BPM-centric. Nevertheless, Software AG will at the very least have a strong set of enabling technologies to support any such strategy in the near future.

What’s most exciting, and potentially differentiating, about the Software AG/IDS Scheer BI portfolio is the combination of CEP with mashup and an in-memory architecture to support truly real-time, interactive analytics. In other words, Software AG/IDS Scheer could take a page out of the book of another SOA full-stack vendor: TIBCO and its Spotfire product group. In doing so, Software AG/IDS Scheer would also be well-positioned to duke it out with SAP, IBM, Microsoft, and Oracle, all of which are beginning to emphasize in-memory CEP-enabled BI strategies. As we noted in a report from late 2008, in-memory architectures are coming to dominate the BI arena. Likewise, Forrester has called attention in a recent report to the growing adoption of CEP for truly real-time BI.

Whether Software AG capitalizes on the opportunity to expand its SOA solution stack into BI remains to be seen. Considering that it took Oracle more than a year to publicly declare how it will position BEA’s CEP and data federation technologies within its own SOA stack, we may have to wait a while before Software AG and IDS Scheer craft an equivalent roadmap--if they ever do.

But if they wait too long, the newly merging vendors may find that the dynamic SOA, BI, and CEP markets have passed them by.

poem A Mortal Mutters

A MORTAL MUTTERS

Sun will shine without
my skin to receive it. Yes,
the sun will remain.

Green will gleam. The leaves
and the slime will all be fine.
As before my time.

Before this blessed
me could conceive that he too
would be forgotten.

poem Terror

TERROR

A studious girl's
laboratory acid burn
continues to sting.

Ears and attention,
fingers also wobble, they're
axes x y z.

Every potion steams,
every motion screams out its
margin of error.

poem Churchgoing

CHURCHGOING

Material as
heavy as religion kills
in the aggregate.

Please pardon me for
preferring the cool air in
empty cathedrals.

An enormous room.
My solitary breath. The
infinite echo.

poem High-Relief Phoenician Sarcophagal Frieze

HIGH-RELIEF PHOENICIAN SARCOPHAGAL FRIEZE

Alexander rears,
points his spear of air into
a prone Persian heart.

Alexander's mount,
all equine response and white
brute alabaster.

In middle assault,
the flanked warriors are all
nude and helmeted.

poem Workend

WORKEND

Sun and day are blank
as Eden, a pleasure dream
free of imagery.

The only pressure
is past: the embossing of
odd sleep positions.

Atlas carried the
atmosphere with grace, as a
cold world's counterweight.

FORRESTER blog repost BI Mashup Maturity Model? Oxymoron? Au Contraire Mon Frère!

BI Mashup Maturity Model? Oxymoron? Au Contraire Mon Frère!

By James Kobielus

In one of my recent tweets, I commented that Forrester has developed a maturity model for enterprise adoption of mashup-style, self-service development of business intelligence (BI) applications. Indeed, we have, and it will appear in my forthcoming Forrester report, “Mighty Mashups: Do-It-Yourself Business Intelligence for the New Economy.”

Another tweeter--an astute, but sadly, non-Forrester BI analyst--scoffed that “BI mashup maturity model” is an oxymoron. Respectfully, I must disagree. Enterprises are adopting self-service BI approaches for many reasons--principally, to cut costs in a tight economy, to unclog the development backlog, and to speed delivery of actionable, targeted intelligence to decision makers. Also, companies are providing users with BI tools to do interactive, deeply dimensional exploration of information pulled from enterprise data warehouses (EDW), marts, cubes, transactional applications, and other systems. Furthermore, organizations everywhere have adopted browser-oriented BI environments that leverage the full Web 2.0 interactivity and collaboration.

Sitting at the convergence of those trends is BI mashup, which Forrester sees as the new paradigm for truly pervasive decision-support systems. What throws off some people is the term “mashup,” which sometimes gets pigeonholed as simply referring to using, say, Google Maps to display geocoded performance metrics and sundry Internet-sourced data in a browser-based dashboard. Yes, BI mashup encompasses that approach to presenting and integrating diverse data, but its application is much broader.

Just as important, BI mashup is not bleeding edge. Rather, BI mashup leverages the in-memory BI clients, semantic virtualization layers, data federation middleware, automated data discovery, and other next-generation BI tools and platforms.

No one vendor or user has yet put together an end-to-end BI environment that is entirely focused on mashup-style self-service development. However, Forrester sees the BI industry converging toward as mashup-oriented architecture over the coming 2-3 years. With that in mind, we sketched out a BI maturity model that encompasses the following four levels (the first 3 of which are represented in case studies in the upcoming report):

• Level 1: Lightweight presentation mashup against transactional applications: This basic maturity level is for companies that have no prior BI or EDW; have little in-house BI expertise; and are comfortable with allowing casual users to use their browsers to customize parameterized reports from data from packaged business applications.
• Level 2: Deep presentation mashup against EDW: This level is for organization that do have prior BI and centralized EDWs, but have an understaffed BI development group and/or power users and data modelers urgently require the ability to mashup and explore historical and current data within sophisticated BI workspaces.
• Level 3: Full BI mashup in federated environment: This level is for organizations that have decentralized, dynamic data management environments, and have the expertise to design reusable, composite data services to seamlessly mashup internal and external information.
• Level 4: Full collaborative mashup with IT governance: This level is for organizations that want to encourage subject matter experts and operational users to collaborate on analytics created through mashup, but who are also concerned that all mashups be controlled, governed, and monitored in accordance with enterprise policies and best practices.

As I said, it will take a few years before we see a substantial number of enterprise case studies that implement the pinnacle of collaborative mashup with tight governance. Nevertheless, when you follow the evolution of next-generation solution portfolios from leading BI vendors such as SAP, IBM, Microsoft, and others, it’s clear that self-service user-centric mashup, to varying degrees, is a core theme.

BI mashup has such a strong business case that we’re confident it’s more than simply a “down economy” theme. It will almost certainly grow in importance for information and knowledge management professionals as the economy improves.

poems Some Detroit-inspired/inflamed pieces from past years

AT THE STRAITS

Detroit riots and rots,/deteriorates and/resists resurrection.//Detroit's distraught, a rut/of debt and death, a depth/charge of desolation.//Dry as snot. A driven/disaster. A drag to/avoid. My home. Destroyed.

URBAN PINES

An immense metropolis: The wary mother of an internal forest.

ZUG GROSSE BELLE AND A TOWN CALLED HELL

Downtown once was fresh. Environment once pristine. Ten millennia plus since last glacier retreat. River brought sweat salt gravel hope bootleg rum and not-so-distant ancestors. Indigenous people were just shunted aside. Our kind built dangerous dump. Tell the ice come back.

FORRESTER blog repost Database Religions Dissolve into the Big Billowing Virtual Data Cloud

Database Religions Dissolve into the Big Billowing Virtual Data Cloud

By James Kobielus

Virtualization is a venerable old computing concept that has achieved new life in recent years.

Virtualization brings to life a new world of more flexible service provisioning while cleverly emulating the old world that is being replaced. Virtualization refers to any approach that abstracts the external interface from the internal implementation of some service, functionality, or other resource.

The promise of virtualization is that, no matter how scattered and diverse, all pooled resources behave as if they were a single unified resource, both for usage and administration. In a sense, this is the practical magic that Arthur C. Clarke identified with advanced technology. The external interface may conceal various facts about the implementations of the underlying resources. The virtualized resources may run on diverse operating and application platforms;have been deployed on nodes in diverse locations; have been aggregated across diverse hosting platforms (or partitioned within a single hosting platform, either through virtual machine software, separate CPUs, or separate blade servers); and have been provisioned dynamically in response to a client request.

When Noel Yuhanna and I presented on enterprise database virtualization last week at Forrester IT Forum, we took pains to point out that is not a radically new paradigm. In fact, database administrators (DBAs) have been doing virtualization for a long time and not realizing it. We’re all familiar with such database virtualization approaches as policy-based server clustering, massive parallel processing database grids, and enterprise information integration. In these environments, you can identify the virtualization layer as “single system image,” “semantic abstraction,” or some other approach.

What all these approaches share is that they make two or more repositories behave as if they were a single database for unified access, query, reporting, predictive analytics, and other applications. If you wish, I could drill down further into the layers of database virtualization--data virtualization, transaction virtualization, and platform virtualization--but that would be too much for a mere blogpost.

One twist that I didn’t have time to explore in depth last week is the notion that the traditional hub-and-spoke enterprise data warehousing (EDW) architecture is itself a form of database virtualization. The hub-and-spoke model transforms analytic data to a common “spoke-side” semantic access model, such as star schema or columnar. As such, this approach abstracts from the data models (usually 3NF relational) implemented at the EDW hub tier, the staging tier (perhaps file-based), and OLTP sources (perhaps hierarchical, XML, or what have you).

When you realize that each data-persistence approach has its optimal deployment sphere, you’re thinking database virtualization. At that point, you start to realize that the various database religions--relational is supreme, columnar is king, and so forth--are not absolute truths. They’re simply sectarian texts in a tradition of longer vintage: the evolution of truly all-encompassing data virtualization clouds.

Yes, I’m using “cloud” in this context because it best describes this new paradigm. Cloud-based virtualization is beginning to seep into analytic infrastructures. To support flexible mixed-workload analytics, the EDW, over the coming five to 10 years, will evolve into a virtualized, cloud-based, and supremely scalable distributed platform.

What are the outlines of this new paradigm? The virtualized EDW will allow data to be transparently persisted in diverse physical and logical formats to an abstract, seamless grid of interconnected memory and disk resources and to be delivered with subsecond delay to consuming applications. EDW application service levels will be ensured through an end-to-end, policy-driven, latency-agile, distributed-caching and dynamic query-optimization memory grid, within an information-as-a-service (IaaS) environment. Analytic applications will migrate to the EDW platform and leverage its full parallel-processing, partitioning, scalability, and optimization functionality. At the same time, DBAs will need to make sure that cloud-based DW offerings meet their organizations’ most stringent security, performance, availability, and other service-level requirements.

I won’t opine here and now on how much enterprise data will be persisted in public clouds vs. private environments that incorporate many of the same platform virtualization technologies. I’ll save that discussion for the upcoming Forrester reports that Noel and I are developing in virtualization of transactional and analytic databases, respectively.

Expect those in Q3 or thereabouts. Thanks everybody who attended our preso last week in Vegas!

poem Tormé

TORMÉ

Faux Paris is as
good as being there. People
kiss oblivious.

Faux New York is so
obviously not to scale.
The model city!

Walking Mel Tormé
Way I supply the missing
melody and fog.

poem The Hard Rock

THE HARD ROCK

Big Deb the Vegas
waitress with the colossal
lungs could really sing.

Deb and the twenty-
first birthday girlie gave it
their best Benatar.

Earned a big tip by
saying nothing to me. Just
refreshing my tea.

poem Unspeakable

UNSPEAKABLE

Vegas: the best god
damned museums anywhere.
The names, anyway.

Atomic Testing
Museum: Should I risk it?
Leadline my eyeballs?

Or the Erotic
Heritage: Endure a long
grinding and blinding?

TWEETLOG Mon May 18-20 so far

RT @jilldyche: #haiku @haikulove @twaiku. Haiku du jour: jk--"ANTIMAY: Memorial Day. It's honorary Summer. Spring's sunburned demise. "6 minutes ago from TweetDeck

RT @CompositeSW: #FITF09. Jim sees lots of opportunity in "database" virtualization: jk--Noel Y. and I copresenting on Fri on DB virt'zn.22 minutes ago from TweetDeck

RT @mikojava: Arrived in Las Vegas for #FITF09 in the cab on the way to venue: jk--Oh no, Miko's almost here.
Don't tell him SOA's dead.about 2 hours ago from TweetDeck

Newsgathering is rampant in Web 2.0. Newsvetting is everywhere as well. Newsreporting is ubiquitous. News"papers" not. Paper not essential.about 8 hours ago from web

#FITF09: Interop 09 happening a mile-plus down the Strip at Mandalays. I don't find Interop interesting anymore. Easy "temptation" to avoid.about 8 hours ago from TweetDeck

#FITF09: This morning Forrester founder George Colony speaks.about 8 hours ago from TweetDeck

#FITF09: Preparing for a busy day of 1:1s at Forrester IT Forum. Multitasking these plus other tasks/projects I
brought on the road.about 8 hours ago from TweetDeck

Twitter's char-count constraints lead some to think it can't both report and critique. I usually attempt former in first 70, latter in last.about 8 hours ago from TweetDeck

Tweeting from field is news gathering and reporting in one swift gesture. Vulnerable to "mindless real-time stenographic reportage" syndromeabout 9 hours ago from TweetDeck

RT @lorita: Just finished a great report from @forrester "To BW or Not To BW." jk--Thanks Lorita. Boris Evelson and I co-authored that one.about 9 hours ago from TweetDeck

Any flack can "report" what others say--govt officials & official lies, vendors & self-serving PR. True pro reports what they find on own.about 9 hours ago from TweetDeck

Twas then I realized that reporting not core of news biz. Gathering is. Hunting-gathering fresh meat/fruit. Stalking/slaying the news beast.about 9 hours ago from TweetDeck

Jerry ter Horst had me doing archival news searches a Google would nail these--nice guy (quit Ford admin over Nixon pardon)--smelly pipeabout 9 hours ago from TweetDeck

Ah yes, I remember my internship at the Detroit News Wash. Bureau in summer 1978; twas Jimmy Olsen-Kobielus, cub reporter, reading the wiresabout 9 hours ago from TweetDeck

Slower tech newsday on the wires than yesterday, now settling into the summer slough, next newsrush day in early/mid Septabout 9 hours ago from TweetDeck

RT @alyswoodward: @jameskobielus I'm only usually [in LV) for 72 hours!: jk--And then you're whipsawed by timezone and climate disruptions!about 9 hours ago from TweetDeck

RT @alyswoodward: @jameskobielus ahh, the sun over the desert, love it. The dry eyes/mouth/lungs, ugh. jk--Takes 48 hrs min to acclimate.about 9 hours ago from TweetDeck

Twittering about Twitter is like writing poetry about poetry: grooving on your own nerdishness.about 9 hours ago from TweetDeck

RT @kexpplaylist: People Got A Lotta Nerve by Neko Case #KEXPabout 10 hours ago from TweetDeck

Listening to Neko Case "People Gotta Lotta Nerve" from her great new album "Middle Cyclone." She says she's the brass section in any group.about 10 hours ago from TweetDeck

Another dry morning in Vegas, following a night of continual waking to re-hydrate, irrigate the inevitable cottonmouth.about 10 hours ago from TweetDeck

@JAdP : Architecture is the bridge. Alignment anchors the bridge to terra firma at both ends. Vision ensures it connects the right ends.about 10 hours ago from TweetDeck in reply to JAdP

@rschmelzer: A lone wolf howling attracts more attention than one in unison with in or out crowd. If howls in odd harmonic with both, best.about 10 hours ago from TweetDeck in reply to
rschmelzer

Realizing that crowds, in or out, crowd, i.e. cramp, and are to be avoided, unless need ferment of friction/discourse, then go mosh/mash.about 22 hours ago from TweetDeck

RT @rschmelzer: ... feel like part of the out-crowd. ...struggle to be accepted. What's the secret? jk--Realizing the out-crowd's a crowd.about 22 hours ago from TweetDeck

Thanking Shadi for Advils, taking a breather upstairs, getting ready for Sybase dinner, listening to Bill Callahan "The Wind and the Dove"about 22 hours ago from TweetDeck

RT @JoeBarkan: @jameskobielus There's no such thing as "too guitar-rocky." #FITF09: jk--I beg to differ. Guitar-band rock can easily overdo.5:04 PM May 19th from TweetDeck

RT @markmadsen: Five Reasons to NOT follow someone on Twitter: jk--Have no control over who we FOLLOW. Depends on each follower's view.4:35 PM May 19th from TweetDeck

#FITF09: Another shameless self-plug, related to previous tweet, but calling out my (guilty?) love of Enya's vibes (http://bit.ly/ZKMce).4:26 PM May 19th from TweetDeck

#FITF09: Another closing thought on that session. Theme music was too guitar-rocky. No, I don't want hiphop or electronica. One word: Enya!4:24 PM May 19th from TweetDeck

#FITF09: Post-mortem on Cameron talk: Value-based architecture? No. Instead, value-based alignment. Architecture is tail, shouldn't wag dog.4:23 PM May 19th from TweetDeck

RT @passion4process: IT also needs to adopt bus. processes that monitor/track IT bus. value .

#FITF09: jk--BI applied to IT org/iniatives4:06 PM May 19th from TweetDeck

#FITF09: "Definition of [IT's business value] will change over time...As regards what the metrics are, ask business." Business POV rules!4:04 PM May 19th from TweetDeck

RT @passion4process: Bobby Cameron's purple bow tie -.... #FITF09: jk--It's cocked at jaunty, Sinatrian angle. How's that for Vegas cool?!4:01 PM May 19th from TweetDeck

#FITF09: Cameron: "Embedding key IT roles in the business orgs is what we see happening anyway." "Need to help by embedding resources."3:57 PM May 19th from TweetDeck

#FITF09: Cameron: "Challenge for CEO is based on how plugged-in they are.....When talking to CEO, should be focused on growth, EPS, etc."3:55 PM May 19th from TweetDeck

#FITF09: Cameron: "Wonderful opportunity for IT...to bring clarity to the problem [of which services should be shared vs. localized]."3:53 PM May 19th from TweetDeck

#FITF09: Cameron: "Can't optimize in our job unless you're connecting with the business." Optimization = alignment.3:50 PM May 19th from TweetDeck

RT @merv: Designing my template for PPT - easy to get swept away by the possibilities. Simple, simple, keep repeating....: jk--Pictoreality!3:48 PM May 19th from TweetDeck

#FITF09: IT talking in the business terms. Essential, but always a challenge, especially as IT deep-ends on nouveau clouds, virtz'n, SOA.3:47 PM May 19th from TweetDeck

#FITF09: Forrester's role-based avatars resemble a pantheon. Incarnations of the same principle: IT serves the business role.3:44 PM May 19th from TweetDeck

RT @jameskobielus: #FITF09: I see tweets by @passion4process, @mgualtieri, @gleganza, @pleclare, @rbkarel, @lauraramos, @akarlin, & yrs trly3:42 PM May 19th from TweetDeck

RT @gcolony: Cloud computing is over-stated. Cloud and local devices will share processing.

#FITF09. jk--Cloud's the resource pool.3:41 PM May 19th from TweetDeck

#FITF09: Any way to batch transform all of my previous tweet hashtags to the correct one?3:37 PM May 19th from TweetDeck

#FITF: I see tweets by @passion4process, @mgualtieri, @gleganza, @pleclare, @rbkarel, @lauraramos, @akarlin: multiple Forrester tweetstreams3:31 PM May 19th from TweetDeck

#FITF: How does business POV shape perception of IT? Remember J. Fallon on SNL as jerk IT support guy? A bit scary, a bit reassuring.3:27 PM May 19th from TweetDeck

#FITF: Bobby Cameron on "Making Value Core to IT's Business." Wants us to comment on "scary vs. reassuring" images. POV-shaped perception.3:24 PM May 19th from TweetDeck

#FITF: Tom's telling them about the legendary gauntlet that Forrester analyst candidates must run. Grueling. Excellent pre-onboarding.3:22 PM May 19th from TweetDeck

#FITF: Tom's striped tie doesn't do it for me. Accost him in the hallways and give him a piece of YOUR minds on the matter.3:20 PM May 19th from TweetDeck

#FITF: We give you actionable next steps to take back to your companies. We provide practical guidance. Take us up on 1:1s, you'll see.3:19 PM May 19th from TweetDeck

#FITF: Attendees will notice that every Forrester preso has upfront and closing slides that nail the value prop of that particular tech.3:18 PM May 19th from TweetDeck

#FITF: A lot of people in this room. Good turnout.3:16 PM May 19th from TweetDeck

#FITF: Protecting and promoting innovation. Uber-theme for Kobielus/Yuhanna Friday preso on enterprise DB virtualization (shameless plug).3:15 PM May 19th from TweetDeck

#FITF: At UPS, no technology strategy apart from business strategy. That's the fundamental value prop: IT entirely instrumental to business.3:14 PM May 19th from TweetDeck

#FITF: IT-role-based definitions of value? Value specific to your business contribution? How do you measure /communicate that? Justify job?3:12 PM May 19th from TweetDeck

#FITF: Value defn's: Pos: We're building a future we actually want. Neg: We prevent you from being the next headline on CNN.3:10 PM May 19th from TweetDeck

#FITF: "Redefining IT's Value to the Enterprise." Uber-theme of this year's Forrester IT
Forum. What's "value"? Is it indispensability?3:07 PM May 19th from TweetDeck

#FITF: Tom Pohlmann says we're tired of the economy gloom and doom. Yes, for sure. A lot of the moaning is hypochondriacal overreaction.3:06 PM May 19th from TweetDeck

#FITF: Wow. Side-by-side Forrester analysts co-tweeting the event. Mike Gualtieri tweeted the music issue on my mind. Sting's insinuation?3:04 PM May 19th from TweetDeck

@AskMrsHR: "Yoga, where have you been all my life. I feel great". jk--Same effect here. You can never be too calm, flexible, tensile.11:38 AM May 19th from web in reply to AskMrsHR

#FITF: "Expressamente Illy" over on first level. European-based company, or faux Europa, like so much Vegas? Mongrel Euro-American, like me?11:29 AM May 19th from TweetDeck

#FITF: Amazing the burst of energy/concentration that a little coffee buys you. The caffeine rush is the next best thing to an analgesic.11:26 AM May 19th from TweetDeck

#FITF: Noting the crush of press releases today. This week is the last of the spring season of vendor announcements. Prepare for summer lull11:23 AM May 19th from TweetDeck

"Larry Fulton... to Present on Six Principles for Addressing the Unique Challenges of Multi-site Integration" ( http://bit.ly/ZFlkU)11:21 AM May 19th from TweetDeck

Last year's Vegas poems: "Jade" ( http://bit.ly/oxeNP)11:16 AM May 19th from TweetDeck

Last year's Vegas poems: "Neural Carpal" ( http://bit.ly/m8Kcu)11:15 AM May 19th from TweetDeck

Last year's Vegas poems: "These Plains" ( http://bit.ly/VcTXQ)11:14 AM May 19th from TweetDeck

Last year's Vegas poems: "Center of Conventions Exhibitions Conferences and Expositions" ( http://bit.ly/106hnE)11:13 AM May 19th from TweetDeck

Listening to XTC "The Ballad of Peter Pumpkinhead," by the deliciously misanthropic Andy Partridge. "Dear God" "Making Plans for Nigel" etc.11:10 AM May 19th from TweetDeck

@chaskielt: I use "secret sauce" ironically. It's a multi-DB world, behind virtualization layer. Told Sybase that. No monoculture.11:06 AM May 19th from TweetDeck in reply to chaskielt

@alyswoodward: It was too noisy at the "Dos Caminos" restaurant for others to listen in. Could barely hear my own voice, over head pain.11:04 AM May 19th from TweetDeck in reply to alyswoodward

@mikojava: Miko: the tweetup was last night. But I'd very much like to see you again. Give me a holler when you get here.11:03 AM May 19th from TweetDeck in reply to mikojava

#FITF: Strong attendance and customer interest, in soft economy, shows that Forrester events deliver value. We don't take that for granted.11:02 AM May 19th from TweetDeck

#FITF: Great opportunity to meet Forrester analysts, and for us to meet each other. We're so big and diversified, not all of us have met.11:00 AM May 19th from TweetDeck

#FITF: I've always found industry events a superb opportunity for deep-dive research. Even better when top analysts present latest research.10:58 AM May 19th from TweetDeck

#FITF: Today, e.g., various cloud (Hammond, Rymer, Staten, Wang), storage (Reichman), green IT (Mines), EA (Gilpin, Heffner, Leganza) topics10:56 AM May 19th from TweetDeck

#FITF: Forrester has an exciting agenda for this event. Many analysts have great topics. I'm still working out which ones I can attend.10:52 AM May 19th from TweetDeck

#FITF: Anybody who wants a preview of mine and Noel's preso, "Enterprise Database Virtualization," is urged to engage us in 1:1 sessions.10:51 AM May 19th from TweetDeck

#FITF: Doing work outside Zeno, waiting for the event to formally open. Checking my 1:1 requests: data warehousing, advanced analytics, BI.10:49 AM May 19th from TweetDeck

Wondering why TweetDeck won't load this morning. Wondering when columnar database vendors will stop trying to promote that as secret sauce.9:28 AM May 19th from web

@akarlin @pleclare: Thanks for hosting the Forrester TweetUp. Let's do more.1:15 AM May 19th from web

At TweetUp, Friedberg talked up Kognitio's "trusted cloud" notion. I call it "transparent cloud." See right thru: hosting, security etc.1:13 AM May 19th from web

Wondering how Steve Momorella of @cvillenews got my Twitter name. Probably through @ohjko. I doubt that latter's sister Sonya will follow.1:00 AM May 19th from web

Dodged the usual crazy foot traffic on Strip. Showed 'em a lilttle of my Red Grange moves. Had my music: Arthur and Yu: "Lion's Mouth."12:50 AM May 19th from web

Had to bug out early, splitting headache. Tried my walk outside for air. Didn't help. Did I mention that Vegas is an oven? Even at night.12:47 AM May 19th from web

At TweetUp, met with Steve Friedberg, @StevePR104. Steve's a networker's networker. Also writes articles for TDWI, other pubs. Good guy.12:46 AM May 19th from web

Brought my laptop to the TweetUp, in a restaurant/bar. Learned we were to do everything there but tweet. Beginner's misunderstanding.12:44 AM May 19th from web

Tonight at Forrester TweetUp, chatted with Laura Coronado: @lollieshopping. Asked if she's a "fashion victim," when I meant "fashionista."12:41 AM May 19th from web

Expecting strong turnout for Forrester IT Forum. Everybody's starting to emerge from the recessionary funk. IT budgets starting to perk up.12:38 AM May 19th from web

Did some Forrester internal bonding. LIFO more useful than Myers-Briggs. White wine more fish-friendly than red. Las Vegas hotter than hell.12:35 AM May 19th from web

Yesterday wore me out, but my son graduated in cold rain at UVa, has the Camry, is going to new acting job10:40 AM May 18th from TweetDeck

In transit to Forrester IT Forum, Detroit airport, helping daughter long-distance with anti-virus, not eating yet, working on webinar slides10:37 AM May 18th from TweetDeck

poem Antimay

ANTIMAY

Memorial Day.
It's honorary Summer.
Spring's sunburned demise.

poem Embalm

EMBALM

Spirit is second-
hand balm, recycled mist of
communal refresh.

Spent energy is
the release of pent-up breath
and remembered hurt.

The exhalation
returns the body to its
natural blackout.

FORRESTER blog repost Self-Service Business Intelligence Depends on Automated Data Discovery

Self-Service Business Intelligence Depends on Automated Data Discovery

By James Kobielus

If you tuned into my Forrester teleconference yesterday, you heard me discuss the end-to-end infrastructure necessary to fully support mashup-style self-service business intelligence (BI).

One of the key features for BI mashup is automated source-data discovery, which spares information workers from having to find new data sources or fresh updates from existing sources. Instead, the user simply relies on the BI and back-end data virtualization infrastructure to perform these critical activities as ongoing background tasks. Once new sources and feeds are discovered, transformed to a common semantic model, and published to a BI-mashup registry, all the user needs to do is drag and drop them visually into their mashed-up reports, dashboards, and other analytics.

Automated discovery is not only key to BI mashup, but to trustworthy data as well, because it helps detect and remediate anomalies across disparate data sources. Only a few vendors on the market today provide strong features for automated source discovery. One of them is Composite Software, which recently released an appliance that performs these functions. Another is Exeros, which is the closest thing to an automated-data-discovery pure-play in the market today.

Or, rather, was the closest thing, until IBM announced this morning that it is acquiring Exeros. I’ve been following Exeros for several years and have long considered them a strong candidate for acquisition by a leading BI, data warehousing (DW), data integration (DI), or data quality (DQ) vendor. On IBM’s part, this acquisition makes great sense as a complement to its InfoSphere and Optim portfolios on the data management and governance side of the house.

It will also fit nicely with IBM’s Cognos portfolio as a key enabler, potentially, for BI self-service mashup. As I stated on my teleconference, some vendors are further ahead on putting together a completely mashup-enabling end-to-end BI solution, and Cognos is among them. You can download the teleconference slides from Forrester’s website, listen to my streaming audio, and/or wait for my forthcoming report for more in-depth thoughts on this topic.

Now the ball’s in IBM’s rivals’ courts regarding whether, when, and how they plan to add automated source discovery to their BI portfolios.

TWTR-EXTRA imho Hardcopy news won’t vanish but like hardcopy photos will materialize only when we hit print, which will be seldom

All:

Like all of you, I’ve been following the decline of traditional journalism, and the increasingly frantic efforts of the industry to save itself in the face of the greatest nemesis of all: the Internet. Just this evening, I read excellent discussions on all this by the ever-stimulating Jason Pontin and Clay Shirky.

Just today I noticed that the New York Times is threatening to close down the Boston Globe (didn’t realize the former owned the latter) if the Globe’s unions don’t make serious contract concessions. I have no position on this dispute, but, after seeing longtime Seattle and Denver papers bite the dust, it’s clear to me that the days of every major city having its own dedicated daily newspaper are coming to a close. Why is this unthinkable? Does every major city have its own major league baseball team? Its own world-class research university? Its own internationally renowned symphony orchestra? Its own locally owned chain of department stores?

In the newspaper business, what’s coming is almost certainly a new order where we have national daily papers with city-specific local news sections. Just as Macy’s grew into a nationwide department store chain in large part by acquiring shaky local store-chains, it’s not inconceivable that the New York Times might become a truly national newspaper, inserting, say, a substantial daily Boston section for distribution in New England, a Detroit section for southern Michigan, a Dallas-Fort Worth section for north Texas, and so on. This is not unprecedented: indeed, the Washington Post has an excellent array of local news sections for communities throughout the District of Columbia, Maryland, and northern Virginia. Like many people, I often turn primarily to my region’s section (Fairfax County) and only get to the “A” section (national and international) later, or not at all, on any given day.

Another, almost inevitable feature of the coming order is that some of us will continue to pay for the convenience of having a daily hardcopy of a subset of the news that most interests us dropped off at our residence first thing in the morning--or at various intervals. There are plenty of reasons why we may want this to continue, such as having it to read over breakfast. But there are also many reasons why we will insist on not being delivered sections that we never read and don’t want (e.g., I’ve long since lost interest in sports, and routinely toss it unread; others don’t care for business; many people couldn’t care less about international; and op/ed pages are almost never looked at in most households). Many of us would gladly scale back to a weekly hardcopy paper that only publishes a summary of the news and features we care about--and only comes, say, on Sunday, when we actually have spare time to read it, and only comes bundled with coupons, comics, special glossy magazines, and other cool things.

But most of us will prefer to access most of our news most of the time online, and only online. Many people will only desire a hardcopy now and then, and only of particular stories. In those cases, that hardcopy will issue from their own printer, not from huge printing presses staffed by contentious union members.

Newsgathering will still be done by large institutions, descendants of today’s newspapers and magazines, but will become more of an aggregation of loosely shifting groups of “reporters,” sometimes known as bloggers. Opinions and analyses will come from this same huge global pool of knowledgeable individuals, many of whom will make little or no money directly from their published viewpoints. Many, if not most, of these “journalists” will multitask that work alongside paying “day jobs,” doing so to further some personal passion or supplement some other business model. For example, IT industry analysts have long contributed articles to trade papers in order to strengthen their “branding” as analyst/consultants, while making only a pittance from their “journalistic” activities.

I see that as an important model of future journalism. For years, I’ve had trouble explaining to people how my graduate degree, M.A. in Journalism from the University of Wisconsin, prepared me for my ultimate career as an analyst. I’ve never actually had a paying job as a journalist, though I’ve been a freelance IT writer for many years. More and more, though, it feels like I’ve never really left the field I trained for.

Instead, my field has returned to me.

Jim

TWTR-EXTRA Imho IT analysts & journalists the same--skills, place in industry ecosystem--folks move back and forth between

All:

I’ve been noticing the recent tweet-backs between Curt Monash, Lance Walter, and Seth Grimes on the topic of what constitutes an IT analyst vs. an IT journalist. I thought I’d replay their tweets for you (stripped of time-sent and reply-to dimensions...sorry ‘bout that, but I’ve kept, per each tweeter a sequence from most to least recent). Then I offer my summary commentary of all that:

Their tweets:

• @CurtMonash: "That's the main benefit I see to being categorized not just as an analyst, but as a journalist too." "I want companies to be supportive if I pick a news-cycle approach to publishing on some specific story or topic." "Not sure I know when I'd want a big analyst firm to view me as a journalist. What am I missing?" "Exactly. I break news now. And a lot of the commentary published by the trade press is subcontracted to working analysts." "I don't mind being categorized as both press AND analyst. (And increasingly that's happening.) It's the either-or that causes trouble."
• @lancewalter: "I think a lot of "pure" analysts are also blurring the journalist line (good thing) cuz of blogs, syndication, death of print..."
• @SethGrimes: "I suspect some analyst firms don't want to legitimize rival, independent analysts so they ignore us as journalists."

Now Kobielus’ kommentary:

There’s no such thing as a “pure analyst” and never has been. IT analysts and IT journalists play the same role in the industry ecosystem. There’s no clear demarc between the two fields.

We all publish or perish--that’s our primary business model. We’re all essentially reporters--in other words, we research, analyze, publish, and speak on the new things that are going on in the IT world. Clearly, there are many distinctions among us: some “reporters” (analyst/journalist) have more specialized beats than others, some report on a more regular basis than others, some go a bit deeper and broader in the research than others, some do more consulting and speaking than others, some have bigger firms marketing their offerings than others, some are better known than others, some have better access to the movers/shakers than others, some have more industry/vendor background than others, some have more corporate IT background than others, and so forth.

The working relationships among IT analysts and journalists are entirely symbiotic. One open secret in this industry is that many IT analysts began as journalists, and many have essentially stayed journalists by continuing to publish widely in the trade press. Another is that IT journalists are often excellent analysts; if they weren’t, their reportage would be subpar and they wouldn’t stay in that line of work for long. Yet another is that IT journalists often rely on IT analysts for perspective setting, information, leads, and quotes. And, of course, analysts “market” ourselves in great part on our ability to be featured prominently in journalists’ stories.

As I said above, we all play the same basic role in the IT industry ecosystem. From vendors’ point of view, analysts/journalists are a key channel for getting their go-to-market messages out to customers. From users’ point of view, analysts/journalists are a key channel helping them to make sense of those messages. Clearly, as an intermediary in this flow, analysts/journalists, as a community, provide an “information brokering/filtering” role that is indispensable.

Some analysts/journalists have more influence than others--no one denies that. We’re a huge community of many voices. Each of us, analyst/journalist (individually and/or as firms, large and small), is in a constant struggle to get our viewpoints out and to strengthen our brands. Hence we turn to blogs, podcasts, Twitter, and other channels to underline those brands. Each of us is in business as well--these are our careers. None of us is “the final word” on anything.

That said, I’m a huge fan of most other analysts/journalists in the industry. There are lots of smart people who do excellent work, and I’m constantly learning from everybody else. This is an extraordinarily stimulating line of work to be in.

Curt, Lance, Seth: Tweet’s back in your courts.

Jim

poem Bankruptcy Sonnet

BANKRUPTCY SONNET

Suddenly vacant space. So suddenly
the landlords haven’t vacuumed. So very
vacant the brokers are at a loss for
adjectives. Special and spacious, this place
represents a rare opportunity
to make your statement in a property
that towers over the Interstate. So
special you and your partners can claim your
respective pieces of executive
real estate, sleek and smart, catercorner
suites overlooking the soon-to-be heart
of whatever new business you now must
build to play a part and have a shot at
being there when the economy swells.

FORRESTER blog repost Oracle’s Sun Acquisition Accelerates Push into Data Warehousing Appliances

Oracle’s Sun Acquisition Accelerates Push into Data Warehousing Appliances

By James Kobielus

Last fall, Oracle CEO Larry Ellison announced that his company was getting into the hardware business, but I think he misspoke. At that time, he was referring to the new HP Oracle Database Machine with Exadata Storage, a high-end data warehousing (DW) appliance that incorporated hardware from his partner, as well as intelligent storage software technology from that partner--and even had the partner’s name first in the product name. If that was the criterion for “getting into the hardware business”--i.e., running on someone else’s hardware--then every software vendor on earth is in the hardware business, by my reckoning.

But today’s Oracle announcement is the real deal. Oracle is acquiring longtime partner Sun Microsystems, putting the software powerhouse fully into the hardware business--and hitting the DW industry like an earthquake. I’ll let my Forrester colleagues blog on the other implications of this deal--for the open source, Java, middleware, SOA, and other markets that Sun is in--and give you a few quick thoughts on the deal’s implications for the DW market.

For starters, this deal will give Oracle the ability to engineer a completely integrated DW appliance composed of all Oracle components, including hardware and software. Now Oracle will be able to take on Teradata and IBM--both of which have long offered their own integrated solutions--more aggressively with high-performance DW offerings. Just as important, Oracle will be able to leverage Sun’s manufacturing scale economies to bring its all-Oracle DW appliances below the $25K-per-terabyte threshold needed for penetration into the midmarket.

Also, Oracle will now have another widely adopted transactional database, the open-source MySQL, that it can--and should--consider tweaking and packaging on an DW appliance. To the extent that Oracle gives customers a choice of DBMSs on a DW appliance platform, it can gain a differentiator that Teradata, IBM, Microsoft, Sybase, and Netezza lack (you have to go to a startup such as Dataupia for multi-DBMS choice on an appliance). Many information managers prefer to stick with their existing DBMSs when building a DW, and prefer to implement that DW on an appliance to take advantage of its out-of-box balanced configuration of CPU, memory, storage, and I/O.

Furthermore, Oracle is acquiring a hardware and operating system vendor that has long been one of the primary platforms on which its own DW/DBMSs, middleware, and tools have been deployed. This acquisition can only be welcome news for joint Oracle-Sun DW customers who have worried about Sun’s solvency for some time now and began to sweat serious bullets when IBM failed to emerge as a white knight. For many Sun customers, an Oracle-powered DW platform will now look like a safer bet than ever.

Of course, there are clear risks in this pending acquisition.

First, a combined Oracle/Sun sows uncertainty among the DW appliance vendors--such as Greenplum and ParAccel--who have partnered with Sun and now find themselves in earnest “co-opetition” with full-competitor (and then some) Oracle.

Second, Oracle’s other DW appliance hardware partners--including HP, IBM, and EMC/Dell--must be concerned that Oracle will now shift focus away from their respective appliance products in favor of those it builds with its own Sun hardware group.

And finally, Oracle’s acquisition of Sun--and possible future development of a MySQL DW appliance--may discourage customers from considering third-party DW appliances, such as from Kickfire--that build on MySQL. If that happens, and a market for non-Oracle-branded MySQL DW appliances never takes root, Oracle will be denying its MySQL customers the choice that Oracle Database customers already enjoy. Currently, Oracle Optimzed Warehouse customers can deploy that enterprise DBMS as a DW on their choice of Sun, HP, IBM, and EMC/Dell platforms.

Let’s hope that Oracle makes the most of its pending Sun acquisition. Ellison either misspoke last fall, or was speaking prophecy. Like most DW vendors, Oracle’s destiny is to grow ever more hardware-dependent for its long-term scalability, performance, and optimization story.

Follow me on Twitter!

@jameskobielus
http://twitter.com/jameskobielus

poem Falling Sonnet

FALLING SONNET

I

Comedy is all
banana slippage.

Tragedy is trapped
in bodies that are

forever falling,
never quite finding

their footing. Walk a
mile in a clown’s shoes,

you’ll know what Bozo
goes through, but not so

Pozzo. Even an
unlucky bastard

can presume descent
from a long line of

long lines. Too narrow
a furrow to toe.

Sun a midday moon
raking truculence.

The Moon resumes its
tragic flatulence.

Comedy is a
coma. Tragedy

is a crack, a tout
le monde with trembling

hemispherics, a
slippery sole and

a hole daring to
keep us from footing.

II

Comedy is all banana slippage.
Tragedy is trapped in bodies that are

forever falling, never quite finding
their footing. Walk a mile in a clown’s shoes,

you’ll know what Bozo goes through, but not so
Pozzo. Even an unlucky bastard

can presume descent from a long line of
long lines. Too narrow a furrow to toe.

Sun a midday moon raking truculence.
The Moon resumes its tragic flatulence.

Comedy is a coma. Tragedy
is a crack, a tout le monde with trembling

hemispherics, a slippery sole and
a hole daring to keep us from footing.

III

Comedy is all banana slippage.
Tragedy is trapped in bodies that are
forever falling, never quite finding
their footing. Walk a mile in a clown’s shoes,
you’ll know what Bozo goes through, but not so
Pozzo. Even an unlucky bastard
can presume descent from a long line of
long lines. Too narrow a furrow to toe.
Sun a midday moon raking truculence.
The Moon resumes its tragic flatulence.
Comedy is a coma. Tragedy
is a crack, a tout le monde with trembling
hemispherics, a slippery sole and
a hole daring to keep us from footing.

IV

Comedy is all banana slippage. Tragedy is trapped in bodies that are forever falling, never quite finding their footing. Walk a mile in a clown’s shoes, you’ll know what Bozo goes through, but not so Pozzo. Even an unlucky bastard can presume descent from a long line of long lines. Too narrow a furrow to toe. Sun a midday moon raking truculence. The Moon resumes its tragic flatulence. Comedy is a coma. Tragedy is a crack, a tout le monde with trembling hemispherics, a slippery sole and a hole daring to keep us from footing.

FORRESTER blog repost Dislocation Intelligence in a Brutal Economy

Dislocation Intelligence in a Brutal Economy

By James Kobielus

It's painful to see the auto, newspaper, construction, financial services, and so many other formerly vibrant sectors of the world economy go down the proverbial tubes. One of the most nauseating realities is when millions of people lose their jobs, homes, and communities in a seeming blink.

I grew up in the perpetually recessionary Detroit area. I'm attuned to the regional dislocations that come from depending too much on an industry that has seen better days. Abandoned storefronts, dilapidated housing, vacant lots, tumbleweed-quiet city streets--all of it evidence of a growing ghost town, telltale signs of a marginal economy that depends on government programs, private charity, and low-wage service jobs. During my college years, I was a policy analyst with an urban coalition in downtown Detroit, and I could see that the slide was long-term and nigh irreversible.

Even in relatively well-off areas such as Washington DC, where I've spent close to a quarter-century, we’re not immune to serious economic dislocations. The National Capital Region, so reliant on federal spending, is likely to feel the brunt of whatever cuts Obama will almost certainly make to close this massive deficit. And even here you can’t escape dislocations in sectors that we all depend on, such as retailing, as evidenced by, for example, the ex-Circuit City big boxes that seem even bigger and boxier now that they’re totally empty.

In recent years, corporations have adopted location intelligence solutions to support their market-entry strategies, such as adding new stores and waging marketing campaigns. They also use these tools--essentially, geographic information systems coupled with predictive modeling--to optimize their footprint in existing markets. And to a lesser extent, they also use location intelligence to plan their exit strategy of closings and retrenchments. But when departures are hasty--such as any Chapter 11 proceeding--all we’re left with are vast tracts of vacant real estate, plus many formerly employed people who must find a way to survive amid ruins. Imagine how a General Motors or Chrysler bankruptcy will hit Detroit--it will be a liquidation to rival Hurricane Katrina in its devastation of a major city.

Dislocation intelligence is something that the leaders of the auto companies--and the Obama administration--should exercise when making the tough decisions to restructure this critical industry. People’s pain should be factored into decisions to close and relocate plants, so that, for example, whole cities--such as Flint, Toledo, and Janesville--can make a smooth exit from over-reliance on this one industry. As part of that effort, urban planners should consider the infrastructure that remains behind, so that, for example, whole regions of a city are not suddenly deprived of hospitals, grocery stores, and other basic amenities. Imagine you’re on welfare and have to somehow go 10 miles to buy a loaf of bread.

Mapping tools are just tools, not salvation. Detroit long ago passed a point of no return. The US auto industry will never recover the manufacturing jobs that attracted people from all over the world to southern Michigan, northern Ohio, and other regions.

But we as a country should try to smoothe over these economic dislocations so that they don’t completely wipe some places off the map.

FORRESTER blog repost Inmon’s vitriolic slap at “virtual data warehousing” does not withstand scrutiny

Inmon’s vitriolic slap at “virtual data warehousing” does not withstand scrutiny

By James Kobielus

In a recent article, Bill Inmon incinerates a strawman concept that he refers to as “virtual data warehousing (DW).” For those unfamiliar with Inmon, he is generally considered the founder of DW as a data management discipline, has been at it since the 70s, and has more published books and articles to his name than most mortals. So he clearly may be considered an authority on the topic of DW.

But methinks Mr. Inmon doth protest too much on this “virtual DW” bugaboo, however defined (we’ll get to that in a moment). Also, he attacks this concocted notion with such emotional vehemence that it’s clear he considers it a threat to the centralized EDW paradigm upon which he has built his career and reputation.

For starters, his definition of this concept is oddly vague and questionably narrow: “a virtual data warehouse occurs when a query runs around to a lot of databases and does a distributed query.” Essentially, Inmon defines “virtual DW” as the ability to a) farm out a query to be serviced in parallel by two or more distributed databases, b) aggregate and join results from those databases, and c) deliver a unified result set to the requester.

That’s an important query pattern, but not the only one that should be supported under (pick your quasi-synonym) data federation, data virtualization, or enterprise information integration (EII) architectures. Inmon’s definition excludes the many federated queries that may only hit on a single database, with no joins and results aggregation, and with the EII fabric handling the necessary on-demand transformation from that source’s schema to an abstract semantic model.

Per my data federation report from last fall, Forrester has a broader perspective on the topic than does Mr. Inmon. Data federation is any on-demand approach that queries information objects from one or more sources; applies various integration functions to the results; maps the results to a source-agnostic semantic-abstraction model; and delivers the results to requesters. Nothing in the scoping of data federation necessarily requires the multi-source aggregation and joining that Inmon puts at the heart of “virtual DW.”

Putting Inmon’s narrow scoping of “virtual DW” behind us for the moment, let’s consider his chief objections to this approach. First, it requires the “analyst to integrate data” (as if that’s something analysts are ill-suited for or regard as some inordinate burden). Second, it consumes resources, experiences suboptimal performance, and “shuffles a lot of data around the system that otherwise would not need to be moved” (as if centralized DWs don’t consume resources, experience performance bottlenecks, and move data). Third, it is “limited to the [historical] data found in the [source] databases.” Fourth, it suffers from “no reconcilability of data...[hence] no single version of the truth for the corporation.”

It’s a fairly straightforward matter to dispatch these objections:

First, data integration--through ETL, EII, and other approaches--is a core job function for DW professionals, not some alien function outside their core competency.

Second, data federation is often the optimal approach for low-latency BI (just check out the case studies in my data federation and really urgent analytics reports). Federated environments can be tuned to provide top-notch performance and minimize source-system impacts when “shuffling” data around in a decentralized fabric.

Third, the source databases in a federation environment often include DWs, which, per their core function, usually manage a considerable amount of historical data. Once again, see my data federation report with discussion of case studies for a) Federation of Local DWs via Centralized EII Infrastructure and b) Federation of Dispersed EDW and ODS Data Into Siloed BI Environments.

Fourth, data federation is not totally incompatible with data reconciliation. In fact, federation environments can be architected for single version of the truth, data governance, and master data management. However, it can indeed be tricky to manage data quality in federated environments (see Rob Karel’s coverage of MDM and DQ for a deep dive on that issue).

My basic objection to Inmon’s line of discussion is that he treats data federation as mutually exclusive from the enterprise DW (EDW), when in fact they are highly complementary approaches, not just in theory but in real-world deployments. Yes, data federation can be deployed as an alternative to traditional EDWs, providing direct interactive access to online transactional processing (OLTP) data stores. However, data federation can also coexist with, extend, virtualize, and enrich EDWs, as well as other data-persistence nodes such operational data stores (ODS) and online analytical processing (OLAP) data marts. The case studies in the cited reports bear that out.

Inmon’s arguments are worth consideration. The centralized EDW model he touts is useful for illuminating some traditional best practices. But by no means can it do justice to the stubbornly heterogeneous, distributed, mixed-latency BI and DW requirements of most enterprises.

FORRESTER blog repost Lean Information Management Strategies for Lean Times

Lean Information Management Strategies for Lean Times

By James Kobielus

When the going gets tough, the tough get lean, focused, and flexible. To help organizations survive the bad times and thrive in all climates, their information management initiatives must remain agile and adaptable.

If you feel your information management strategy is anything but lean, you’re not alone. Many organizations struggle to gain control over information infrastructures that have become too bloated, rigid, and slow to realign with new business drivers.

Lean information management practices are essential for corporate survival. They are far more than belt-tightening exercises. They also help you build analytic muscle for excelling in any business environment. Here are some basic pointers for keeping your information management strategy lean:

• Trim your information infrastructure of excess cost. Lean means you should cut excessive, budget-busting overhead from your information management environment. Careful cuts are best, because they optimize your existing operations without gutting the core information, analytics, and applications that underpin your core competencies. Silo, server, database, and application consolidation should be your principal approaches. Also, you should re-evaluate vendor-sourcing strategies and renegotiate licenses at more favorable terms. And you should investigate lower-cost alternatives, such as software-as-a-service, to address business intelligence, business performance solutions, enterprise data warehousing, master data management, enterprise content management, and other information management requirements.
• Fit information initiatives to key business imperatives. Lean also means you fit, focus, and fully align your information management initiatives to mission-critical business imperatives. Strategic alignment ensures that you leverage information assets across diverse application domains and business processes, rather than allow that intelligence to languish underutilized in silos. To sustain this approach, you should establish an information management framework, such as a Business Intelligence Solution Center, that enables ongoing collaboration between business and IT stakeholders. You should engage all key business and technical groups in information management planning discussions.
• Flex information architectures to changing circumstances. Finally, lean means maintaining an approach that is flexible and adaptable, able to shift course as your needs and environment change. In yoga terms, lean is all about building, toning, and stretching analytical muscle to keep it from tearing when you need to transition rapidly from one strategic alignment to the next. You need the flexibility to swing between centralized information management infrastructures and decentralized or federated environments. For end-to-end data management environments, Forrester has developed an architecture decision support tool that helps information managers to determine which of several topologies is best suited to their needs: centralized enterprise data warehouse, hub-and-spoke, independent data marts, data federation, and information-as-a-service.

Considered as a comprehensive strategy, these lean practices are true bloat-busters and recession-beaters. They allow organizations to deliver practical insights that address all pain points, even--especially!!!--within strict budgets.

What if you had the voice of Jim Cramer inside your head screaming at you while you were trying to make sense of your personal investment portfolio?

All:

Speaking of maddening echo chambers, has anybody noticed that that's what the US television industry has become in recent years, with the proliferation of cable channels, the 24-hour news channels, the steady stream of talking--AND SCREAMING--heads?

Thank you Jon Stewart for tearing a new one in the hide of one of the most obnoxious--and dangerous--of those SCREAMING HEADS: Jim Cramer of CNBC. That Cramer is one dude that I've been avoiding from the start--partly due to simply how he looks, but more to the point how he talks and acts on the tube. In every way, this guy has always struck me as pure huckster, hypester, trickster. That, plus the fact that I find CNBC and all other business and news channels almost unwatchable--due in part to the overcluttered screens with TOO MUCH INFORMATION, including, especially, those obnoxious crawlers--which Bloomberg, in particular, stacks so deep that I get deeply claustrophobic just switching past that channel.

Anyway, re Cramer on the "Daily Show" recently, I just found a transcript of that exchange between him and Stewart, and also watched the video for the first time. I'll give most of the rest of this post over to the best of Stewart's rants. They're simply too perfect and summarize my feelings exactly:

*************************

"JS: But why, when you talk about the regulators, why not the financial news network? That's the whole point of this. CNBC could be an incredibly powerful tool of illumination for people that believe that there are two markets. One, that has been sold to us as long term. Put your money in 401k's, put your money in pensions and just leave it there, don't worry about it. It's all doing fine. Then there's this other market, this real market that's occurring in the back room. Where giant piles of money are going in and out, and people are trading them and it's transactional and it's fast, but it's dangerous. It's ethically dubious and it hurts that long term market. So what it feels like to us, and I'm speaking purely as a layman, it feels like we are capitalizing your adventure by our pension and our hard earned.. and that it is a game that you know, that you know is going on. But that you go on television as a financial network and pretend isn't happening."

"JS: But the gentleman in that uh, uh, video is a sober rational individual. And the gentleman on Mad Money is throwing plastic cows through his legs and shouting "sell, sell, sell". Then coming on two days later and going "I was wrong, you should have bought". Like, I can't reconcile the brilliance and knowledge that you have of the intricacies of the market, with the crazy b****** I see you do every night."

"JS: I gotta tell you. I understand that you want to make finance entertaining, but it’s not a f---ing game. When I watch that I get, I can’t tell you how angry it makes me because it says to me, “You all know.” You all know what’s going on. You can draw a straight line from those shenanigans to the stuff that was being pulled at Bear and at AIG and all this derivative market stuff that is this weird Wall Street side bet."

"JS: No, no, no, no, no. I want desperately for that, but I feel like that’s not what we’re getting. What we’re getting is… Listen, you knew what the banks were doing and yet were touting it for months and months. The entire network was and so now to pretend that this was some sort of crazy, once-in-a-lifetime tsunami that nobody could have seen coming is disingenuous at best and criminal at worst."

" JS: It’s very easy to get on this after the fact. The measure of the network, and the measure of mess. CNBC could act as—No one is asking them to be a regulatory agency, but can’t—but whose side are they on? It feels like they have to reconcile as their audience the Wall Street traders that are doing this for constant profit on a day-to-day for short term. These guys companies were on a Sherman’s March through their companies financed by our 401ks and all the incentives of their companies were for short term profit. And they burned the f---ing house down with our money and walked away rich as hell and you guys knew that that was going on."

"JS: But isn’t that part of the problem? Selling this idea that you don’t have to do anything. Anytime you sell people the idea that sit back and you’ll get 10 to 20 percent on your money, don’t you always know that that’s going to be a lie? When are we going to realize in this country that our wealth is work. That we’re workers and by selling this idea that of “Hey man, I’ll teach you how to be rich.” How is that any different than an infomercial? "

**************************

Me back again: "How is that any different than an infomercial?" Ah...hee hee...picture the sprayed-on-beard-face of pitchman infomercial screamer Billy Mays. Now, and I know this is painful, replay Mays' voice in your head. Now replay Jim Cramer's voice.

Same guy, right? How would you like those guys to take up permanent residence in your psyche...especially when you're trying to manage your finances in a rational manner?

Hard, right? Annoying, right? Of course, Mays just tries to sell isolated consumers harmless bullshit. Cramer pretends that he's advising the country on how to manage our collective investment portfolio.

Harmless?

Jim

Did the media in the 1930s obsess over the Great Depression the way we obsess over this current recession?

All:

Talking out of school here, but are you as tired as I am by the echo-chamber of media commentary on the current recession, on how long it is going to last, how deep it will be, how painful it is for so many people, how we can cope with it, and so forth?

Not to minimize all these serious matters, but sometimes it feels like a sore hurts much more and lasts a hell of a lot longer if all you ever do is obsess over it, dwell on it, pick at it, bandage it, unbandage it, rebandage it, comment on it, apply various treatments to it, worry that it's cancer, and the like. The queasy middle ground between prudent attention to one's own health and self-fulfilling hypochondria.

Did people in the 1930s, crappy as that era was, regard that particular business-downturn slump as the existential apocalyptic horse latitudes that seems to frame all current discussions of the current period? Or did they dub it a "great" depression only in retrospect? Just curious.

Culture has changed a lot since then, and our expectation of self-regulating economic cycles makes us more nervous and less risk-tolerant than ever. Great Depressions nowadays seem like they should be amenable to Great Anti-Depressants, and/or Great Stimulants.

Prozac Culture. Viagra Culture. Starbucks Culture.

Jim

imho Semantic Web Grounds SOA in a World of Meanings

By James Kobielus

Semantics is just a fancy word for understanding what things truly mean.

In distributed IT environments, semantic interoperability enables applications to understand the precise meaning of each piece of data that they import, acquire, retrieve, and otherwise receive from elsewhere. Without a transparent view into the semantics of externally originated content, applications cannot know how to validate, map, transform, correlate, and otherwise process that information without garbling its meaning.

Semantic interoperability is and always has been one of the principal tasks in real-world integration projects. Typically, it requires sweat equity by business analysts and data architects, who must define mappings to ensure that meaning is not lost or misconstrued when data is transformed to the requisite schemas of target applications. This can be a complex, error-prone exercise, because separate application domains often use different data syntaxes, schemas, and formats to describe semantically equivalent entities, such as a particular customer’s various records or a specific product’s multifarious descriptions.

Complicating the integration process is the fact that application domains rarely describe their semantics—in other words, the entity-relationship conceptual models that inform their data structures—in any formal or consistent way. Furthermore, relational data structures can be frustratingly opaque to developers who are trying to associate a complex set of linked tables with a coherent, business-level conceptual model. Integration specialists must often infer semantics from sketchy documentation, and then create cross-application data mappings that are based on those inferences.

What is the Semantic Web?

In an ideal world, semantics standards would be implemented universally, thereby accelerating, automating, and tightening semantic integration among heterogeneous environments.

Semantic Web refers to a long-running industry initiative that is working toward this ambitious goal. The vision of a Semantic Web has been percolating within the service-oriented architecture (SOA) community since the 1ate 1990s. It has been promoted primarily by World Wide Web (WWW) inventor Tim Berners-Lee. And it continues to be developed through a formal activity of the World Wide Web Consortium (W3C), which Berners-Lee heads.

At heart, Semantic Web is a vision for how the WWW should evolve to realize its full potential (indeed, some industry observers have taken to calling it “Semantic SOA” or “Web 3.0”). Since its birth in the early 1990s, the WWW has transformed the Internet into an open book that—through common interoperability standards such as HyperText Transfer Protocol (HTTP), HyperText Markup Language (HTML), and Extensible Markup Language (XML)—allows content everywhere to be available, readable, searchable, and comprehensible to human consumers. The Semantic Web initiative extends that concept to include non-human consumers. Organizations can implement W3C-developed semantics standards—such as Resource Description Framework (RDF) and Web Ontology Language (OWL)--to make the meaning of content unambiguously comprehensible to services, applications, bots, and other automated components.

Nevertheless, people vary widely in how they interpret the scope of the Semantic Web initiative, and the market is swarming with a wide range of projects, products, and tools that implement different variants of this vision. In the broadest perspective, Semantic Web may be understood as referring to an all-encompassing metadata, description, and policy layer that enables universal, automatic, comprehensive end-to-end interoperability across every macro or micro entity—including data, components, services, applications, and services—on every conceivable level. At its most down-to-earth, though, Semantic Web is usually construed as the ability to associate structured data with controlled, application-domain-specific conceptual models known as “ontologies.”

The potential benefits of semantic interoperability fall into several application domains:

• Enterprise content management (ECM): Semantic approaches can support more powerful discovery, indexing, search, classification, commentary, and navigation across heterogeneous stores of unstructured and semi-structured content. Semantic search—driven by concepts, not mere text strings--is regarded by many as the potential killer application of Semantic Web technology. Indeed, many Semantic Web vendors are primarily implementing the technology in search engines that leverage ontology-based concepts to improve search accuracy and reduce spurious hits.
• Enterprise information integration (EII): Semantic approaches enable consolidated viewing, query, and update of structured data that has been retrieved from diverse sources. Indeed, most commercial EII environments present an abstract semantic layer that mediates access to heterogeneous data, such as enterprise resource planning (ERP) and customer relationship management (CRM) applications, converging it all to a common presentation-side schema. A handful of those EII vendors—including BEA and Red Hat/MetaMatrix--have begun to support Semantic Web standards, primarily through third-party software plug-ins.
• Enterprise service bus (ESB): Semantic approaches can facilitate multilayered application, process, and service interoperability across disparate environments. To date, there has been little production implementation of Semantic Web standards in the ESB arena, though vendors such as Telcordia Technologies have adopted semantics, ontologies, and RDF to describe the conceptual models implemented by application endpoints, agents, and intermediary nodes within ESB-like middleware approaches such as event stream processing (ESP).

To some degree, the Semantic Web community is also loosely associated with Web 2.0 “social bookmarking” or “folksonomy” initiatives such as Del.icio.us, Digg, and Reddit, which provide online communities within which users may collectively link, tag, classify, and comment on Web content originated elsewhere (however, usually without reference to W3C specifications). The key difference between the Semantic Web and these folksonomy efforts is that the former relies primarily on professional developers to create and maintain standards-based ontologies, whereas the latter relies on end users to create informal, non-standard collections of descriptive tags applying to content they find while surfing the Web.

What are the Principal Standards and Approaches for Implementing the Semantic Web?

On the standards front, the Semantic Web vision is starting to bear fruit, slowly but inexorably.

In the past year, there has been an upsurge in industry attention to the W3C’s Semantic Web activity, due in part to the growing realization that SOA-based interoperability demands attention to semantics issues. To date, W3C-developed Semantic Web specifications—most notably, RDF and OWL—have begun to gain significant traction in commercial products. Startups continue to emerge, offering ontology modeling tools, inference engines, RDF repositories, and other necessary components of Semantic Web solutions. And more and more users are incorporating semantics-based approaches in their search, text analytics, ECM, EII, and other mission-critical applications.

At the heart of Semantic Web environments is the notion of ontologies, which are conceptual models comprising entity-relationship statements that have been expressed in a “knowledge representation language.” For Semantic Web, the principal knowledge representation language is RDF, which is an official W3C Recommendation. RDF uses XML to define a rich data model, syntax, and vocabulary for the exchange of machine-understandable ontologies about URI-designated resources. Within an RDF ontology, statements consist of well-defined “subjects,” “predicates,” and “objects.” For example, in the statement “This BCR article has an author whose value is James Kobielus,” the subject is “This BCR article,” the predicate is “has an author,” and the object is “whose value is James Kobielus.” Under RDF notation, each of these “nodes” is designated with its own unique URI, and a syntactically complete statement can be created by concatenating subject, predicate, and object node URIs into a single structure called an “RDF triple.”

RDF is the core specification in a growing range of Semantic Web standards and specifications under W3C, including:

• OWL: This specification, which is an official W3C Recommendation, extends RDF to support richer description of resource properties, classes, relationships, equality, and typing.
• SPARQL Query Language for RDF: This specification, which is currently a W3C Candidate Recommendation, leverages XQuery and XPath to support queries across diverse RDF data sources.
• Gleaning Resource Descriptions from Dialects of Languages (GRDDL): This specification, which is currently a W3C Candidate Recommendation, specifies how an XML document can be marked up to declare that it includes RDF-compatible data and also to specify links to algorithms--typically represented in Extensible Stylesheet Language Transformations (XSLT)--for extracting this data from the document.

At the very least, all Semantic Web implementations use RDF as their core ontology language, though many also support OWL for its semantic richness (and a growing number are implementing SPARQL, GRDDL, and related W3C specifications). Ontologies figure into Semantic Web environments in any of the following scenarios:

• Semantic modeling: In this scenario, developers explicitly model semantics as RDF/OWL ontologies, and/or as such related logical structures as taxonomies, thesauri, and topic maps. The ontologies are used to drive creation of structured content that instantiates the entities, classes, relationships, attributes, and properties defined in the ontologies. This is the classic model of greenfield development of application data under the Semantic Web paradigm.
• Semantic mediation: In this scenario, developers explicitly model semantics as RDF/OWL ontologies, and use the ontologies to drive the creation of mappings, transformations, and aggregations among existing, structured data sets. This describes the typical use of Semantic Web approaches within heterogeneous EII and other data integration environments.
• Semantic mining: In this scenario, developers use natural-language processing (NLP) and pattern-recognition tools to extract the implicit semantics from unstructured text sources. The extracted entities, relationships, facts, sentiments, and other artifacts are used to fashion RDF/OWL ontologies that drive the creation of indices, tags, annotations, and other metadata that layer a consistent semantic structure across the various items within an unstructured text store. This describes the typical use of Semantic Web in search and text mining/analytics environments.

To sustain an ontology-centric Semantic Web environment, the following functional components are necessary:

• Semantic tools: Application developers require a broad range of tools to help them work with ontologies, taxonomies, thesauri, topic maps, and other semantic constructs. Developers need tools to discover, query, browse, analyze, visualize, model, design, edit, classify, and annotate semantic constructs. They also need tools to map among dissimilar ontologies, define transformation rules, and attach descriptive tags and metadata. Tools should support semantics development by individual developers or collaborative teams. And semantics tools should integrate with Eclipse and other common development platforms, and support visual development in Unified Modeling Language (UML) and other modeling frameworks.
• Semantic engines: Application environments require runtime components to mediate interactions among semantic-aware components, and also to interface with legacy systems. Runtime semantic engines should support such functions as validating ontologies against standards; matching, mapping, transformation, correlation, and merging of data to conform with standard ontologies; and inference-based extraction of implicit ontologies from unstructured text sources. Semantic inference engines should support deterministic mapping across ontologies, as well as fuzzy equivalence-matching between extracted entity-relationship models and concepts specified in formal ontologies.
• Semantic repositories: Application environments require repositories or libraries to manage ontologies and other semantic objects, and also to maintain the rules, policies, service definitions, and other metadata to support life-cycle management of application semantics. Semantic repositories should support storage, synchronization, caching, access, import/export, registration, archiving, backup, and administration of ontologies and the data that instantiate those ontologies. The most prevalent semantic repositories are “RDF-triple store” databases.
• Semantic controls: Application environments require that various controls—on access, change, versioning, auditing, and so forth—be applied to ontologies (otherwise, it would be meaningless to refer to ontologies as “controlled vocabularies”). Controls might be enforced at the repository-, engine-, and/or tool levels. Developers might be constrained by the corporate-standard semantic tool to only use particular standard ontologies, which could vary depending on the type of application or project on which they’re working. To the extent that developers work in teams, the semantic-application development tool might provide a role-based workflow to structure interactions in accordance with best practice.

Who are the Major Semantic Web Solutions Vendors?

In the marketplace, the Semantic Web community is spawning a expanding group of promising startups, as well as some tentative commitments by larger, established software vendors.

It’s no surprise that academic research institutions and open-source communities play a substantial role in catalyzing the development of the Semantic Web. Coordinating semantics projects are such communities as Advanced Knowledge Technologies, Digital Enterprise Research Institute, Gnowsis, Rx4RDF, and SemWebCentral.

As befits an embryonic market pushing a bleeding-edge technology, many Semantic Web vendors are in fact consultants pursuing ontology-based projects in ECM, EII, ESB, and other areas. In fact, many Semantic Web vendors are attempting to jumpstart a self-sustaining software business from a handful of consulting jobs. Still, there are many semantics firms that make their living primarily from consulting and other professional services engagements. These firms include Articulate Software, Business Semantics, EffectiveSoft, Mindful Data, Pragati Synergetic Research, Semantic Arts, Semantic Light, Taxonomy Strategies, and Zepheira.

As noted earlier, many software vendors are seeking the low-hanging commercial fruit of semantic search. The growing list of semantic search engine vendors includes Aduna, AskMeNow, ChaCha, Cognition Technologies, Copernic, Endeca, FAST Search and Transfer, Groxis, Hakia, Intelliseek, ISYS Search Software, Jarg, Metacarta, Ontosearch, Powerset, Readware, Semaview, Siderean, Syntactica, Textdigger, Vivisimo, and ZoomInfo. Most of these vendors rely heavily on NLP, pattern-matching, and text analytics to power the semantics-aware crawlers that they deploy to extract ontologies from unstructured text throughout the Web, intranets, and other content collections.

Just as important, Semantic Web pure-play vendors have come into their own. Dozens of vendors offer flexible, sophisticated solutions that can support a wide range of semantics-aware applications in addition to search. Pure-plays in this space include Access Innovations, Axontologic, Cycorp, Fourthcodex, DATA-GRID, Franz, LinkSpace, Metatomix, Modus Operandi, Mondeca, Ontology Works, Ontopia, Ontoprise, Ontos AG, Revelytix, Sandpiper Software, SchemaLogic, Semagix, Semandex Networks, Semansys, Semantic Insights, Semantic Research, Semantra, Semtation GmBH, Teragram, Thetus, TopQuadrant, Visual Knowledge, Wordmap, and XSB.

Semantic Web vendors vary widely in their functionality, development interfaces, deployment flexibility, and standards support. None of these vendors are staking their success on rapid, universal adoption of the full stack of Semantic Web standards. Instead, they all provide tools, platforms, and applications that can be deployed for tactical, point, quick-payoff IT projects. They address specific business needs with their solutions while enabling customers to integrate semantics solutions to varying degrees with their existing application and middleware infrastructures.

What follows are snapshots of a handful of these vendors, illustrating their diverse backgrounds, approaches, and business models:

• Cycorp: Headquartered in Austin TX, Cycorp develops turnkey solutions in artificial intelligence, knowledge representation, machine reasoning, NLP, semantic data integration, information management, and search. Its Cyc middleware combines an ontology (which has been placed in the public domain) with a knowledge base, inference engine, natural language interfaces, and semantic integration bus. The vendor offers a no-cost license to its semantic technologies development toolkit to the research community. In the Semantic Web arena, Cycorp is doing R&D into scenarios in which end users create lightweight local ontologies that are subsequently elaborated, enriched, and mapped to more formal global ontologies by semantic inference engines.
• Sandpiper Software: Headquartered in Los Altos CA, Sandpiper Software provides semantics tools, consulting, and training. Its Visual Ontology Modeler (VOM) 1.5 tool supports component-based ontology modeling through frame-based knowledge representation. VOM, an add-in to IBM Rational Rose, leverages UML to capture and represent knowledge unambiguously. VOM supports RDF/OWL-based modeling of domain, interface, process, and user ontologies. As a subscription service, Sandpiper also offers the Medius Ontology Library, which extends VOM’s bundled ontology libraries to include application-specific ontologies plus utility ontologies for national, international, and general metadata standards.
• SchemaLogic: Headquartered in Kirkland WA, SchemaLogic provides an SOA-based business semantics middleware suite, as well as semantics consulting and training services. The company’s SchemaLogic Enterprise Suite includes server components that gather, create, refine, reconcile, and distribute ontologies, taxonomies, tag libraries, and other semantic metadata to subscribing applications over a real-time pub-sub integration fabric. The suite includes a governance layer that supports collaborative, Web-based participation and feedback by users and subject matter experts in the creation and refinement of business semantics. Collaborative semantic governance may span organizational boundaries, with the resultant semantic artifacts capable of being propagated automatically to third-party search engines, content management applications, portals, and other systems. For example, customers can use SchemaLogic Enterprise Suite to synchronize content categories and descriptions across distributed deployments of Microsoft Office SharePoint Servers.
• TopQuadrant: Headquartered in Alexandria VA, TopQuadrant is a software vendor that provides an open Java-based platform for development of Semantic Web applications. The TopBraid Suite includes tools and components for building ontologies; developing inference rules and SPARQL-based queries; collaboratively creating and browsing RDF-enabled content; extracting semantics from various data sources via GRDDL and other interfaces; mediating between RDF/OWL and other formats; displaying rich model-driven user interfaces; configuring and orchestrating semantic inference operations; and storing ontologies in third-party RDF triple-store databases. The suite supports browser-based access, collaborative semantic governance, and ontology-based search.

As noted earlier, some Semantic Web vendors are partnering with established EII vendors to offer ontology-aware semantic-integration layers for federated data query/update. These vendors are:

• Modus Operandi: This vendor’s Wave Semantic Data Services Layer product integrates with BEA’s EII solution--AquaLogic Data Services Platform (ALDSP)—via RDF/OWL ontologies. In so dong, it enables semantic integration of information across diverse, dispersed corporate applications, databases, and data warehouses. It supports user-driven ad-hoc semantic search and query, relying on ontologies to reconcile semantic conflicts among heterogeneous data. It also incorporates runtime services to crawl and index data services, to visualize the integrated data, and to monitor data services status. Modus Operandi’s ontology development tool can be launched from within BEA WebLogic Workshop, and can also import any standard OWL ontology developed in external tools. The tool deploys Wave semantic data services directly to ALDSP running on BEA’s WebLogic Server.
• Revelytix: This vendor’s MatchIT integrates with the semantic data services layer in Red Hat/MetaMatrix’s EII environment. MatchIT supports automated semantic mapping to help domain experts reconcile, map, and mediate semantics across heterogeneous environments via RDF/OWL ontologies. It provides an extensible ontology development tool that implements various sophisticated algorithms for determining semantic equivalence.

Some major data management vendors have begun to dip their toes in the Semantic Web market through solutions of their own. These vendors are:

• Oracle: Released in July 2005, Oracle Spatial 10g Release 2 provides a data management platform for RDF-based applications, supporting new object types to manage RDF data in Oracle. Based on a graph data model, RDF triples are persisted, indexed and queried, similar to other object-relational data types. The Oracle 10g RDF database ensures that application developers benefit from the scalability of the Oracle database to deploy scalable semantic-based enterprise applications. Metatomix, Ontoprise, and TopQuadrant have all announced support for Oracle Spatial 10g Release 2.
• IBM: Downloadable from vendor’s AlphaWorks site, IBM Integrated Ontology Development Toolkit supports storage, manipulation, query, and inference of ontologies and corresponding data instances. It includes an ontology definition metadata model, workbench, and repository. Its metamodel is a runtime semantics library that is derived from the OMG's Ontology Definition Metamodel (ODM) and implemented in Eclipse Modeling Framework (EMF). The Java-based workbench enables RDF/OWL ontology building, management, visualization, parsing, and serialization, plus transformation between RDF/OWL and other data-modeling languages. The repository, Minerva, is a high-performance DBMS optimized for OWL ontology storage, inference, and query, implementing a subset of SPARQL.

How Mature is the Semantic Web Market?

Even with all of this industry activity, the Semantic Web market is still far from mature. First off, RDF, OWL, and kindred W3C specifications have not exactly taken the SOA world by storm. One could not name a single pure-play vendor of Semantic Web technology that’s well-known to the average enterprise IT professional. And rare is the enterprise IT organization that’s looking for people with backgrounds in or familiarity with Semantic Web technologies. This remains a young, highly specialized niche in which academic research projects outnumber commercial products, and in which most products are point solutions rather than integrated features of enterprise databases, development tools, and application platforms. As noted above, no EII vendor has natively integrated Semantic Web specifications, and neither Oracle nor IBM has ventured much beyond their initial tentative forays into this new arena.
Commercial progress on the Semantic Web front has been glacial, at best, with no clear tipping point in sight. It’s been eight years since RDF was ratified by W3C, and more than three years since OWL spread its wings, but neither has achieved breakaway vendor or user adoption. To be fair, there has been a steady rise in the number of semantics projects and start-ups, as evidenced by growing participation in the annual Semantic Technology Conference, which was recently held in San Jose CA. And there has been a recent resurgence in industry attention to semantics issues, such as the recent announcement of a “Semantic SOA Consortium” involving Science Applications International Corporation (SAIC), and others. Some industry observers have even attempted to rebrand Semantic Web as “Web 3.0,” so as to create the impression that this is a new initiative and not an old effort straining to stay relevant.
Surprisingly, the SOA market sectors that one would expect to embrace the Semantic Web have largely kept their distance. In theory, vendors of search, ECM, EII, ESB, business intelligence (BI), database management systems (DBMS), master data management (MDM), and data quality (DQ) solutions would all benefit from the ability to automatically harmonize divergent ontologies across heterogeneous environments. But only a handful of vendors from these niches has taken a visible role in the Semantic Web community, and even these vendors seem to be taking a wait-and-see attitude to it all. One big reason for reluctance is that there are already many established tools and approaches for semantic interoperability in the SOA world, and the new W3C-developed approaches have not yet demonstrated any significant advantages in development productivity, flexibility, or cost.
One of the leading indicators of any technology’s commercial adoption is the extent to which Microsoft is on board. By that criterion, the Semantic Web has a long way to go, and may not get to first base until early in the next decade, at the very least. The vendor’s ambitious roadmap for its SQL Server product includes no mention of the Semantic Web, ontologies, RDF, or anything to that effect. So far, the only mention of semantic interoperability in Microsoft’s strategy is in a new development project codenamed “Astoria.” Project “Astoria,”, which was announced in May at Microsoft’s MIX conference, will support greater SOA-based semantic interoperability on the ADO.Net framework through a new Entity Data Model schema that implements RDF, XML, and URIs. However, Microsoft has not committed to integrating “Astoria” with SQL Server, nor is it planning to implement any of the W3C’s other Semantic Web specifications. Essentially, “Astoria” is Microsoft’s trial balloon to see if a Semantic Web-lite architecture lights any fires in the development community.
Clearly, there is persistent attention to semantic interoperability issues throughout the distributed computing industry. Microsoft is certainly not the only SOA vendor that is at least pondering these issues on a high architectural plane. Over the remainder of this decade, most major SOA, EII, DBMS, and BI vendors are going to make some strategic acquisitions in the Semantic Web community. Increasingly, leading enterprise platform, application, and tool vendors will integrate ontologies, inference engines, RDF-triple stores, and other semantics components and interfaces into their solutions.
But it may take another decade before the likes of IBM, Oracle, Microsoft, SAP, and other leading enterprise software vendors fully integrate semantics into all of their solutions. Until such time, we must continue to view the Semantic Web as an exciting but immature work in progress.
******
Original publication date: October 2007, Business Communications Review.

Author's note: I've posted this here because I'm tired of telling people about this great article I wrote on Semantic Web a couple of years ago for some now-defunct publication that practically nobody read. That wasn't very long ago and this still holds up quite well. Judge for yourself. Same principle applies with my poetry; it's more important to make your own audience than wonder why one never materializes. Better to self-publish than forever perish.

VENDORS MENTIONED IN THIS ARTICLE:
Access Innovations: http://www.accessinn.com/
Aduna: http://www.aduna-software.com/
Advanced Knowledge Technologies: http://www.aktors.org/akt/
Articulate Software: http://www.articulatesoftware.com/
AskMeNow: http://www.askmenow.com/
Axontologic: http://www.axontologic.com/
BEA: http://www.bea.com/
Business Semantics: http://www.businesssemantics.com/
ChaCha: http://www.chacha.com/
Cognition Technologies: http://www.cognition.com/
Copernic: http://www.copernic.com/
Cycorp: http://www.cyc.com/
Fourthcodex: http://www.fourthcodex.com/
DATA-GRID: http://www.data-grid.com/
Digital Enterprise Research Institute: http://www.deri.ie/
Endeca: http://www.endeca.com/
FAST Search and Transfer: http://www.fastsearch.com/
Franz: http://www.franz.com/
Gnowsis: http://www.gnowsis.org/
Groxis: http://www.groxis.com/
Hakia: http://www.hakia.com/
IBM: http://www.ibm.com/
Intelliseek: http://www.intelliseek.com/
ISYS Search Software: http://www.isys-search.com/
Jarg: http://www.jarg.com/
LinkSpace: http://www.linkspace.net/
Metacarta: http://www.metacarta.com/
MetaMatrix: http://www.metamatrix.com/
Metatomix: http://www.metatomix.com/
Microsoft: http://www.microsoft.com/
Mindful Data: http://www.mindfuldata.com/
Modus Operandi: http://www.modusoperandi.com/
Mondeca: http://www.mondeca.com/
Ontology Works: http://www.ontologyworks.com/
Ontopia: http://www.ontopia.net/
Ontoprise: http://www.ontoprise.de/content/index_eng.html
Ontos AG: http://www.ontos.com/de/company/index.php
Ontosearch: http://www.ontosearch.com/
Oracle: http://www.oracle.com/
Powerset: http://www.powerset.com/
Pragati Synergetic Research: http://www.pragati-inc.com/index.html
Readware: http://www.readware.com/
Revelytix: http://revelytix.com/
Sandpiper Software: http://www.sandsoft.com/
SchemaLogic: http://www.schemalogic.com/
Semagix: http://www.semagix.com/technology.html
Semandex Networks: http://www.semandex.com/
Semansys: http://www.semansys.com/
Semantic Arts: http://www.semanticarts.com/
Semantic Insights: http://www.semanticinsights.com/
Semantic Light: http://www.semanticlight.com/
Semantic Research: http://www.semanticresearch.com/
Semantra: http://www.semantra.com/
Semaview: http://www.semaview.com/
Semtation GmBH: http://www.semtation.de/
Siderean: http://www.siderean.com/
Syntactica: http://www.syntactica.com/
Taxonomy Strategies: http://www.taxonomystrategies.com/
Taxonomy Warehouse: http://www.taxonomywarehouse.com/
Telcordia Technologies: http://www.telcordia.com/
Teragram: http://www.teragram.com/
Textdigger: http://www.textdigger.com/
Thetus: http://www.thetus.com/
TopQuadrant: http://www.topquadrant.com/
Visual Knowledge: http://www.visualknowledge.com/index.html
Vivisimo: http://www.vivisimo.com/
Wordmap: http://www.wordmap.com/
XSB: http://www.xsb.com/
Zepheira: http://zepheira.com/
ZoomInfo: http://www.zoominfo.com/