Sunday, August 26, 2007

thenagin The SOA Hip-Hop Mixmaster Metaphor




This particular podcast, recorded on March 2 of this year, involved myself plus Messrs. Gardner, Garone, McKendrick, and Baer—and, accepting my request to participate for the first time, Dave Linthicum.

I invited Dave because, obviously, he’s a leading SOA industry analyst with plenty to say. But what prompted me to invite him at this particular juncture was a stimulating column he’d published in February on the topic of “mashup governance.” I was intrigued by the juxtaposition of anarchy (“mashup”) and control (“governance”) in that concept. I also liked the fact that “mashup”—a term from popular culture, in particular, from the world of sample-happy hip-hop music—is invading the world of SOA, highlighting the need to foster an x-factor of creativity, quirkiness, spontaneity, and fun in the crafting of reuse-happy composite services. Definitely had to have Dave on the call, and he’s been a semi-regular since (though he’s often tied up on Friday mornings with actual work, so it’s hard for him to set aside a late-in-the-week hour of probono jawboning—hard for the rest of us too, for that matter).

I’ll let him speak first, then me and the others. Dana started it off by asking Dave to explain how “mashup governance” ties into SOA and “Enterprise 2.0” (Jim now-note: I’m not endorsing that latter term—SOA is already nebulous enough—“Enterprise 2.0” takes SOA, mashes it up into the also-nebulous “Web 2.0,” and generates one of the most perfect semantic vacuums in today’s tech universe):

“Linthicum: …. That was a feature article, by the way, that InfoWorld sponsored, and it’s still up on their website. It basically talked about how mashups and SOA are coming together, since they are mashing up. As people are becoming very active in creating these ad-hoc applications within the enterprise, using their core systems as well as things like Google Maps and the Google APIs, some of the things that are being sent up by Yahoo!,, and all these other things that are mashable. There's a vacuum and a need to create a governance infrastructure to not only monitor-track, but also learn to use them as a legitimate resource within the enterprise. Right now, there doesn’t seem to be a lot of thinking or products in that space. The mashup seems to be very much like a Wild West, almost like rapid application development (RAD) was 15 years ago. As people are mashing these things up, the SOA guys, the enterprise architecture guys within these organizations are coming behind them and trying to figure out how to control it….[Y]ou really need a rudimentary notion of governance when you deal with any kind of application or service that works within the organization….[G]overnance is a management practice. It’s running around knocking people on their heads, if they are not using the correct operating systems, databases, those sorts of things. In the SOA world, as Joe McKendrick can tell you, it's about a technical infrastructure to monitor-control the use of services. Not only is it about control, but it is about productivity. I can find services. I can leverage services, and they are managed and controlled on my behalf. So, I know I am not using something that’s going to hurt me….The same thing needs to occur within the mashup environment. For mashing up, there are lots of services that we don’t control or that exist outside on the Internet. It's extremely important that we monitor these services in a governance environment, that we catalogue them, understand when they are changed, and have security systems around them, so they don’t end up hurting productivity or our existing IT infrastructure. We don’t want to take one step forward and two steps back.”

Now me, seconding everything that Dave and Dana had just said; adding a groaning new metaphor that I won’t extend here-now; mashing some 2-year-old Kobielus content into the mix {thanks, Dana, for linking to that article from your page, hence from here-now too); and then suggesting how we can enable a practical balance between creativity and governance in the SOA design-time:

“Kobielus: ….”[T]he whole notion of mashups is half-way to anarchy, as it were, creative anarchy. In other words, empowering end-users, subject-matter experts, or those who simply have a great idea. They typically slap together something from found resources, both internal and external, and provision it out so that others can use it -- the creative synthesis.

This implies that governance in the command-and-control sense of the term might strangle the loosey-goosey that laid the golden egg. So, there is that danger of over-structuring the design-time side of mashups to the point where it becomes yet another professional discipline that needs to be rigidly controlled. You want to encourage creativity, but you don’t want the mashers to color too far outside the lines.

Dave hit the important points here. When you look at mashup governance, you consider both the design-time governance and the run-time governance. Both are very important. In other words, if these mashups are business assets, then yes, there needs to be a degree of control, oversight, or monitoring. At the design-time level, how do you empower the end-users, the creative people, and those who are motivated to build these mashups without alienating them by saying, "Well, you've got to go to a three-week course, you've got to use these tools, and you've got to read this book and follow these exact procedures in order to mashup something that you want to do?" That would have clearly stifled creativity.

I did a special section on SOA for Network World back in late 2005. I talked to lots of best practice or use cases of SOA governance on design time, and the ones that I found most interesting were companies like Standard Life Assurance of Scotland. What they do is provide typical command-and-control governance on design time, but they also provide and disseminate through the development teams a standard SOA development framework, a set of tools and templates, that their developers are instructed to use. It's simply the broad framework within which they will then develop SOA applications.

What I am getting at here is that when you are dealing with the end users who build the mashups, you need to think in terms of, “Okay. Tell them in your organization that we want you to very much be creative in putting things together, but here is a tool, an environment, or enabling technology that you can use to quickly get up to speed and begin to do mashing up of various resources. We, the organization that employs you, want you, and strongly urge you, to use these particular tools if you wish your mashups to be used far and wide within the organization.

"If you wish to freelance it internally, go ahead, but doesn’t mean we are necessarily going to publish out those mashups so that anybody can see them. It means we are not necessarily going to support those mashups over time. So, you may build something really cool and stick it out there, but nobody will use it and ultimately it won’t be supported. Ultimately, it will be a failure, unless you use this general framework that we are providing."

Dana then suggested that we try not for force-fit existing SOA governance approaches to the new frontier of content-aggregating mashups under “Web 2.0” (yeah, I know, another x2.0 phrase in common currency, but this one has a bit more differentiated substance than “Enterprise 2.0,” so I can sorta live with it):

Gardner: I think we need to re-examine some of these definitions. I'm not sure what we are talking about with mashup governance is either "run time" or "design time." It strikes me as "aggregation time." Perhaps we don’t even need to use existing governance and/or even federate to existing governance. Perhaps it's something in the spirit of Web 2.0 and Enterprise 2.0, as simple as a wiki that everyone can see and contribute to, saying, “Here is how we are going to do our mashups for this particular process." Let’s say, it is a transportation process, "Here are the outside services that we think are worthwhile. Here are the APIs, and here is a quick tutorial on how to bring them into this UI." Wouldn’t that be sufficient?”

Steve Garone then swung the discussion back from mashup anarchy (the forbidding pole toward which Dana appeared to be mushing his SOA dogsled) toward the even icier antipodes of governance risk and compliance. Here’s Steve now, stating a set of concerns that are sure to warm the cockles of IP lawyers hearts everywhere:

“Garone: I am going to push back on that a little bit. What we are wrestling with here is achieving a balance between encouraging creativity and creating new and interesting functionality that can benefit business, and keeping things under control. The best way to look at that balance is to understand what the true risks are. The way I see it, there are several major areas. The first has to do with what I call external liability, meaning that if you, for example, publish a mashup to a customer base that has a piece of functionality you got off the web, and for some reason that has wrong information and does the customers some harm, who is responsible for that? How are you going to control whether that happens or not? The second has to do with what I call internal risk, which has to do with making available to the outside world information that is sensitive to your organization. In that case, a little more than what you described is going to be necessary, and can also leverage some of the governance infrastructure that people are building generally and relative to SOA…..[W]e all know that in this world anybody can sue for anything, and the reality is that if I go to a company’s website and use a function that incorporates something that they grabbed off the web, and it does me harm, the first place I am going to look is the site that I went to in the first place.”

After this cold dousing from Mr. Garone on our SOA mashup rave, Mr. McKendrick, being the professional analyst that he is, had to go and get all analytical and definitional on us all, asking what, precisely, distinguishes a “mashup” from more familiar SOA development constructs, such as “composite applications.” Take it from here Joe:

“McKendrick: …..[W]hat exactly is the difference between a mashup and a composite application that we have been addressing these past few years within the SOA sphere? The composite application is a service-level application or component that draws in data from various sources, usually internal to the organization, and presents that through a dashboard, a portal, or some type of an environment. It could be drawn from eight mainframes running across the organization. Obviously, the governance that we have been working so hard on in recent years to achieve in SOA is being applied very thoroughly to the idea of composite applications. Now, what is the difference between that and a mashup? Other than the fact that mashups may be introducing external sources of data, I really don’t see a difference. Therefore, it may be inconsistent to "let a thousand flowers bloom" on the mashup side and have these strict controls on the composite application as we have defined in recent years.”

Then Mr. Linthicum closed the loop very well, responding to Joe’s concern by stating that “mashup governance” is just another set of requirements that must be addressed under a comprehensive SOA governance approach—not through a stovepipe governance domain:

“Linthicum: The reality is that there is no difference. You are correct, Joe, and I point that out in the article as well. There are really two kinds of mashups out there: the visual mashups, which are what we are seeing today, where people are taking basically all of these interface APIs and using the notions of AJAX and other rich, dynamic clients, and then binding them together to form something that is new. The emerging mashups are non-visual. It's basically analogous, and is not exactly the same, as traditional composite applications that are -- if you can call them traditional -- in the SOA realm today. They have to be controlled, managed, governed, and developed in much the same way.”

But I then pushed back on this too-early consensus, arguing that--though we need a comprehensive governance environment for visual and non-visual mashup/composite applications--we also need to make sure that an enterprise’s governance environment clearly delineates which content/resources in which internal and external domains is available/kosher for end users (i.e., the primary “developers” in this new paradigm) to mashup:

"”Kobielus: There is a difference here. I agree with what Dave just said that mashups are not qualitatively different from composite apps, but there is a sort of difference in emphasis, in the sense that a mashup is regarded as being more of a user-centric paradigm. The end-user is empowered to mash these things up from found resources.

It relates to this notion that I am developing for a piece on user-centric identity as a theme in the identity management space. The whole Web 2.0 paradigm is user-centric -- users reaching out to each other and building communities, and sharing the files and so on. Mashing up stuff and then posting that all to their personal sites is very much a user-centric paradigm.

There's another observation I want to make. I agree that the intellectual property lawyers are starting to salivate by mashups invading their clients or encroaching on their clients’ rights. Actionable mashups are good from a litigator’s point of view. In terms of governance then, organizations need to define different mashup realms that they will allow. There might be intra-mashes within their Intranet -- "Hey, employee, you can mash up all manner of internal resources that we own to your heart’s delight. We will allow intra-mashes, even extra-mashes within the extranet, with our trusted partners. You can mash up some of their resources as well, whatever they choose to expose within the extranet. And then, in terms of inter-mash or Internet wide mashing, we’ll allow some of it. You can mash Google. You can mash the other stuff of the folks who are more than happy to let you mash. But, as an organization, your employer, we will monitor and block and keep you from mashing up stuff that conceivably we might be sued for."

“Intra-mash”? “Extra-mash”? “Inter-mash”? I just mashed those words together on the spur of the moment. No, they haven’t entered my everyday working vocabulary either. Anyway, Tony Baer came next, and had a cool linkage of what I just said to the notion of a “walled garden” for controlled-yet-spontaneous (yikes!) mashup within an enterprise environment:

“Baer: ….Composite apps, at least as I've understood the definition, came out of an SOA environment. That implies some structure there, whereas mashups essentially merged to Web 2.0 with the emergence of AJAX-style programming, which lets anybody do anything anywhere with this very loosely structured scripting language. There are practically no standards in terms of any type of vocabulary…..So, there is a bit of a "Wild West" atmosphere there. As somebody else said, you really need to take a two-tiered approach. On one hand, you don’t want to stifle the base of innovation, a kind of a skunk works approach. Having a walled garden there, where you're not going to be doing any damage to the outside but you are going to promote collaboration internally, probably makes some sense. On the other hand, even if the information did not originate from your site, if you're retransmitting it there is going to be some implication that you are endorsing it, at least by virtue of it coming under your logo or your website….

So, you need a tiered approach….You really need to exert control on the sources of information. Therefore, for the types of information that are exposed internally -- for example something from an internal financial statement -- you need to start applying some of the rules that you've already developed around internal databases. Different classes of users have a right to know and to see it and, in some cases, some read-write privileges.

You need to apply similar types of principles at the source of information. Therefore, if I have access to this, this means implicitly that I can then mash it up, but you have to really govern it at the original point of access to that information, at least with regard to internal information. With external information, it probably needs to go to the same type to clearance that you would exert for anything that goes out on the corporate website, the external website.”

Dana then had a brilliant observation on the need f or third-party high-quality Internet-mashable data sources that are generally trusted and reside in a litigation-free zone:

Gardner: …. I was thinking about the adage that nobody was fired for using IBM, which was a common saying not that long ago. What if we were to take that same mentality and apply it here -- that if you're going to do mashups, make sure they are Windows Live mashups, or Google mashup services for mashup; or maybe So, is there is an opportunity on the service provider side to come up with a trusted set of brands that the IT people and the loosey-goosey ad-hoc mashup developers could agree on to use widely? They could all rally around a particular set of de-facto industry standard services? That would be perhaps the balance we're looking for…..[T]he people who are providing the current set of internal services and/or traditional application functionality need to be thinking, "Shouldn’t I be out there on the wire with a trusted set?" We're already seeing Microsoft move in this direction with its Windows Live. We're seeing Google now putting packaging around business-level functionality for services. is building an ecology, not only of its own services, but creating the opportunity for many others to get involved -- you could call them SaaS ISV’s, I suppose. And I don’t think it’s beyond the realm of guesswork that Oracle and SAP might need to come up with similar levels of business application services that create what would be used as mashups that can be trusted to be used in conjunction with their more on-premises, traditional business applications.”

Dave then pointed out that there are already some SaaS-oriented third-party providers of business-oriented data that can—at a price—to leveraged to enrich, enhance, augment, mash, aggregate, and integrate into existing SOA apps. He mentions StrikeIron, whom both Baer and I have spoken with, and whom I agree is an important player in this growing space. But I’d also like to point out that the leading business intelligence (BI) vendor, Business Objects, has also committed publicly to this direction, under its “Information on Demand” strategy (see their press release from May 22 of this year). Now for what Dave said:

Linthicum: ….People are moving to use interface-based applications through software-as-a-service. All you have to do is look at the sales of to monitor how that thing is exploding. And, they are migrating over to leveraging services to basically mix-and-match things at a more granular level, instead of taking the whole application interface and leveraging those within your enterprise. This is what I call "outside-in" services. I wrote about that three years ago.

People are going to focus on that going forward, because it just makes sense from an economic standpoint that we leverage the best-of-breed services, which typically aren’t going to be built within our firewall. We don’t want to pay for those services to be built, but they're going to be built by the larger guys like, Google, and Microsoft. It's going to be a slow evolution over time, but I think we are going to hit that inflection point, where suddenly people see the value. It’s very much like we saw the value in the web in the early '90s -- that it really makes sense not only to distribute content that way, but distribute functional application behavior that way…..

You are going to see some aggregators out there. Right now, you’re seeing guys like StrikeIron, which is a small company, but they aggregate services. They are basically a brokerage house for services they control, validate, and make sure they are not malicious. Then, you rent the services from them, and they in turn pay the service provider for providing the service. I think Google is going to go for the same model.

Gardner: It’s about trust ultimately, right?

Linthicum: It’s about trust ultimately. If I were a consultant with an organization and my career was dependent on this thing being a success, I'd be more likely to trust StrikeIron and Google than some kind of a one-off player who has a single service which is maintained in someone’s garage.”

Then the conversation shifted abruptly to a then-just-recently announced corporate-level mashup: Oracle’s acquisition of rival BI, corporate performance management (CPM), and master data management (MDM) vendor Hyperion. What most distinguished this particular acquisition was Hyperion’s strong positioning in the financial CPM market, addressing the strategic planning, budgeting, consolidation, and reporting needs of CFOs. My prime concern then, and still, is the amount of functional overlap between Oracle’s and Hyperion’s products—making the decidedly non-hostile deal feel a bit like Siebel-acquisition-redux and PeopleSoft-acquisition-redux—in other words, merging, munging, and mashing a direct rival under the growing Oracle big top. What I said on that day, and still stand by:

Kobielus: It makes sense knowing Oracle. First of all, because [Oracle Chairman and CEO] Larry Ellison has been very willing in the past to grab huge amounts of market share by buying direct competitors like PeopleSoft, Siebel, and so forth, and managing multiple competing brands under the same umbrella -- and he is doing it here. A lot of the announcement from Oracle regarding this acquisition glossed over the fact that there are huge overlaps between Oracle’s existing product lines and Hyperion’s in pretty much every category, including the core area that Hyperion is best known for, which is financial analytics or Corporate Performance Management (CPM). Oracle itself provides CPM products for CFOs that do planning, budgeting, consolidation, the whole thing.

Hyperion is a big business intelligence (BI) vendor as well, and Oracle has just released an upgrade to its BI suite. You can go down the line. They compete in master data management (MDM) and data integration, and so forth. The thing that Oracle is buying here first and foremost is market share to keep on catapulting itself up into one of the unchallenged best-of-breed players in business intelligence, CPM and so forth. Oracle bought the number one player in that particular strategic niche, financial CPM , which is really the core of CPM -- the CFOs managing the money and the profitability.

It’s a great move for Oracle, and it definitely was an inevitable move. There will be continuing consolidation between the best-of-breed, pure-play data management players, such as Hyperion and a few others in this space, which are Business Objects and Cognos. They will increasingly be acquired by the leading SOA vendors. Look at the SOA vendors right now that don’t have strong BI or strong CPM, and look at the pure-plays that have those tools. The SOA vendors that definitely need to make some strategic fill-in acquisitions are IBM, Microsoft to a lesser degree, BEA definitely, and a few others, possibly webMethods. And, look at the leading candidates. In terms of CPM and BI and a comprehensive offering, they are down to three: Business Objects, Cognos, and SAS.”

Yes, there certainly will be more corporate mix-and-mashups in the SOA, BI, and CPM space—soon. Oh wait, there has been one noteworthy mashup since March: Software AG and webMethods.

Fall’s coming, I can just feel it, even through the late-August humidity.