Thursday, August 30, 2007

thenagin The SOA Literal Non-Metaphor




This podcast, recorded Friday, April 6, 2007, featured Messrs. Gardner, Garone, McKendrick, Kobielus, Baer, and Biske.

Let me see. We discussed Software AG’s recently announced intention to acquire webMethods. We got into Web 2.0 and SOA (e.g., wikis’ potential role in SOA governance!). We delved into “SOA and the Hype Curve.” There’s hype in this space? Really? Some would say it’s all hype, but, of course, there’s plenty of substance and substrate—which is why we have a never-ending stream of SOA-related industry announcements to discuss.

If SOA were all surface and no depth, then the metaphors that I’ve used as the motif for this blogpost thread would be pure affectations. But SOA, like most uber-important architectural IT topics, demands metaphors, similes, analogies, and other figures/handles (note these tactile nouns) of thought/speech to help us grasp (note this tactile verb) the choreography/behavior (note these anthropomorphic nouns) of their underlying applications, patterns, models, and so forth. The most important aspects of SOA, like IT in general, are either above the threshold of everyday perception (e.g., complex networked integration patterns distributed and virtualized over local, wide, wireless, and other “areas”) or below that threshold (i.e., the invisible execution, flow, and behavior of software, storage, CPU, memory, and other components within each and every node). In this environment, metaphors aren’t literary devices—they’re planning, design, engineering, and management tools.

SOA, like IT in general, is the most multidimensionally plastic of all human innovations, absolutely demanding agile metaphors to press it into the proper shapes. To the extent that I quote Andy Warhol in my normal rounds, it’s to second his statement that plastic is the new bedrock reality of the tech-mediated universe: “Everybody’s plastic. I love plastic. I want to be plastic.”

In terms of what SOA’s literal, non-metaphorical substrate might be, I took what I feel is my best whack at it in September 2004, when I published the following column (“SOA and the death of platforms”) in Network World. I will now quote and second my then-self at length:

“Kobielus: As high-tech buzz phrases go, service-oriented architecture is vaguer than most. SOA isn't a technology, product, service or protocol. But it's been mentioned in many marketing pitches lately, promising greater developer productivity and standards-based interoperability if you buy this vendor's application platform or that one's visual-modeling tool. Over the past three months, Sun, Oracle, Microsoft and IBM all announced products or initiatives for implementing SOA in their respective environments.

SOA is a substantial and growing body of practical techniques for designing shareable, reusable, interoperable Web services. Just as important, the past few years have seen the emergence of a universal, standardized, SOA-enabling middleware fabric built on the Web Services Description Language (WSDL), Simple Object Access Protocol (SOAP) and Universal Description, Discovery and Integration.

SOA is a disruptive approach to building distributed services. Until now, we've developed new functionality on and within concepts such as platform, application and language. Each of these concepts has traditionally had a well-defined sphere of reference: The platform hosted the application, and the application was developed in a language. Now all that is changing, thanks to the emergence of SOA.

The first of the old computing concepts to wither away will be the platform. This term originally applied to operating systems, then included application servers that implement a particular development framework (Java 2 Platform Enterprise Edition or .Net) over one or more operating systems. But the growth of standards-based, distributed Web services has made it clear that fewer and fewer business processes will execute entirely within the confines of a J2EE 1.3 server or Windows Server 2003, or Linux, but will execute across them all. When all platforms share a common environment for describing, publishing and invoking services, the notion of self-contained platforms disintegrates in favor of SOA, which is essentially a platformless service cosmos.

Another casualty of this evolution is the notion of applications as discrete, functional components that execute on particular platforms. SOA is founded on the notion of virtualization. Under this paradigm, services describe abstract interfaces within standard, platform-independent metadata vocabularies such as WSDL. The underlying service functionality may be provided from components on any platform without needing to change the interface. Under SOA, the application dissolves into a service that may have no fixed implementation but simply bids for on-demand networked software and hardware resources.

Programming languages also are becoming something that fewer developers touch directly. Visual model-driven development and automated code generation are at the forefront of the SOA revolution. You're more likely these days to see a vendor boast of its ability to support visual modeling in Unified Modeling Language than development in Java, C# or any other declarative programming language. For complex, orchestrated, multiplatform Web services, visual modeling is the most effective approach for specifying, implementing and maintaining the end-to-end logic and rules on which the service depends.

SOA has spawned a range of terms to describe what developers actually develop. IT professionals increasingly define their creations in terms of services, models and patterns, rather than platforms, applications and languages. The notion of patterns will become critical to discussions of distributed services. A pattern is a generic approach - such as service proxying or service coordination - to architecting interactions in the infrastructure. Every pattern defines its own abstract Web services functional elements and SOAP-based interactions.

Welcome to the dizzying new world of SOA. Platforms are dissolving, new concepts are taking over, and Web services will become everywhere shareable and reusable."

SOA as “platformless service cosmos”? I’d forgot I’d said that. Metaphor, sort of, or at least a visual image that suggests universality without trying to map it to stars, galaxies, quasars, etc. It’s abstract but tactile and tensile in the manner of any good figure of speech.

Solid plastic, all the way through.

Oh, back now to the April 6, 2007 podcast. Any good thought handles re plasticity in the SOA universe, at the technology, market, product, and/or vendor level?

On the increasing irrelevance of single-vendor all-purpose closed-source SOA suites in this platformless service cosmos:

McKendrick: Well, as we discussed in a previous podcast, the whole notion of an SOA suite runs counter to the SOA philosophy. SOA is especially about loosely coupling, and it doesn’t matter where the applications or the system resides. SOA should be independent of all that, and the idea of an SOA suite runs counter to that thinking. Nevertheless, we see lot of companies glomming onto each other, a lot of gelling taking place. This Software AG-webMethods merger is an example. Progress Software has been very astute in assembling a collection of companies that deal with different aspects of SOA. They want to compete with Oracles, and we saw that just recently with JBoss.

Gardner: SOA Software?

McKendrick: Yeah, right. SOA Software and JBoss. Their goal, their intention is to compete against these bigger guys, while servicing the smaller business market, of course.

Gardner: More of an ecology approach to how to bulk up.

McKendrick: Yeah, we’re going to see more of these alliances, more of these acquisitions and mergers.”

On wikis as a potential enabler for loose collaborative SOA governance in a platformless service cosmos:

“Gardner: …. I want to go first to Jim Kobielus, because in some emails this week you had some interesting thoughts. We’ve seen a few companies say, "Let’s leverage Web 2.0." We’ve seen Intel come out and say, "We’re going to corral a number of 'open-source' Web 2.0 functions." We’ve seen Cisco get involved with Web 2.0. We’ve seen BEA just announce an embrace of Web 2.0. Even IBM is tinkering with this, I’m sure.

So, there is something there, but let’s focus on this collaborative aspect and particularly in terms of governance. Annrai O’Toole at Cape Clear mentioned this to me probably about nine months ago that he looked at governance and at Web 2.0 and thought, "Gee, maybe wikis would be a good concept for how people manage their services." They could say, "I think the policy should be this, and I’m going to use it in this way, and you can pick and choose."

It's sort of an open source, open collaboration approach to policy and use of services and their agreements. I suppose provisioning rules would come about. You said, Jim, this is kind of a scary concept, that wikis seem to be anti-governance and that it could be a collaboration with no structure. Tell us a little bit about why you’re concerned?

Kobielus: Well, when I think of governance, like everybody I think of the capital "G," like Government, but governance has a broader concern. You often think about crack-the-whip, controls, and setting controls on how people interact and how policies are created? When I think about wikis, I think of the exact opposite. There are no controls. It’s basically a shared space to which everybody can post, everybody can overwrite, and everybody can erase everybody else’s comments.

Gardner: The wisdom of the crowd, right?

Kobielus: Well, okay, that’s a religious faith. Wiki seems like the wild, wild West. It’s a free-for-all as collaboration. That’s great. It’s definitely got a very valid role in many environments, like open-source initiatives where they are very peer oriented. Everybody is an equal, even-steven, participant. There is a high emphasis on collaboration, design, reciprocity, and all of that. So, when I think about the whole SOA governance space, both design-time governance and run-time governance, I think of wiki as in the design-time governance side, where you have teams of designers or business analysts and IT people sitting around the virtual table, trying to hash out policies.

Gardner: Requirements?

Kobielus: Requirements, policies, data designs, data hierarchies like with their master data management environment. Quite often these are creative processes involving teams of smart people who sit around a virtual table and hash out the overall design approach. Wikis and the whole Web 2.0 repertoire of collaborative tools can be very valuable in this upfront design, modeling, simulation, and shoot-the-breeze aspects that are critically necessary for design time. But runtime SOA governance really depends on clear-cut policies, designs, data definitions, and so forth that have been handed down by the policy gurus, and now are governing ongoing operations without ambiguity.

In that case, you don’t necessarily want any Joe Blow to be able to overwrite the policies and the business rules that are guiding the ongoing monitoring, management control, or security of your SOA.

Gardner: Yes. You said that they needed adult supervision, but I didn’t think that this would be open to anybody. I thought this would be open to the people who have impact, the architects and the line-of-business people perhaps. You're not going to open this up to anybody. There would be a directory and an provisioning, so that only those most close would have access…..

Biske: I don’t know that you really want a wiki-style collaboration for governance. I tend to agree with Jim that you can’t just open it up to the masses, and even if you look at collaborative environments, whether it’s the large open-source projects, or something like Wikipedia, there's some hierarchy that eventually was put in place, where certain people were allowed to do commits or were designated as senior editors.

So, you always wind up with some form of governance structure around that. The area where I think wikis are going to be important in the SOA space is in the service management lifecycle or service development lifecycle. You got companies that have to move to a service-provider model, whether it’s internally to internal consumers or externally. I have talked a lot about this in my blog.

If we compare that to traditional product management, for a long time it was really just one-way communication. The product marketers went out and said, "Here’s our product. Here are its features. Go and use it." They pushed it out to the market place, and the only response they got back was did people buy the product or not.

Over time, they recognized the need for much more collaboration with their end consumers on how to evolve that product, whether it was the formation of customer advisory boards or even leveraging the Internet and some of the technologies to establish a community around those products. The same thing can apply when adopting SOA.

If I am providing a service out to the rest of the enterprise, I am going to be interested in what the consumers of that have to say. If I am not listening to them, not establishing that bi-directional communication, eventually they are going to go somewhere else or they are going to build it on their own and put that redundancy back into the organization, which is the opposite of what we are trying to achieve.

Gardner: Okay. So social networking has a value here, but you wouldn’t want it necessarily hardwired into the way in which the technology actually operates?

Biske: Exactly.

Gardner: Okay, I can buy that. Does anyone else have any thoughts on this notion of wikis and collaboration as applied to SOA governance?

Garone: I don’t really have a lot to add. The two guys who just talked gave a great view of this particular question, and I agree with virtually all of it. After a while, it becomes something that a few key contributors, people who actually have the power, control, and knowledge, would be part of. I waxed philosophical in my head and asked, "Well, is it then still a wiki or is it in fact a collaborative tool among a set of decision-makers who, through policies themselves, are able to make changes or not?"

Gardner: I think you're right. The key word here is collaboration. I believe that IBM is going to be coming out soon with something called Jazz, which is a collaborative application lifecycle management approach. We are seeing more collaboration across different aspects of the whole development-to-deployment environment and we are looking for the tools to do that. Web 2.0 is perhaps stepping up and saying, "We have some tools that can be used in that way." Perhaps, it's not just a matter of dropping a wiki into the process, but something a bit more tailored to an aspect, but somehow availing all the participants of each others' wisdom in the process.

Kobielus: It’s like an open source project. You have a broad range of contributors, but only a handful of committers who can actually commit changes to the underlying code base. So, you might have a wiki that has potentially 3,000 different contributors, but ultimately there might be a moderator or two whose job it is to periodically weed out the nonsense, and crack the wiki whip to make sure that what’s actually been posted reflects the wisdom of the crowd of 3,000 people and not necessarily the vandalism of the few who decide to just disrupt the process.

Garone: And that’s a governance model in itself.

Gardner: Yeah, that’s right. It’s governance, and it relates back to what we’ve discussed in trying to find analogies in geo-politics for technology governance. You want the best of both. You want a federated approach, where they get grassroots input, but you also need command and control. So, it’s Jefferson and it’s Hamilton, right?

Kobielus: It’s Aaron Burr occasionally too. You’re going to follow the pistols.”

On the growing role of wetware as the underlying implementation of SOA-accessible resources in a platformless service cosmos:

Kobielus: Web 2.0 is really HOA, Human Oriented Architecture. It is pretty much giving human beings the tools to share what’s in their minds, to share their creativity with the big wide world. SOA, Service Oriented Architecture, is about sharing and reusing all matter of resources in a standardized way. HOA, the Web 2.0, is the most critical resource, and the most inexhaustible energy supply is human ingenuity and creativity.

Gardner: That’s like Cosmic 2.0. Woah!

Kobielus: Yeah, I have been known to get cosmic. People who have read my blog realize that I am a total space cadet.

Baer: Jim, I'll give you the award this morning for coining the best buzz words, like "Cracking the wiki whip" and "Human Oriented Architecture."

On the financial substrate—mergers, acquisitions, revenues, profits, etc--that continues to drive and shape all the SOA plasticity in a platformless service cosmos:

McKendrick: One effect of the whole compliance scenario was that it gave vendors another hype factor. I'm going to try another buzz word. How about Hype-Oriented Architecture? HOA. Using Jim’s HOA for another purpose.

Kobielus: It’s better to have hype-oriented accounting.

McKendrick: Hype-oriented accounting? I think the whole compliance thing gave a boost to the IT industry in the early 2000s. What do we call this decade anyway, the 2000s?

Gardner: The oughts.

McKendrick: It gave the whole industry a boost at a time when things were kind of down with the IT recession. The whole compliance scenario helped business intelligence and the data-environment vendors who directly addressed the flow of financial information.

Gardner: I’d like to conduct an experiment. I think we should take an accountant out to lunch. Anyone who knows an accountant, take them out to lunch and tell them how great IT is and what SOA can do in terms of long-term efficiency and lower total costs. Bring in some of the other mega trends, such as software as a service, virtualization, and data master management. It behooves us all to educate the accounts on why IT is important, because I think they are suffering from a lack of understanding.

Better yet, take a chief financial officer out to lunch and then take the accountants out to lunch. This is the crowd we need to work on. We keep talking about trying to convince the CEO and the CIO, and I think we need to get right down into the bean counters' frontal lobes on SOA.”

Convincing the wetware that controls the money that shapes the contours of this exploding plastic inevitable universe.

Lunch is about as real as it gets.