SOA: The TBDs
What still needs to be defined, developed, delivered, deployed, and demonstrated to realize SOA’s potential in the real world
--James Kobielus
SOA is clearly a work in progress. Actually, SOA is a target state that many organizations may approach but never fully attain. It’s all about achieving maximum reuse and minimum redundancy of services throughout complex, multiplatform distributed environments. Raising the bar even higher, the industry continues to elaborate the SOA vision by adding all of Web services, IT governance, enterprise service bus (ESB), business process management (BPM), model-driven development (MDD), and other hot topics to the bubbling brew.
TBD: SOA conceptual clarity
If SOA is going to endure as a useful paradigm, the IT industry will need to continue clarifying what it means by the term and why anybody should care. Given the restlessness of the IT industry, another paradigm may some day surface to steal SOA’s fire as the architectural buzzphrase du jour.
For sure, there’s no shortage of SOA whitepapers, consultant reports, position papers, articles, courses, books, and other prescriptive guidelines on the market. Every SOA consultant has their own vision of what the term means—or should mean—as does every IT vendor that has aligned its products and services with this paradigm. For enterprise IT professionals, it’s often difficult to map conceptually between different SOA visions presented in vendor materials, consultant reports, and press articles. At the same time, distinguishing between substantive SOA discussions and marketing fluff can also be a bit tricky.
Paradigms are slippery things to standardize, but that doesn’t stop well-meaning industry groups from attempting to do just that. The Organization for the Advancement of Structured Information Standards (OASIS) is one of the principal IT standardization groups these days, and it has established two technical committees (TCs) to clarify industry understanding of SOA approaches. In April 2004, OASIS created the Electronic Business SOA (ebSOA) TC. In February 2005, OASIS chartered the SOA Reference Model (SOA-RM) TC. The overlap among these groups’ charters is considerable, but the general division of responsibilities is as follows.
OASIS’ ebSOA TC is defining a reference architecture, guidelines, and best practices for implementing SOA within B2B environments that implement ebXML standards. The TC is also defining the ongoing roadmap for OASIS’ ebXML Technical Architecture, which is currently in version 1.04. The group is attempting to align the ebXML Technical Architecture with ongoing changes to the various ebXML standards, which address such B2B requirements as service brokering, reliable messaging, orchestration, and trading partner agreements. It is also addressing how ebXML standards will evolve to integrate more thoroughly with the growing range of Web services standards being defined at OASIS, W3C, and elsewhere. The ebSOA TC has promised its first draft deliverable, an architectural specification, for July 31, 2005. An ebSOA best practice document is promised for mid-2006.
OASIS’ SOA-RM TC is defining an SOA reference model that is broader in scope than that being worked out at the ebSOA TC. The SOA-RM TC is defining an abstract reference model that can encompass ebXML, Web services, and other implementation environments within which SOA may be applied. The group will release a first draft reference model for general industry comment by the end of 2005.
One of the SOA-RM TC’s principal goals is to define a set of SOA concepts, functional elements, architectural patterns, and best practices that can be applied unambiguously within and between different implementation environments. The TC has stated that ebXML and Web services aren’t the only possible SOA implementation environments that it is addressing. Per the group’s ongoing work, core SOA abstractions include services, contracts, policies, semantics, discovery, presence, and availability. In addition, the group is drawing an important distinction between SOA use cases of varying complexity:
• Simple SOA: This involves a single shared service on which there are no requirements for reliable messaging, transactional rollback, long-running orchestration, or quality of service.
• Intermediate SOA: This involves multiple shared services that are presented to consumers through a single “root” or “façade” service; are hosted and managed within the same administrative domain; and may require transactional rollback, long-running orchestration, and/or quality-of-service behind the service root.
• Complex SOA: This involves multiple shared services that are hosted and managed within one or more administrative domains, and that involve reliable messaging, transactional rollback, long-running orchestration, and/or quality-of-service policies to be enforced in interactions among services. Relationships among services may be hierarchical, parent-child, peer-to-peer, and other models.
In addition to the work that OASIS TCs have been doing to clarify SOA, vendors of all sorts have been pitching their SOA frameworks into industry circulation. Whether all of this SOA framework-mongering has clarified anybody’s thinking on the topic, or simply added more clutter, is a point worth arguing. Usually, any given vendor’s take on SOA is closely aligned with its commercial offerings—a fact you should weight into your evaluation of their SOA recommendations. But that shouldn’t stop you from considering what they have to say on SOA, and on how you can realize cost savings and other SOA benefits through real-world IT solutions.
IBM, for example, has established several SOA-related initiatives that leverage its worldwide dominance in IT professional services. Many enterprises will adopt the SOA approaches of their principal integration partners, so IBM’s SOA framework will increasingly find its way into many real-world deployments.
IBM recently launched an SOA implementation framework—called Service-Oriented Modeling and Architecture (SOMA)--that encourages enterprises to use professional services for internal and external enterprise integration. Managed by the IBM Global Services organization, the SOMA framework guides enterprises in SOA planning, design, implementation, and management. IBM Global Services implements SOMA through reusable software components, best practices, and business modeling tools that it developed in engagements in many vertical markets. The company provides online and classroom SOA education courses and customizes one-on-one SOA workshops to client needs. It provides a free online assessment tool that enables businesses to evaluate their current level of SOA readiness and identify SOA priorities. And—no surprise here--IBM has integrated SOA patterns, processes, and tools into its WebSphere, Rational and Tivoli products.
SAP, as the world’s dominant business application vendor, is also promoting SOA as a core theme for its ongoing product development initiatives and customer engagements. Many enterprises will find themselves moving deeper into SOA as SAP—their ERP vendor—adopts service orientation throughout its product suite.
SAP’s SOA focus is on its mySAP generation of packaged business applications and on the underlying NetWeaver platform. NetWeaver implements the vendor’s Enterprise Services Architecture (ESA), which encompasses a broad range of Web services standards. Through native Web services support, integration middleware, and visual MDD tools, developers can built composite applications that bridge diverse SAP and non-SAP environments. SAP is in the process of decomposing its vast monolithic R/3 and mySAP application suites into functions that can be exposed as modular business services. By 2007, all SAP application functionality will be exposed through WSDL and registered within a UDDI registry native to NetWeaver, making SAP’s product portfolio thoroughly SOA-enabled. SAP is also ramping up its SOA professional services offerings, providing a broader range of best-practices workshops and tools.
Systinet, a leading vendor of UDDI registries, has developed a framework that stresses the central role of registries in policy-based SOA. Systinet’s Governance Interoperability Framework (GIF) positions the SOA registry as the principal policy-management platform for service composition, integration, security, management, and other governance functions. Systinet has gained support for GIF from many Web services management business partners, including Actional, AmberPoint, DataPower, Layer 7, Reactivity, and Service Integrity. For several years, Systinet has been a vanguard vendor in Web services and SOA, so these partnerships should be interpreted as strong industry support for GIF as a basis for SOA governance.
To date, Systinet has not published the GIF specifications that are being implemented by the company and its partners. However, Systinet has stated that the GIF includes specifications that cover the following policy management requirements (aka governance) across multivendor SOAs:
• Governance data integration: GIF-compliant SOAs will leverage the registry as the authoritative repository and management platform for service descriptions, attributes, and policies.
• Governance control integration: GIF-compliant SOAs will implement a common framework for service event notification and response based on policies managed in the registry.
• Governance user-interface integration: GIF-compliant SOAs will provide a common view of service descriptions, attributes, and policies managed in the registry.
Infravio is a WSM and Web services registry vendor with its own SOA framework, which it calls “Intentional SOA.” Basically, Infravio’s framework is a checklist that enterprises can use to determine their requirements and readiness for SOA, Web services, Web services management, and registries. One of the most useful features of Infravio’s Intentional SOA whitepaper is that it clearly delineates the various Web services standards that will be necessary for a layered, full-function, platform-agnostic SOA. In this regard, Infravio is like many vendors, whose Web services and SOA strategies are one and the same thing.
TBD: SOA on mature, comprehensive, ubiquitous, standards-based Web services
Truth be told, that’s not such a bad SOA strategy. Web services are the first middleware and development environment that can truly be called platform-agnostic. Web services enable more complete service virtualization, abstraction, loose coupling, and reuse than any prior environment. Web services are the preferred environment for thoroughgoing SOA. Within Web services environments, WSDL service contracts provide the principal platform-agnostic APIs for service virtualization.
In order to realize the nirvana of SOA everywhere, the following, at minimum, must happen:
• SOAP—the principal Web services middleware protocol--must be implemented natively in all new application development and integration projects.
• WSDL—the principal Web services API--must be used to expose the functionality of new applications, and to wrap existing services, applications, operating platforms, and other resources as Web services.
• UDDI—the principal Web services registry environment—must be implemented throughout your application infrastructure and all WSDL and other service metadata and policies must be published to those registries.
Of course, enterprises can implement a limited form of SOA within traditional middleware environments, such as Common Object Request Broker Architecture (CORBA) and Distributed Complement Object Model (DCOM), relying on these environments’ native service contract, registration, location, and binding interfaces. Indeed, many of the SOA case studies we describe in this Network World special section rely on older middleware approaches in addition to Web services. However, those pre-Web services SOA implementations are tightly coupled to the native object models, methods, and application programming interfaces of particular operating and application platforms. Consequently, changes to the underlying, tightly coupled platforms can disrupt interoperability with consuming services.
Ubiquitous adoption of Web services is a necessary, but not sufficient, condition for universal SOA. There are SOA-friendly and SOA–unfriendly ways to implement Web services. With Web services-based SOAs, developers have the option of coupling services either loosely or tightly, based on whether they design WSDL interfaces to correspond closely with service implementations—though loose coupling is the best practice for flexible SOAs. Some Web services implementations may introduce tight coupling with underlying platforms, but, generally, Web services encourage loose coupling via the following service design practices:
• Define WSDL-based service APIs so that these interfaces don’t contain types, collections, or other artifacts specific to the platform—such as J2EE or .NET—on which the service is implemented;
• Expose service functionality through standard document-oriented approaches, especially SOAP’s Document/Literal style interface;
• Create self-describing XML-based messages/documents that convey context information pertaining to ongoing interactions among services.
One of the drawbacks of a purely Web services-based approach to SOA is that some of the most critical “WS-*” standards have not yet been finalized, ratified, or adopted broadly by vendors and users. According to a recent IDG Research Services Group survey, IT professionals ranked incomplete and immature Web services standards as one of the top inhibitors of SOA implementations in their companies. Other significant SOA inhibitors include the immaturity of solutions that implement Web services standards and limited developer support for these standards.
The WS-* “stack,” like SOA, is a work in progress. It is not yet a full-featured alternative to older object-brokering and message-oriented middleware protocols. And it is not a unified set of specifications under single authorship, management, or governance, and probably will never become such. It is being defined by diverse organizations, including OASIS, the World Wide Web Consortium (W3C), and various vendor partnerships.
Sharing and reusing Web services is complicated by the fact that many platforms implement diverse versions of various WS-* specifications for federated identity, security, reliable messaging, event notification, publish and subscribe, and other critical infrastructure functions. Consequently, IT professionals will continue grappling with diverse platform-specific implementations of WS-* specifications until vendors converge on consistent implementations of common standards in various layers. In this regard, then, the current WS-* stack—as implemented in real products and tools--doesn’t yet support truly platform-independent SOA.
There is considerable industry disagreement and rivalry regarding which WS* specifications should prevail in many areas, such as federated identity, reliable messaging, event notification, and transactions. And even in functional areas where the industry has already converged on principal WS-* standards, it may take years before most vendors have implemented those standards in their products and have worked through the interoperability issues associated with diverse implementation of agreed-upon standards.
Until such time as the industry converges on universal, comprehensive, stable WS-* standards in various functional layers, Web services—as a complete SOA stack—will remain immature. Some of the most noteworthy areas of WS-* industry rivalry are as follows:
• Federated identity: In March 2005, OASIS ratified Security Assertion Markup Language 2.0 as a standard, incorporating specifications that had been contributed by the Liberty Alliance initiative. The previous SAML versions—1.0 and 1.1—have achieved considerable commercial adoption throughout the federated identity marketplace, with the Liberty Alliance specifications achieving some commercial adoption. Another competing specification—the Microsoft-developed WS-Federation—remains in the industry picture, despite having achieved far less commercial adoption and not being managed by any standards group. WS-Federation, released in July 2003, covers a similar functional range as SAML 2.0. However, SAML 2.0 and WS-Federation are entirely non-interoperable specifications. WS-Federation doesn’t build on or extend SAML, and it isn’t directly interoperable with SAML or the Liberty interfaces. WS-Federation is kept alive through Microsoft’s decision to build its Active Directory Federation Services feature of “Longhorn” on the specification.
• Reliable messaging: In November 2004, OASIS ratified WS-Reliability 1.1 as a standard. Produced by OASIS’ Web Services Reliable Messaging Technical Committee (WSRM TC), the standard defines a protocol for asynchronous or synchronous message exchange with guaranteed, once-only, ordered delivery, even in the presence of component, system, or network failures. It defines mechanisms for message persistence, message acknowledgement and resending, duplicate message elimination, ordered message delivery, and delivery status awareness for sender and receiver applications. However, WS-Reliability is not the only WS-* specification to achieve considerable industry support. An industry group dominated by Microsoft has implemented the rival WS-ReliableMessaging specifications, which addresses the same core requirements and features as WS-Reliability.
• Transactional coordination: In October 2003, Sun, Oracle, and other vendors submitted WS Composite Application Framework (WS-CAF) to OASIS. WS-CAF defines a standard framework for transactional coordination of long-running business processes within Web services environments. WS-CAF consists of three specifications (none of which have been finalized or ratified yet within the OASIS WS-CAF TC: WS-Context, WS-Coordination Framework, and WS-Transaction Management. Meanwhile, in September 2003, WS-Coordination was published by Microsoft, IBM, and BEA. WS-Coordination provides an extensible framework for defining transaction coordination patterns and protocols. At the same time, Microsoft, IBM, and BEA also published WS-Atomic Transaction, which references WS-Coordination and defines an “atomic” (i.e., ACID) coordination type. Several months later, these vendors published WS Business Activity Framework (BAF), which defines protocols for transactional coordination on long-running business activities. Since that time, Microsoft, IBM, and BEA have not submitted WS-Coordination, WS-AtomicTransaction, or WS-BAF to OASIS or any other standards group.
Of course, the existence of rival WS-* specifications doesn’t doom Web services as a basis for SOA. Sharing and reuse can still proceed in a heterogeneous WS-* stack and application infrastructure. To address this stubborn reality, a flexible enterprise service bus (ESB) infrastructure can provide “impedance matching” between diverse WS-* standards, versions, and implementations. ESB products come from a broad range of middleware vendors, including Cape Clear Software, Fiorano Software, IBM, IONA, Sonic Software, Systinet, TIBCO Software, and webMethods. Depending on the vendor and product, ESB solutions provide any and all of the following features for a robust SOA infrastructure:
--Wrap legacy environments with WS-* interfaces, thereby supporting legacy integration;
--Provide any-to-any interoperability through integration brokers, which provide orchestration, transformation, and routing services among distributed services, applications, data repositories, and other resources;
--Support content-aware monitoring and enforcement of performance, security, and other enterprise policies pertaining to Web services traffic;
--Support the request-response, event-driven, and method-invocation message flows among distributed services;
--Support reliable messaging, transactional coordination, and other robust interaction patterns; and
--Support hub-and-spoke, decentralized, and peer-to-peer message exchange patterns
Figure 6 illustrates the many functional service layers that are necessary for a full-fledged, robust SOA interoperability environment. WS-* standards and specifications have been developed in all of these layers. However, standards in some layers are more mature and widely adopted than in others. The core layers for today’s Web services are messaging and communications (SOAP); description, semantics, and policy (WSDL, XML Schema, WS-Policy); and brokering, registry, and discovery (UDDI).
Even the consensus WS-* standards—such as WSDL, SOAP, and UDDI—have not been implemented consistently and universally by all platform, application, and tool vendors. The standards provide many options that can hinder interoperability, unless organizations agree on implementation profiles that were published by the Web Services Interoperability (WS-I) Organization.
Vendors may claim conformance with WS-I profiles for various WS-* standards. But what those vendors have actually implemented, as default settings in their products, may be another thing entirely. And that doesn’t stop application developers and administrators from overriding the vendor defaults and implementing WS-* specifications in entirely non-standard ways, thereby frustrating any-to-any interoperability throughout their SOA environments.
TBD: SOA developed through visual modeling techniques
For practical SOA, it’s not enough simply to have conceptual clarity on what SOA means, or to have a ubiquitous Web services or ESB environment within which services can be reused, invoked, monitored, and managed. It’s also critical that developers have new visually oriented modeling tools for building distributed services upon SOA principles.
Visual models are essential for developers who need to navigate the unfamiliar seas of SOA. If you’re a developer, SOA may make you feel a tad queasy, because it’s rocking your familiar paradigms to the core.
Fundamentally, SOA is a disruptive new approach to building distributed services. SOA defines an environment within which the most ancient computing paradigm is rapidly dissolving. From the dawn of computing, we’ve developed new functionality upon and within such concepts as “platform,” “application,” and “language.” Each of these concepts has traditionally had a well-defined sphere of reference: the platform hosted the application, and the application was developed in a language. Now all of that is changing, thanks to the emergence of SOA.
The first of these ancient computing concepts to wither away will be that of the platform. This term originally applied to operating systems, and then broadened to include application servers that implement a particular development framework (such as Java 2 Platform Enterprise Edition or .NET) over one or more operating systems. However, the growth of standards-based, distributed Web services has made it clear that fewer and fewer business processes will execute entirely within the confines of a J2EE 1.3 server, or Windows Server 2003, or Linux, but will execute across them all. When all platforms share a common environment for describing, publishing, and invoking services, the notion of self-contained platforms disintegrates in favor of SOA, which is essentially a platformless service cosmos.
Likewise, another casualty of this evolution is the notion of applications as discrete functional components that execute on particular platforms. SOA is founded on the notion of virtualization. Under this paradigm, services describe abstract interfaces within standard, platform-independent metadata vocabularies such as WSDL. The underlying service functionality may be provided from components on any platform—J2EE, .NET, or otherwise—without the need to change the abstract interface. Under SOA, the application dissolves into a service that may have no fixed implementation but simply bids for on-demand networked software and hardware resources.
And programming languages are also becoming, if not entirely obsolete, something that fewer developers touch directly. MDD—which refers to visual model-driven development and code-generation tools—is at the forefront of the SOA revolution. You’re more likely these days to see a vendor boast of its ability to support visual modeling in the Unified Modeling Language (UML) than development in Java, C#, or any other declarative programming language. For complex, orchestrated, multiplatform Web services, MDD is the most effective approach for specifying, implementing, and maintaining the end-to-end logic and rules upon which the service depends.
SOA has spawned a range of new terms—such as “orchestration” and “recomposition”--to describe what it is that developers actually develop. IT professionals are increasingly defining their creations in terms of services, models, and patterns (rather than in terms of platforms, applications, and languages). The notion of MDD-orchestrated service patterns will become increasingly critical to discussions of distributed services. An orchestrated service pattern is simply a generic approach—such as “service proxying” or “service coordination”--to architecting interactions within the SOA. Every pattern defines its own abstract Web services functional elements and its own SOAP-based interactions that are executed by integration brokers within your ESB.
Welcome to the dizzying new world of SOA. Platforms are dissolving, new concepts are taking over, and Web services will become everywhere shareable and reusable. There’s so much new work to be done, and so many new tools—both conceptual and computational—within which to do it.
