SOA: The ABCs
A practical primer to a complex development methodology
--James Kobielus
Service-oriented architecture (SOA) is an inescapable IT-industry buzzphrase these days. It’s also a sprawling, eye-glazing abstraction that means many things to many people.
SOA concept, benefits, and business drivers.
However, there is considerable agreement on SOA’s core meaning. At heart, SOA is a development methodology that encourages greater sharing of remotely invocable application functions throughout networks. SOA best practices help enterprises create more flexible, adaptable IT organizations. Other SOA benefits include accelerated application development, reduced development costs, reduced duplication of work across development teams, consolidated deployment and management of application code, and increased business-process consistency across applications that share common services.
SOA development practices have taken hold throughout the corporate IT world over the past several years. One of the major factors driving this trend has been the growth of Web services, which provide a ubiquitous middleware fabric that makes SOA feasible.
SOA, though often linked in industry discussions with Web services, isn’t necessarily tied to that environment. SOA is a set of service design principles that may be implemented within and across diverse development, application, middleware, and operating environments. Many enterprises, including the ones we discuss in this article, are rigorously applying SOA service design principles to environments that haven’t yet implemented all of the core Web services standards, such as XML, Web Services Description Language (WSDL), Simple Object Access Protocol (SOAP), and Universal Description, Discovery, and Integration (UDDI).
Some would argue that a true SOA implementation requires the full Web services stack, including UDDI, WSDL, SOAP, and the growing range of specifications beginning with “WS-.” Others, including this author, try not to be Web services bigots. SOA is a set of objectives and practices that may be realized to greater or lesser degrees under various platform and middleware environments, using various programming languages and development tools. In this article, we discuss several examples of successful SOA implementations that aren’t predicated on widespread Web services deployment.
“For the purposes of SOA, Web services aren’t fundamentally different from CORBA, Java Remote Method Invocation (RMI), and other application frameworks,” says Christopher Crowhurst, vice president and principal architect at Thomson Learning, a Stamford, Connecticut-based business unit of Thomson Corporation that provides technology and assessment services worldwide. “These are all middleware environments within which you can build loosely coupled distributed services.”
Another factor behind SOA’s popularity is the never-ending business imperative to accelerate application development and cut costs. Under SOA methodology, developers write new applications by looking up the requisite functionality in online service registries, plugging those services’ APIs into their code, and writing some fresh orchestration logic to tie it all together. Taken to its logical extreme, SOA can make new development a “connect the dots” exercise that greatly shortens the time-to-market cycle for software deliverables.
SOA service-development practices.
Before we delve into SOA case studies from today’s corporate world, let’s define SOA as an overall approach. SOA refers to any distributed application environment that emphasizes the following service-development practices:
• Service virtualization: An SOA-based environment virtualizes or abstracts external service-invocation interfaces from those services’ internal platform implementations. Under pure SOA, the external application interface—or API—should be agnostic to the underlying platforms. Under SOA and other virtualization paradigms, applications may run on diverse operating and application platforms; have been deployed on nodes in diverse locations; have been aggregated across diverse hosting platforms; and have been provisioned dynamically in response to client requests. Some refer to service virtualization or abstraction as “loose coupling.” Within Web services environments, WSDL “service contracts” provide the principal platform-agnostic APIs for service virtualization.
• Service reuse: An SOA-based environment maximizes standards-based reuse, interoperability, orchestration, management, and control of distributed, virtualized services. In a pure SOA-based nirvana, developers would never consider rewriting an application function from scratch if an equivalent service has already been developed and is being hosted elsewhere in their enterprise (or in a partner organization). They would simply invoke that service from within their applications, using standard messaging-based protocols. By the same token, they would make all of their application functions available to other programmers as remotely invocable services. Within Web services environments, SOAP is the principal remote-invocation messaging protocol enabling cross-platform service reuse.
• Service brokering: An SOA-based environment facilitates reuse through infrastructure that allows service interfaces to be registered by their providers and to be discovered by other services that wish to integrate with them. A service broker is any entity that facilitates registration, location, retrieval, synchronization, replication, and/or delivery of services. Within Web services environments, UDDI registries are the principal service-brokering nodes for platform-agnostic SOA.
We now discuss each of these SOA practices in greater detail.
Service virtualization.
Services are the core concept of SOA. A service is a reusable unit of functionality that can be invoked through a published metadata interface, which is often known as a “service contract.” Under pure SOA, a service contract—such as an API expressed in WSDL--exposes only as much application logic, behavior, and context as that service is willing to grant to other services, leaving opaque all other aspects of a service’s internal implementation.
Typically, virtualized services present their abstract interfaces via standard service description languages. The services expose their functionality through standard message/document-oriented middleware approaches, such as the SOAP Document/Literal interface. Services implement end-to-end semantic interoperability through standard, agreed-upon schemas—such as XML Schema--for inputting and outputting message/document-oriented data structures. Interactions among consumer and provider services often involve exchange of self-describing messages/documents—such as those in XML markup--that convey session state and application context information. Figure 1 illustrates these essential concepts of service virtualization.
Effectively, a service’s contract, such as a WSDL definition, constitutes its SOA virtualization layer. The contract abstracts the service’s external invocation interface from the platform that hosts the service’s underlying software components. A service may be abstracted from its enabling platform in various ways, depending on how closely its published interface it corresponds to the platform’s native APIs, class libraries, object model, and other programming artifacts. Developers shouldn’t simply echo their platforms’ native object models in their service contracts. As external interfaces, abstract service contracts should remain stable through platform changes, presenting a consistent invocation interface for consuming applications. By the same token, developers should enable interoperability among services using platform-agnostic, self-describing XML messages/documents that convey context information pertaining to ongoing interactions among services.
On any given application platform, service contracts may be decoupled from each other in various ways, depending on business requirements. SOA best practice calls for decomposition of applications into service primitives that correspond to basic business functions or processes.
“SOA requires that you focus on your core business capabilities,” says Crowhurst of Thomson Learning. “These are what you should be exposing as Web services within your SOA environment. SOA requires that you stay at a high level in defining these reusable business services, rather than drill down too quickly into application code.”
Under SOA, each service should have a recognizable business function that plays a clear role in many applications, says Derek Ireland, group tech solutions manager at Standard Life Group, an Edinburgh, Scotland-based insurance company. “Examples of these reusable business services in our SOA include ‘provide pension valuation,’ ‘verify identity,’ ‘provide bank details,’ ‘maintain address,’ and ‘produce statement,’” says Ireland.
“Currently, we have around 300 business services in our SOA service catalog,” says Ireland. “These services are high-level business functions that abstract away from the underlying complexities of our principal platforms: WebSphere Application Server, WebSphere Business Integrator, WebSphere MQ IMS, and .NET. All of these business services are accessible from the languages—Java, COBOL, and C#--that are implemented in the development tools that provide our SOA software framework.”
“Our SOA software framework allows the development teams to concentrate on the business aspects of the application under development,” he says. In this way, Standard Life’s SOA software framework allows developers to avoid getting distracted by platform-specific implementation details.
Another core SOA tenet is that services should be factored into stable, well-bounded sets of functionality. The greater the stability of a service’s external interface, the less likely it is that a change in the service’s underlying implementation will disrupt interoperability with existing consumers of that service. “SOA is an approach for defining clear boundaries between business and technical services that need to be decoupled,” says Jayson Minard, chief information officer at Abebooks (www.abebooks.com), an online book marketplace based in Victoria, British Columbia.
Typically, stable service contracts are those that have been defined to be coarsely granular. A coarse-grained service contract describes the interface to an entire business process or a substantial subprocess, rather than to the fine-grained details of a particular platform’s object model, classes, and APIs. Coarse-grained service design allows developers to migrate to new platforms while reducing the need to update the service contracts through which platforms expose application functionality.
Though some might argue otherwise, coarse granularity is not a hard and fast rule of SOA. The appropriate granularity depends on the nature of the services being defined and integrated.
Abebooks’ SOA is based on the need to define fine-grained business services that correspond to particular B2B technical-integration connections. Abebooks uses SOA practices to decouple the integration logic through which its websites connect with each of its myriad business partners. More than 13,000 booksellers from 48 countries list their books on several Abebooks sites, and major online booksellers such as Amazon and Barnes and Noble have outsourced their used-book operations to the firm. “We have a lot of legacy Java code, which we need to continually tweak to address the data-translation and other functions specific to various partner integrations,” says Minard. “Without clear boundaries among various code segments associated with partner integration, we would risk disrupting global interoperability with all of our partners every time we changed, say, a data-translation routine for one partner.”
Under SOA, a business service should be factored as broadly or narrowly as appropriate to the business process being automated, according to Maja Tibbling, application architect with Con-Way Transportation Services in Portland, OR. “Services can be fine-grained, such as a logging service, or coarse-grained, such as those services that contain an entire business process. Services should be defined at whatever granularity best promotes their reuse.”
Con-Way’s ongoing SOA initiative began in the late 90s as an initiative to implement component-based development (CBD) of mainframe applications. “Most of our back-end business functionality has been partitioned into components and our business functionality exposed as component operations,” says Tibbling. “These meet the functional definition of a service as they are consumed by other applications through interfaces. Initial implementations were on the mainframe. Since then, we have successfully ventured into the J2EE world and have deployed many business applications in the middle tier with Web-based front ends. So, the next step [for SOA] was consuming the same back-end shared services through vendor-provided Java proxies to the mainframe.”
Con-Way has also implemented SOA-based component partitioning on many mid-tier J2EE-based and Web-accessible applications. “It was anticipated that many J2EE-based composite applications may need services from some other business area,” says Tibbling. “Collections of related services are defined in business interfaces that are implemented in J2EE Session Beans. To invoke these services, applications use a couple of useful J2EE patterns, BusinessDelegate and ServiceLocator, which abstract the service invocation, so that the underlying technology could be changed at any time. These same services are also available to external customers directly on our company website for things such as rate quote and tracking.”
Some enterprise IT groups take coarse-grained SOA to (what some might regard as) its logical extreme. “SOA for us is mostly providing a rational framework for distributed server connectivity between distinct application servers,” says Adam Blum, director of server engineering at Good Technology, Inc., a Santa Clara CA-based provider of wireless products. “For example, our e-mail servers behind the customer company firewall which connect to Microsoft Exchange and forward e-mail need to connect to our hosted router for forwarding traffic and to our Web store to validate purchased licenses. We do this through a set of XML messages. An important aspect of SOA for us is expressing common abstractions (users, customer companies, messages, licenses, server information) in common ways.”
Service reuse.
Service reuse is where SOA pays off. Companies that make the most of SOA are those that train, encourage, and reward programmers to reuse existing services—no matter who developed them--to the maximum extent possible. Developers should also be encouraged to publish and share their own services widely. In an SOA nirvana, programmers would write as little new code as possible when constructing new applications, and the only new code would simply orchestrate new interaction patterns among existing services.
Maximizing service reuse demands an SOA-focused culture that spans all IT disciplines. “We have a virtual team representing all development areas that identifies, exhibits, and promotes best practice for delivering reusable components,” says Ireland of Standard Life. “The team manages the catalogue of reusable business services and the development process required to ensure consistency of XML interfaces and interoperability of services.”
Greater service reuse translates into lower costs and accelerated development cycles. “In 2004 we were able to implement several major business initiatives within extremely short timelines,” says Tibbling of Con-Way Transportation Services, “because of our ability to reuse existing functionality and to protect existing consumers from the impact of changes to existing functionality. Our core business components such as Customer and Shipment offer services that every single application ends up using. This has allowed greatly enhanced time to market for subsequent projects.”
In practice, service reuse depends on middleware that allows any service to interoperate with any other service over networks. Increasingly, Web services environments are the middleware fabrics of choice for SOA, leveraging WSDL, SOAP, and other WS-* standards. But, in many real-world SOA implementations, services are being shared, reused, orchestrated, and invoked over a broad range of legacy middleware environments that interoperate in various ways. A Web services environment is just one type of integration environment—or “enterprise service bus” (ESB)--in real-world SOAs. Figure 2 illustrates service reuse over an ESB. It’s surprising to find out how many actual SOA case studies involve only a subset of the Web services standards—sometimes, just the core specification, XML, and that simply for structured data interchange.
“We chose XML, not necessarily Web services [for exposing application interfaces across our SOA environment],” says Crowhurst of Thomson Learning. “We transfer XML over HTTP, and XML over file transfer. We also do XML over message queueing transports for abstraction of application interfaces.”
Con-Way Transportation is running SOA over a very heterogeneous, evolving middleware environment in which Web services are still just a bit player. “We are using TIBCO BusinessWorks to reliably orchestrate asynchronous business processes in near real-time using [J2EE] ServiceLocator [patterns], but invoked from the integration infrastructure,” says Con-Way’s Tibbling. “[W]e are defining an XML canonical model in the integration layer in XML Schema (XSD). TIBCO’s transformations are based on XML, Xpath, and Extensible Stylesheet Language Transformations (XSLT). So, we can take a transaction from any source in any format within our very heterogeneous environment, transform the data into a canonical XSD, and then interact with the shared services.”
Con-Way’s XML Web services implementation is growing. Nevertheless, they’ve also given J2EE a growing role in their SOA. “Java Message Service (JMS) is being used more and more,” says Tibbling. “As we upgrade our J2EE environment, we anticipate using MessageBeans as process triggers into the integration layer.”
The company has designed its SOA to facilitate graceful migration from older to newer middleware environments as necessary. “The architecture is such that service invocations using one technology can easily be replaced with another,” says Tibbling. “We also have some Web services implemented to facilitate access to our functionality from other websites and also for use by external customers.”
Con-Way isn’t the only SOA shop that has limited its use of Web services primarily to support external interoperability. “We’ve selected IBM WebSphere as our primary application server,” says Ireland of Standard Life.” “We also use Java as our principal development language, IBM WebSphere MQ Integrator as our [message-oriented middleware] integration platform, and XML as the markup syntax for all interoperability interfaces….We don’t see the advantage of using Web services and SOAP [in place of WebSphere MQ] for integration. But we will use SOAP when it’s appropriate, such as for B2B integration.”
Standard Life has standardized internally on SOA development tools that will facilitate service reuse through Web services interfaces, when appropriate. “It’s easy for us to implement Web services under our SOA framework [which includes design patterns and development tools, such as IBM WebSphere Studio Application Developer],” says Ireland, “and we’re showing we can do that.”
Service brokering.
Service brokering infrastructures encourage reuse by allowing developers to “advertise” their programs’ service contracts and other descriptive metadata in a shared online registry, repository, or catalog.
Service brokering infrastructures take many forms, and are often specific to particular middleware or platform environments. Any entity that facilitates registration, discovery, and retrieval of service contracts may be regarded as a broker. The UDDI standard defines a service-brokering environment for Web services.
Service brokering infrastructures support the following core functions:
• Service registration: Application developers publish their functionality to service brokering nodes, such as UDDI registries. Developers, also known as “service providers,” publish their services’ contracts, which include such descriptive attributes as service identities, locations, methods, bindings, configurations, schemas, and policies.
• Service location: Service consumers—in other words, application developers who wish to invoke registered services--query the broker to find services that match their functional requirements. The broker allows the service consumer to retrieve service contracts.
• Service binding: Service consumers use the retrieved service contracts to develop code that will bind, invoke, and interact with the services. Ideally, the service consumer’s development tool should automatically generate client code during this process. Developers should use integrated development environments to compile service contracts from visual, flowcharted process maps and then to register these contracts to the appropriate service brokers.
Figure 3 illustrates the essential functional elements and interactions in a service brokering environment.
As noted above, the UDDI standard defines a brokering environment geared to the emerging world of Web services (though many Web services implementations lack UDDI registries). WSDL service contracts and UDDI “tModels” are the principal Web service interfaces published to UDDI registries. Other common service-brokering environments include Common Object Request Broker Architecture (CORBA) Naming Service, Distributed Computing Environment (DCE) Cell Directory Service, Windows NT Registry, Java Remote Method Invocation (RMI) Registry, and Electronic Business XML (ebXML) repositories.
If they wish, enterprises can deploy a DBMS as a service-brokering node for their SOA environment. “Our SOA environment includes a runtime business service directory that runs on IBM Universal Database/DB2 on a mainframe,” says Standard Life’s Ireland. “We now have around 300 services published in the business service directory, which is accessible from our SOA framework. We can enforce version controls on services that are published to the directory.”
Abebooks has implemented a hybrid UDDI/LDAP registry architecture, according to company CIO Minard. “We’ve implemented an open-source UDDI registry outside our firewall to publish external service interfaces that our partners can use to connect to our services. The external UDDI registry uses LDAP to do lookups of the master service directory that sits behind our firewall.”
Not all SOA implementations rely on service-brokering infrastructures, but the broker-less SOA is usually a transitory phenomenon. “Early on, a major obstacle to achieving reuse was convincing the development community to first look for services that do what you need or almost do what you need,” says Con-Way Transportation’s Tibbling. “[One of our] main challenges now [is] creating a meaningful catalog or repository of existing services…[We plan to] create a comprehensive repository of business component services for all types of services (both Web Services and other technology implementations), perhaps using WSDL and UDDI.”
As this discussion indicates, UDDI is becoming the service-brokering infrastructure of choice for many SOA implementations, including those that are only partially down the road to Web services. UDDI-based service registries provide a basis for ongoing governance of SOA-based distributed application environments, many of which are migrating toward more thorough implementation of Web services. Enterprise SOA implementations should include policy-based tools that leverage their UDDI registries and support governance of the entire service lifecycle, from design and development through deployment and ongoing administration. Web services management tools are a critical component of the SOA governance equation.
Clearly, SOA is a much broader trend than Web services. SOA shifts developer focus away from individual applications and toward reuse of modular service components hosted throughout distributed environments. As this article has shown, many enterprises will maintain mixed SOA environments for the foreseeable future, combining a bit of Web services with a lot of legacy middleware, and implementing UDDI—in fits and starts--alongside other registries.
This platform-agnosticism is the essence of true SOA. When implementing SOA, enterprises should take care not to introduce any specific dependencies on WSDL, SOAP, UDDI, or any other Web services standards or specifications. SOA implementers should treat Web services—and any other development, interoperability, and operating environment—merely as implementation details.
SOA should be equally applicable to any environment—legacy, current, and future—and should serve as a virtualization blueprint for coexistence and interoperability among many generations of application infrastructure.
