1. Introduction

In the emerging world of the Internet, new APIs, protocols and standards proliferate and abound, adding more choice, more heterogeneous opportunities, and quite possibly more disharmony. Radical changes in business models and fundamental redefinitions of marketplaces, driven by innovative use of the Internet, are causing individual businesses to transform at incredible speeds. To achieve successful transformation, businesses must clearly integrate themselves within such marketplaces.

Vendors can approach the integration problem from one of two ways: the proprietary path, and the open path. On the one hand, vendors that follow the proprietary path may be locking in their customers and business partners on their core infrastructure. As the customer buys more software on this vendor's stack, it may become tougher to switch to something else when this technology no longer serves their needs. On the other hand, vendors that follow the open path advocate a shared core infrastructure, whereby customers can leverage and integrate multiple implementations, and not be penalized for embracing diversity.

Customers want to be able to use the Internet to transact and navigate seamlessly, regardless of the browser or underlying software and hardware serving up each Web page. In other words, in addition to having a web of documents, customers want to have a web of programs: programs provided by a variety of vendors that can easily integrate with the customer's existing infrastructure. For this reason, IBM has embraced the open path, seeking to address customer needs by championing and participating in the definition of open standards. To better ensure success of the standards, the industry needs strong advocates who are willing to build mindshare. These factors not only bolster the chances of a standard succeeding, but they potentially accelerate standards adoption.

We'll have a look here at IBM's formula for championing interoperability:

1. Support the process for defining open standards and contribute to open standards
2. Contribute to (and possibly even seed) implementations of standards to the open source community
3. Use open source code in IBM products

2. Vendor Participation in Defining Open Standards

A key ingredient of the Internet economy is the adoption of open standards. Open standards played a key role in the initial emergence of the Internet, and continue to play a key role in its expansion. As the Internet and IT business infrastructure grow, and as the number of devices to be connected continues to multiply, the need for integration is critical. In effect, standards create common languages, and open source software confers to all the players in the marketplace -- application developers, device developers, service providers, and enterprise customers -- a level playing field. The resulting flexibility gives these players freedom to choose amongst vendors that support open standards in their products. Standards provide the greatest promise to reduce integration costs, and vendors who support open standards have the most to gain.

2.1 Standard Bodies and IBM Participation

IBM has shown leadership and strong support in the development of open standards, participating in several standards bodies: the World Wide Web Consortium (W3C) [2], OASIS [4], The Internet Engineering Task Force (IETF) [7]and others.

2.1.1 W3C

Since 1994 the W3C has been developing specifications to promote the evolution of the Web and ensure its interoperability. Today the consortium has around 450 members, including governments, vendors, users, research laboratories and other standards bodies.

IBM participates in almost every W3C Working Group, working together with other members to reach the W3C goals. IBM representatives to various Working Groups work on the Extensible Markup Language (XML), XML Schema, Extensible Stylesheet Language (XSL), XHTML, Voice Extensible Markup Language (VoiceXML) and many other specifications [3]. IBM led or co-led (with Microsoft, BEA and other vendors) the creation of key Web Services specifications, such as SOAP and WSDL, submitted to and developed by the W3C. Today, IBM works with other vendors on developing other Web services related specifications, such as XML Digital Signature, XML Encryption Syntax.

2.1.2 OASIS

OASIS, previously named SGML Open, was founded in 1993 and has around 600 corporate and individual members. Its goal is to drive the development and adoption of e-business standards. Historically, OASIS has worked on vertical industry standards, developing XML vocabularies or schemas for specific industries, such as insurance, health care, retail, and others. Its scope now extends to several key Web services specifications that have been submitted to OASIS by IBM and other vendors, such as UDDI and Web Services Security [8].

2.1.3 WS-I

While Web services standards and technologies enable interoperability, they do not guarantee it. For this reason, several vendors (IBM, Microsoft®, Oracle, HP, Intel, BEA, SAP, Fujitsu, Accenture and BEA and 46 other companies) announced the formation of the Web Services Interoperability Organization (WS-I) [5]. The goal of this organization is to provide clarity and guidance for developers who wish to build Web services that use the underlying standards correctly and according to industry conventions, Chief Information Officers (CIOs), and others making investment decisions who need to understand when tools, runtimes, and Web services themselves are compatible. The intent of the WS-I is to publish guidelines and testing software, which give companies and customers a way to ensure smooth interoperability between standards-compliant software. Since its launch, WS-I has received over 700 inquiries about joining the community, and it has approximately 160 members, including about 20 companies that are not information technology suppliers.

The basic profile of WS-I describes how to comply with low-level Web services communications specifications, including SOAP and WSDL. In April 2003, the WS-I announced the formation of the Basic Security Profile Working Group [6], expected to deliver a security profile or a series of published guidelines on how to adhere to standards in order to guarantee interoperability. One security specification expected to be under consideration by the WS-I is WS-Security, which is now being developed in the OASIS standards body.

2.1.4 Java Standards

The Java standards represent a set of specifications, initiated and overseen by Sun Microsystems rather than a neutral standards body. IBM participates in this worldwide effort to leverage Java as a standard programming language and to promote the ongoing refinement of this development platform through the Java Community Process (JCP), contributing to over 80 percent of Java 2 Enterprise Edition (J2EE), which is the basis for many Web application servers. One of IBM's focus areas is to define the extensions needed to enable Java in mobile and pervasive contexts.

3. Challenges Working on Standards

Although open standards can be an essential part of a vendor's integration strategy, aspects of a given standard may pose technical problems that don't suit the vendor's preferred technical solution. Indeed, while the different standard representatives typically share a common set of requirements, they may each have unique requirements that specifically solve problems that their customers are facing. Vendors need to accept that compromises and tradeoffs are an inevitable part of standards development process.

3.1 Tradeoffs

During the specification development of W3C DOM [3], vendors needed to compromise on several key design issues. For example, one of the goals of the working group was that the object model be language-independent - that is, have no language-specific features. Because not all programming languages provide a runtime mechanism to query the type of an object, the DOM standard couldn't assume a language implementation could handle this. Instead, DOM provides its own query methods. While this may be bothersome to Java programmers who would naturally use the instanceof operator, calls from different languages would work equally well, whether implemented in Java, C++ or Perl. A similar example also came up when it was decided not to use method overloading in the DOM, because some languages do not support this basic OO concept.

In some cases, vendor participants cannot agree on a single solution for a given problem, given their different customer requirements. If no compromise can be reached after evaluating the various options, multiple approaches for solving the problem might be included in the standard, leading to a bloated specification. For example, the DOM specification includes two different navigation models: one is based on a linked list, where an application requests the first child and then navigates to the next sibling; the other is based on an array, where an application retrieves an object containing all the children and then loops through the array accessing the children by index. Standards implementers need to implement both approaches.

In other cases, specification disagreements are handled by allowing optional features to be defined. These are features that implementations are not required to support in order to claim standards conformance. This approach could lead to the definition of different conformance levels. For example, the XML 1.0 (Second Edition) [10] specification defines two classes for conforming XML processors: validating and non-validating, where the former is required to read the internal and external DTD, while the latter is only required to read the internal DTD subset. In extreme cases, specifications go as far as allowing for proprietary extensions. However, while sometimes necessary for participants to find agreement and be able to tolerate different approaches within a standard, these solutions inherently defeat the primary goal of standards -- to provide interoperability -- and must remain as few as possible.

3.2 Intellectual property and other costs

Developing and later implementing the standard does not come for free.

To help a standard achieve mainstream acceptance, it needs to be implementable by multiple parties, including other vendors and open source communities. For this to be possible, vendors need to agree to license intellectual properties that they have on the standards. Historically, standard organizations have required from their members that they agree to license such patent claims under "reasonable and non discriminatory" terms. But organizations such as the W3C may often require patent claims to be licensed on a royalty free basis. This requires corporations to assess the impact on their patent portfolio, the cost of giving away their patents, and the return they can expect from developing and implementing the standard. Moreover, because royalty free licensing typically conflicts with corporate policies, vendors need to develop new internal processes to make this assessment, leading to yet more cost and complexity to participate in the development of standards.

Because standards often build on top of one another, vendors who are involved in a number of standards organizations and working groups within those organizations need to actively consolidate their efforts to ensure that the resulting specifications don't conflict with one another. One example of such a conflict is between the XML and SOAP specifications. Some argue that SOAP constrains its usage of XML to a subset by requiring that SOAP messages, which are XML documents, not have a document type declaration. From a SOAP perspective, there are performance and security concerns for wanting to take this approach, but still the result constitutes a deviation from the XML specification. A standards-conforming SOAP processor may throw an exception on an XML document with a document type declaration, even though a standards-conforming XML processor would successfully process this well-formed document. Vendors that implement SOAP and XML processing tools may face interoperability problems as one set of tools may not work with the other. Vendors who have representatives in multiple working groups can help mitigate such problems by coordinating their efforts during standards development. By not paying attention to how standards can affect each other, participating vendors risk having to implement conflicting standards and having to support reconciling these problems on an ongoing basis for their customers.

By not participating in the early development of a standard, vendors take the risk of having to support a standard that does not meet the requirements of their customers. For example, because IBM did not participate in the original XML 1.0 working group and bring focus to a particular Unicode feature, the XML specification [10] fails to recognize the new-line character (also know as "NEL" or "#x85"), used on IBM mainframes as a valid line separator. This simple oversight means that XML 1.0 documents generated on mainframes must either violate the local line-end conventions, or employ otherwise unnecessary translation phases before parsing and after generation. By later participating in the W3C Core Working Group, IBM was able to raise the issue and get it addressed according to the recommendation of the Unicode Consortium, which matches the IBM mainframe convention. To ensure that their customer requirements are met, vendors would be wise to get involved early in standards efforts.

4. Community and vendor driven standards

The development of the Web was led by its inventor, Tim Berners-Lee, and the academic community. The industry followed. Contrast this with the case of Web services, where vendors have been driving the standards. It was the cooperation between IBM and Microsoft that seeded the Web Services Description Language (WSDL) specification. Later, together with other vendors, they chose to submit the WSDL 1.1 specification [3] to the W3C for further development and standardization. Following this submission, the W3C created a new working group to work on the next version of WSDL, which attempts to tighten the specification in several areas and include other features to meet requirements of all the participants.

Both IBM and Microsoft had early implementations of the WSDL 1.1 specifications [3] in their products. In addition, IBM has been involved in JSR110 to develop a standard set of Java APIs for representing and manipulating services described by WSDL documents. To foster the adoption of the standards, IBM has released an open source implementation of this specification, WSDL4J on the IBM developersWorks® site (www.ibm.com/developerworks).

By being proactive in the development of specifications and implementations, vendors can play an instrumental role in the early adoption of new technology. However, once such a specification is contributed to a standards body, the contributing vendor loses effective control over the specification. This is the price paid to grow a specification into a standard. The vendor may still choose to stay closely involved in the development of this standard to ensure that the key specification requirements remain addressed and that the standard being developed is backwards-compatible as much as possible with the original specification.

5. Accelerating Open Standards Adoption

To encourage the use of standards and gather feedback from customers, IBM provides implementations of emerging technologies at its alphaWorks® Web site [9], often before these technologies are even folded into IBM products. For example, IBM released an implementation of the SOAP specification (SOAP4J) at roughly the same time the specification was submitted to the W3C.

To facilitate the adoption of standards and to give other vendors another good reason to support the standards, IBM chooses to contribute some of the code available on alphaWorks [9] to the open source community, to also help accelerate adoption by the industry. Using these components, vendors and developers can then more easily build standards-compliant solutions. In the last several years, IBM has contributed several implementations to the Apache Software Foundation. Examples include:

• Apache Web Services Project: implementation of SOAP (Axis), Web Services Invocation Framework (WSIF), Web Services Inspection Language (WSIL).
• Apache XML Project: XML parsers (Xerces) [13], XSLT stylesheet processor (Xalan)

Together with the open source community, IBM continues to invest in the development of these products, to ensure that these implementations comply with the latest specifications released by the standards bodies.

6. A Vendor Working in Open Source

6.1 Challenges faced when vendors work in open source

Although IBM has seeded several open source projects and continues to play an active role in many projects, it's important to characterize day-to-day contributions to these projects as individual activities. A vendor cannot approach an open source project, as a vendor, and take a controlling role in the direction of the project. The open source culture is fiercely individualistic, centrally premised on the notion of merit: the more you contribute as an individual to the project, the more influence you may get. Contributors are not members of companies, but individual voices. Multiple contributors from a single vendor cannot leverage each other's merit. There are cultural nuances that vary from project to project, but some observations were made in the Apache XML Project. Consider the case of a contributor who has spent several months working on a given project, responding to user questions, providing bug fixes, making plan suggestions, writing component code, and eventually being voted in by the community as a committer. A programmer from the same vendor, perhaps someone who is just as familiar with the project code base, cannot simply join the community one day and become a committer the next; they need to start from the bottom and earn their own way. If this second individual builds up their contribution over a period of time, the committer from the same vendor could recommend that the second contributor become a committer, but that committer needs to take care that the recommendation is not premature, to avoid the charge of favoritism.

Indeed, the second contributor from the vendor may need to overcompensate in contributions to avoid the charge of favoritism. In the case of Xerces [13], IBM wanted to increase its contributions, and added more individuals to the project. Existing IBM committers established high criteria that non-committers needed to meet before a recommendation for committer status went through. Notwithstanding efforts to ensure that IBM committers only made it through on individual merit, some community members were concerned that IBM was playing too large a role on the project.

So in the end, although the vendor-employed contributor needs to earn their way as a non-affiliated individual, they may still be labeled as someone who earned their place simply because of corporate ties.

Is there strength in numbers? Depends who you ask. Multiple contributors from the same vendor could help ensure that the kinds of features desired by the vendor have an optimal chance of getting into the code base. Note that on many Apache projects, it takes just a single negative vote to veto a new feature from being added or the plan for the next release, regardless of the number of positive votes. An active vendor with multiple contributors must ensure that they're not trying to steer the project solely to serve their own ends. There can be (and often is) disagreement between contributors from the same vendor on a particular implementation. The vendor needs to find the balance between a) taking its debate to a public forum within the community, and b) showing so much divisiveness that the proposed features are themselves in jeopardy. In IBM's case, they want to ensure above all that the new features support the key open standards in a given technology space.

Culture clash is another area where vendor-sponsored contributors may struggle. The vendor culture is likely based on a top-down management structure, where project goals are based on the imperatives of the business. In contrast, the open source community runs on a meritocratic, bottom-up model. Vendor employees who are paid to work on open source projects are asked to work simultaneously in both models. Too much on the vendor side, and the contributor may be accused in the community for demonstrating an inappropriate bias. Too much on the open source side, and the contributor's commitment to the company may be questioned. A fine balance is required: approach the community with a merit-based attitude, but focus on aspects of the project that further the vendor's business goals. It would help if vendors provided some guidance to their employees on how to maintain this balance.

6.2 Seeding an open source Project

IBM sometimes cooperates with other vendors to establish de facto standards by contributing to open source projects. For example, one of the challenges IBM has been facing is that tools from different companies don't work well together. Developers don't want to spend time on tool integration while tool vendors don't want to spend time on reinventing the wheel.

IBM contributed seed code for the Eclipse project [11], an open, extensible platform for tool integration, built by a community of development tool providers. Eclipse members include Sybase, Rational® Software, Red Hat, Borland International, Hewlett-Packard, Oracle, and many others. Eclipse operates with a Common Public License that provides royalty-free source code and worldwide redistribution rights. The Eclipse platform provides developers with ultimate flexibility and control over their software technology and aims at the large community of software programmers that favor open-source tools and operating systems.

7. Re-use of Open Source Implementations in Vendor Products

Open source adherents are fiercely defensive of the quality of open source code. Some common mantras include: "release early, release often", "given enough eyeballs, all bugs are shallow" [12]. And while such devotion is often warranted, the truth is that the quality of code varies between open source projects, and even components on such projects. As well, if bugs are found, getting quick support from the open source community is not always possible. Some open source contributors provide altruist levels of support to their community, but vendors can't base their support claims around such devotion. It is essential for a vendor to ship well-tested code and provide timely support to their customers.

IBM addresses these concerns by shipping versioned maintenance streams of the main open source code base.

As an example, let's look at Xerces [13], Apache's Java-based XML parser. In Figure 3, evolution of the open source code moves upwards along the vertical trunk. Of course, while the general progression of the open source code base is positive as new features and bug fixes are added, regressions naturally get introduced. When development of a major release of the Xerces code is made available and the IBM team has thoroughly tested the code, IBM takes a snapshot of the Apache code base to create their own maintenance stream, IBM XML4J. New features cannot be introduced into a maintenance stream - only bug fixes. The IBM team will then contribute those same bug fixes back into the open source code. This vendor maintenance stream model honors the open source approach because its code base is a strict subset of the open source base, from which it is derived.

In a very different case, if a so-called maintenance stream were to later include vendor features not contributed to the base, the maintenance stream would be a true fork of the base, and would unfortunately lock in customers who used those features. In fact, this code stream could no longer be legitimately called a maintenance stream.

IBM benefits from creating maintenance releases by having code that is likely more stable than open source code, and by having employees who can provide direct support. The team can also provide guidance to product groups about which maintenance version to use to ensure optimal interoperability across product families.

Though this maintenance release model has worked relatively well, some problems still arose. The popularity of open source software attracted many developers to get downloads directly from Apache instead of from the internal maintenance release site. The team responsible for creating and distributing maintenance releases needed to raise awareness across the product groups about the maintenance releases. As well, some product groups were very keen on taking advantage of the latest features that had been added to the Apache Xerces [13] code base, features that had not yet been versioned off into a maintenance release. The workaround was that the early adopter product group would use the open source version for initial product development and testing, ensuring that they swapped out that version for a maintenance release before their product shipped. The maintenance release team needed to proactively find out the usage plans from the key product groups to ensure that their standards needs were met, and that the delivery schedule of maintenance releases lined up adequately with those of the product groups.

IBM makes its maintenance releases available to the public at alphaWorks [9]. This site also includes discussion forums, which compliment open source support via their mailing lists. IBM customers can get formal support by purchasing an IBM product that ships and externalizes the open source code.

Figure 4 shows examples of how standards map to open source implementations, maintenance version names, and some IBM products that include maintenance versions.

If a vendor simply wants to keep pace with open standards, why should they bother working in open source? Why not just create their own proprietary implementation of the standard? This may in fact make sense if the standard is already well accepted and there is no open source implementation to leverage, or if the vendor feels they may have a competitive edge by not sharing their implementation. These are legitimate business strategies. But note the key benefits to a vendor in supporting open source in this model: a) increase mindshare of a new standard across the industry, and b) not have to bear the full cost of implementing the standard, and c) add one more voice against the potential for lock-in by proprietary implementations.

IBM products benefit immensely from including open source software, and open source software derives clear direction from standards bodies. Figure 5 illustrates the lifecycle we have been discussing. Although there are challenges in working through aspects of this process, it has proven beneficial to IBM and can also work for other vendors.

8. Conclusion

To define the core building blocks for integrating software, the industry will advance more quickly if we collaborate. IBM strives to take a leadership position in the area of Internet interoperability by its strong support of open standards and standards bodies, implementation of those standards in the open source community, and re-use of such implementations in IBM products.