It's impossible to say, really, because the answer varies from one organization to another, and even from one tester or developer to another.

I think most people would agree that it's easier to identify a product that lacks quality. When common features are hard to find, buttons aren't located in what we think are the right places, transactions take too long, or don't provide feedback about their status—these are things that we would consider poor quality.

"The concept of quality has little meaning unless it is related to a specific service, object, experience, or desired result," says Frank Grossman, president and cofounder of Mindreef, whose flagship SOAPscope helped set the standard for Web services testing tools before most people understood the meaning of the term.

To date, software quality initiatives have been most often associated with testing, Grossman says, "and more recently, testing a single Web service using a waterfall approach." This approach is often adequate for a Web service that has a distinct function with a specific life cycle, and can be designed, developed, debugged and deployed. "It then exists in production until being revised, updated and replaced with a new service, which is subjected to the same cyclical process to ensure quality."

On the other hand, SOA deployments don't have such a life cycle, asserts Grossman. "SOA is an architecture made up of infrastructure and services that must constantly interoperate with each other. It does not go offline and it cannot be replaced. It can, however, evolve and expand. It can also improve or degrade." Therefore, the quality of an SOA is reflected by the amount of use and reuse of the services within it, and by how well its implementation meets the needs of the business even as those needs evolve, Grossman says.

This is achieved, according to Grossman, through continual optimization of all components within an SOA environment to ensure maximum adoption and reuse of services, as well as the business agility promised by the technology. "SOA quality is a key component of reaping the intended benefits of an SOA—it's the strategy needed to achieve maximum business benefit."

Five (Not So) Easy Pieces
Grossman, whose SoftICE and BoundsChecker products were ultimately acquired by Compuware, identifies five primary building blocks for developing a quality SOA. How many have your team put into practice?

Prototyping is defined as one of the best ways to reach agreement on a WSDL before any code is written. "Seeing is believing," says Grossman. "Prototyping lets business analysts, architects and developers design and develop very usable interfaces early in the process, thereby creating services designed for reuse. It also allows consumers and testers to get involved much earlier in the design process, reducing the overall development cycle."

Compliance with standards is the cornerstone of any quality project, and its absence is the surest way to smash any hope of positive business returns. "Developers must embrace standards, rules and policies by seeing value in what they provide. Architects and governance teams must educate others as to what the standards are, why they are in place, and most importantly, how developers can improve their projects to become compliant. Finally, analysts and architects must embrace compliant services and leverage them in SOA applications to ensure trust and reuse."

Testing as many Web services as possible, as well as their interactions and dependencies, is critical and should take place throughout all stages of development, says Grossman. "Teams need tools that can provide the necessary user interface, simulate unavailable services, and ensure that all team members—even ones without programming or XML language knowledge—can test services. It is also important that test scripts are accessible so retesting can be easily performed when a service policy changes or if a new service consumer uses the service in a different way."

Diagnostics is defined by Grossman as the ability to solve software issues quickly, even in real-time. "No matter how well Web services and business processes are tested, problems will still occur; after all, it's still software. Determining if a Web service can perform its intended function is often a time-sensitive issue that may require fast identification of a root cause problem." All team members should be able to help diagnose problems, he says.

Support of an SOA development differs from that of conventional applications, Grossman, says, because it involves two phases that span design time and runtime simultaneously. "As services are exposed for use and reuse, consumers will seek support to assist in the development of their applications. In production, support teams need to understand and resolve problems quickly, and often need to reproduce scenarios when a failure occurs. When service developers need to get involved, complete problem data needs to be shared with other team members with the ability to simulate different scenarios, to more effectively diagnose problems."

In addition, your team needs a sound infrastructure that is built on communication, trust and control. Communication includes the ability to collaborate among team members and for services to interoperate in and outside your organization; trust is earned when your team and its services behave as expected; and control is exercised in the form of governance and policy enforcement.