Despite increases in software complexity, engineering processes and techniques are essentially the same as they were 10 years ago. While the emerging global supply chain for embedded software has helped in creating new code components, it has made it much more difficult to integrate them. This is because software test and verification methods that worked well enough when embedded projects were smaller and simpler don't scale sufficiently for today's bigger and more complex efforts. As a result product teams encounter a frustrating and all too familiar scenario. Software development generally appears to progress smoothly through the implementation phase. However, from that point on, projects enter long, churning, and unpredictable integration and system test phases. Companies struggle to integrate and verify software from so many sources. As a short-term fix, software managers add more engineers and resources to do the integration and testing. But this approach is limited and expensive. Software releases are still delivered late with compromised quality. Clearly, just adding more people alone won't solve the problem.

Enterprise and desktop computing faced similar challenges as they evolved. To a large degree, both have successfully managed software complexity by moving toward standardized application platforms such as Windows or Linux running on an Intel processor. Embedded computing is more complex, however, because applications must run on different processor families and in different software environments. The challenge of testing and verifying a huge array of software components on all this hardware can be overwhelming. Without achieving a scalable, reliable, and predictable software process, software verification and integration challenges will only worsen with each new generation of embedded hardware.

In the absence of a standardized embedded application platform, the need for a sustainable verification platform becomes compelling. A sustainable verification platform would provide a scaleable shared framework that could be used across the product teams at all stages of the release cycle. It could be reused on next-generation projects. It could even be extended across organizations, vendors, and suppliers across the globe. The question is how do we achieve this? Neither in-house efforts nor outsourcing have fully addressed the problem. New automated processes, innovative technology, and specialized verification expertise are required. A new business solution, verification-as-services aims to fulfill this need and enable smoother integration, more predictable schedules, and better quality products.

A Fresh Approach To Verifying the Software Supply Chain
A new model is required to significantly optimize the delivery of highly integrated complex embedded software. To address the urgent challenges observed in the integration and system test phases, key objectives of the new model would be to validate software components earlier; reuse and integrate code more effectively; automate more test processes, and increase visibility into software quality earlier in the development cycle (see Figure 2).

The centerpiece of this new approach is the so-called test assets. Test assets are fully automated reusable tests that have lasting value because they can be leveraged throughout the development lifecycle, in derivative projects, and by other functional teams to integrate, triage, and verify software. Test assets also provide detailed and ongoing visibility into the health of the software. In an ideal world, and according to the principles of supply chain management, all software suppliers, both internal and external, would create and deliver test assets with their software.

To create and share test assets effectively a unified verification framework is required. The verification framework provides the common tools and environment needed for collaboration with suppliers, better leverage of domain expertise, and highly automated testing and early verification strategies. It provides the common infrastructure to reuse, automate, and execute tests for all components effectively, and to support test management and reporting. The verification framework must facilitate different kinds of testing such as unit level and API-level testing. Further, it must scale well beyond a single engineering group and be able to aggregate test assets from both internal engineering teams as well as external software suppliers.

Besides the verification framework and test assets, new processes must also make the best use of new capabilities. New processes will leverage, reuse, and apply automated test assets as much as possible to verify software at various stages and levels - by individual developers, by functional teams, on fully integrated engineering builds, daily, hourly, even continuously. With automated reporting, the processes would also provide comprehensive quantitative metrics and visibility into quality and completeness at various stages of development (see Figure 3).

Developing a Sustainable Verification Platform
Virtually all embedded software development groups recognize that software verification is a problem but lack an effective and ongoing means of addressing it. Development teams aren't really able to solve the problem effectively because they're trapped in a vicious cycle: they're too busy developing and fixing problems and have no time to do the heavy lifting required to adopt the technology required to implement a verification strategy. Traditional outsourcing efforts often fall short because they fail to provide the specialized capabilities and the scaleable and reusable infrastructure required for embedded application development.

The verification-as-services (VaS) approach brings together the verification expertise, innovative technology, and ongoing process management needed to provide a sustainable platform for verifying embedded code.

Verification-as-services strives to leverage the strengths of the software-as-a-service (SaaS) model popularized by salesforce.com while applying a managed services model for the infrastructure and technology designed specifically for embedded software verification. VaS means an organization doesn't have to manage a large staff of test and verification engineers, because the resources and domain expertise needed are provided as part of the service. The VaS business model focuses on value delivered. Unlike outsourcing models that charge according to headcount, VaS is a variable cost solution that is priced according to the number of components to be verified, the scope of testing required, and the specific goals of the project.

With VaS, all verification activities are codified as part of a monthly subscription service that provides complete management of embedded software verification. This approach eliminates large upfront software license costs and ensures an organization continues to get full value from the solution over time.

A Sustainable Verification Platform in Action
To implement a verification strategy that meets an organization's unique software challenges, corporate culture, and project timelines, testing and integration experts first interview, observe, and document existing development processes. Next the verification platform is integrated into the key systems used in those processes. Finally, verification experts set up software, processes, and reports and execute verification activities as an ongoing service for development groups.

VaS incorporates a scalable infrastructure that removes the obstacles for early test and validation success. At every stage, development teams will be able to seamlessly blend testing and verification activities, including planning, analysis, test execution, and measurement, with the organization's embedded software development processes. Because most development groups have other high-priority objectives to accomplish, VaS automates testing processes, executes tests, certifies components according to predetermined criteria and applies product enhancements on an ongoing basis during the life of the subscription.

By including defect tracking, test-asset development and validation, VaS helps to deliver a common view of quality across disparate parts of an organization during the entire test and integration lifecycle. VaS provides management-level reports on key testing and integration metrics and helps to incorporate rich diagnostic capabilities such as software analysis, testing automation, and management. Verification-as-services raises the visibility of testing and integration so management and individual technical staff members can see concrete results from the increased focus on testing.

Many embedded software development teams have a pressing need to integrate and verify code from multiple sources, yet lack the time and resources to do it. Verification-as-services lets these teams focus on development, while simultaneously enabling a rigorous process for testing and measurement.

Implementing a sustainable verification platform can seem daunting. Fears about potentially disrupting the development work in progress keep many organizations stuck in the past. Still, relying on outdated test and verification methodologies is straining them. It's perpetuating a dysfunctional development process characterized by product quality problems, unpredictable delivery dates, and a climate of ongoing operational stress.

Verification-as-services, on the other hand, offers a new approach that addresses the new challenges of embedded development. It's a scaleable approach that lets teams start small and focus on specific components first then expand test coverage as needed. VaS offers engineering organizations the opportunity to eliminate most manual testing and automate integration, efficiently support multiple product releases, optimize software reuse, and effectively manage their software suppliers. It's a strategic solution that frees organizations from the day-to-day management of software test and verification. The benefits of VaS extend well past engineering. The decision to embrace this new approach will be a strategic business move to improve market competitiveness, enhance organizational effectiveness, and achieve the on-time delivery of higher-quality software products.