2.  Prepare test plan and test cases
The test plan and test cases are made up based on the Functional Design Specification. "Test plan" defines the scope, method, resource and process of testing activities. This specification outlines the testing project, testing qualities, testing tasks, task principals, and risks. Most of the time the test plan also includes system testing. Because unit testing and integration testing are the responsibilities of developers, they are provided for in the development plan. A simple method of estimating the amount of effort for the various testing phases is to assume that the execution time for each phase is approximately 1.1 or 1.5 times of the previous phase time. For example, system testing time is generally 1.1 to 1.5 times the length of the programming phase (includes unit testing and integration testing).

One of the important aspects of the test plan is the controlling change. The changes mainly come from the project plan, requirement, testing software version, and testing resource. Dealing with change effectively reduces risk.

The test case also provides the description of the testing tasks, testing scheme, method, technique, and strategy. The content involves testing objectives, test environment, input data, testing steps, expected results, and testing scripts. As the key of software testing, the test case guides the execution of the integration test, system test, and regression test. In addition, the test case could be used to measure the test results and analyze the software defects to improve the software quality. To improve the test case design, some mistaken ideas about the test case must be discarded. The test case does not aim to detect "uncommon" defects. As the foundation of testing, the test case must cover the test requirement totally and shouldn't be judged from individual cases. The degree of detail in the test case should account for the testers' familiarity with the software, neither too brief nor too detailed is appropriate. Testers must remember that test cases are "live"; they should be updated during the entire testing process. If not updated with the requirements and design synchronously, test cases could become invalidated.

For any test cases, obvious validation methods need to be added. It is necessary for the tester to judge whether the output is coincident with the expected results.

Similarly, the test plan and test cases need to be audited by the project manager and developers. Only then can all team members have a unified understanding of the project.

3.  Testing implementation
The general testing procedure is to execute the test cases referring to the test plan. It includes programming the automatic testing scripts and executing the scripts repetitiously. It also includes executing the manual test cases. In the testing period, the sequence is from simple to complex; from simple functional testing to integration functional testing; from surface bugs to deep bugs. As the project's developing and testing process proceeds, the product functions and quality become more enhanced. It's necessary to execute test cases repetitiously, particularly on each build during automation test to prevent quality regression. Another task of testers is to maintain the test cases. If problems exist in the test design in the initial specification or bugs come from customers and are not covered in current test cases, some new test cases should be added.

Testing Automation
To advance the testing of Workspace, we use the IBM Rational Functional Tester (RFT) tool to perform the test automation.

As shown in Figure3, the process of automatic testing could be separated into two phases:
1.  Prepare Scripts

• Begin REC: Open the RFT and record the operations that follow the test cases strictly.
• Increase script function: Insert verification points, testing parameters and If Else or Loop control statements to enhance the regression test automation.
• Debug script: Debug and validate the scripts.

2.  Regression Test

• Execute scripts: With the help of testing automation tools, execute scripts and validate the software.
• Result analysis: Inspect the log files to check the execution results, analyze the software problems and report them.

The use of test automation tools improves the testing efficiency and helps testers pay more attention to developing new testing modules and raises the coverage ratio of test cases. In addition, test automation provides the digitization management of testing assets and these testing assets could be reused in the function and regression tests.

Software Problem Report (SPR)
A software problem report (SPR) is one formal report for QA (Quality Assurance) to fill in with software problem information found in testing. This document has a unified format. The SPR should supply the problem description and help developers correct the problem; it is also used to track the problem. The high-quality SPR involves the following information: Problem ID, topic, detailed description, product or project, version of product, function or module, origin, severity, priority, attach, state, requested by, requested Date, assign to, resolution, and version of resolution.

An SPR has six states in its life cycle from creation to closed: Create, Open, WaitToVerify, Resolve, and Close. Figure 4 shows the changing process between different states. In any phase of software development, whenever a problem is found, a SPR should be created and set to the state of New. After QA has verified the problem is a bug, the state is changed to Open. On the other hand, if QA is unable to verify the problem or determines it's not a bug, the state is changed to Close.

When a developer receives the SPR, he should first attempt to reproduce it. If he is unable to reproduce it, he will suggest that QA verify and close it. Otherwise the developer fixes the bug, records the information about the fix, and then changes the SPR state to WaitToVerify to wait the QA's verification. When the QA group has confirmed that the modification resolves the bug, the SPR state is changed to Close. If they determine that it has not, the state is changed to Open. Based on the state chart, every SPR could be traced and ensured to be solved.

Testing Evaluation Methods
1.  Test case priority analysis
When there is not enough time for tough test tasks, some method of prioritizing among the test cases is needed to ensure the important test case could be implemented first. To do this, we need to define the priority for each test case.

We define the "function priority" and "content priority" and multiply the two priorities to get the final priority of each test case.

Function priority:
3.  Important function
2.  Common function
1.  Uncommon function

Content priority:
3.  Normal condition;
2.  Important abnormity condition;
1.  Unimportant abnormity condition;

Test case priority = Function priority * Content priority;