2.  Testing coverage ratio
Testing coverage is the evaluation of testing completeness, it can be calculated from the coverage of test requirements and test cases or executed codes. According to the index of coverage ratio, we can answer the question: "How about the completeness of testing?"

•   Testing coverage ratio based on the requirements
The testing coverage ratio based on the requirements should be calculated many times, especially if it is used to supply the flag of testing coverage in the milestones during testing life cycle (such as the testing coverage of planned, implemented, executed, and succeeded).

Testing coverage ration = T(p,i,x,s) / RfT;

"T" is the number of the test described with thetest processes or test cases (planned, implemented, executed, or succeeded); "RfT" refers to the sum up of requirements for the test.

•   Testing coverage ratio based on codes
The testing coverage ratio based on codes is used to evaluate the amount of code executed; correspondingly, it is the amount of code left. Code coverage could be established on the control flow (statement, branch or path) or data flow. Testing coverage based on codes could be calculated as following:

Testing coverage ratio = Ie / TIic;

"Ie" is the executed item number described with code statements, code branch, code path, data state judging points, or name of data element. "TIic" refers to the total number of items in the code.

For example, when we want to calculate the statement coverage to measure if every executable statement is executed, then the parameter "Ie" refers to the executed number of code statements, and the "TIic" refers to the total number of code statements in the software. The ratio of the two parameters is the testing statement coverage ratio. In black box testing, we can use IBM Rational PureCoverage tool to get the results efficiently.

3.  Analysis of SPR information
During the entire software testing process, we could receive lots of information about SPR and the modification process. Through analyzing the information, we can extract the more valuable information to grasp the work schedule and make decisions.

•   Problem classification in terms of grade or type
The statistics of problem information according to the classification could reflect the problem's appearance situation as a whole. The statistic figure shows us which is the major degrade for most of the problems.

•   New problem time distribution
It is useful to study the new problems' distributing trend in a phase by analyzing new problems with the submitting date. If the curve displays the stable downtrend in recent time, it indicates that the software is becoming stable gradually; if the curve displays continuous large fluctuations even near the deadline, it is necessary to consider delaying the release and spending more time finding the causes.

•   Fixing rate
This could reflect the effects of developers in modification; at the same time, this could be used as one index to evaluate the performance and it could also indicate that the developer needs to execute unit tests more carefully.

•   Testing delay curve
Figure 5 is the testing delay curve. The blue line indicates the delivery date of the program and the red line indicates the finish date of testing. The intervals between the two lines represent the delay of "testing" compared with the "program delivery." The bigger the intervals are, the lower the efficiency of the testing. In this figure, the fifth and sixth testings delayed more time than the others.

•   Modification delay curve
Figure 6 is the modification delay curve. The red line represents the dates that the problems were submitted, and the blue line represents the date that the fix was finished. The intervals between the two curves display the delay situation between "modify" and "problem submit." If the intervals become bigger, it shows that the fixing time has become longer and the fixing rate is lower. In this chart, there are three phases that have longer delays.

Testing Workflow Monitoring
To enhance the cooperation of developers and testers and get an objective evaluation of software quality, a process monitoring method is used in management. It means letting a third party evaluate the work of monitoring. The third party could be the director of the software testing department, someone in SQA, or the director of SQA. According to the different phases of the projects, we separate the monitoring method into three parts according to different time: test beginning period, test implementing period, and test ending period.

Test Beginning Period

• Does the testing work have a specific working range?
• Does the software to be tested have a specific performance index?
• Does the testing work have a specific phase division (such as unit, integration, system, accept)? Does each phase have a specific condition for delivery?
• Has the company confirmed the acceptance level and range with the customer? Is there a specific successful and failed standard for software installation and maintenance?

• Is the defects management workflow standard? Can every defect be rechecked for submitting and closing?
• Is the configuration management work standard? Can all relative versions in the test process be traced back completely? Does the frequency of the testing version provided correspond to the actual requirement?
• Does the resource for key testing activities supply on time? If not, is there a reasonable plan to finish the delayed testing work?
• Do the test strategy, test plan, and test cases pass the formal auditing? Have all the problems in them been corrected?
• Have all the test documents been updated according to the actual condition and been executed strictly?
• Are there some parts missing in following plans according to the initialized testing arrangement?
• Does the testing process follow the company's plan?

• Is the defect trend curve in testing converging toward stability? Are the trend curves of different modules coincident?
• Is there a formal document to judge whether the product could be delivered to the customer?
• Is there a formal final evaluation meeting being held? Does the customer join the meeting? Do all the remaining problems have definite resolutions and every problem has a principal to resolve and trace?
• Does the software match all the delivery conditions established earlier?