Triaged Tester

January 27, 2009

Stress Testing

Filed under: Stress — Triaged Tester @ 7:16 am
Tags:

Stress test is targeted to find issues with stability and robustness of the product, such as AVs, resource leaks, deadlocks, and various issues/inconsistency under the stress/load condition.

 

Most time stress tests are running in combination with other diagnostic utility such as debugger, page heap, any fault injection tools/instrumentations, performance monitor, and all other necessary tracing and logging tools that deemed to be useful for debugging and analysis. These are crucial information for developer and tester for analyzing the final outcome of the results.

 

Feature validation is important part of the stress, though it is sometimes not done as much as it should. As a good practice, all stress run should implement feature validation to ensure the system behave consistently under the load condition.

 

Stress tests should also focus on negative conditions. Systems tend to work fine when everything is tested for valid conditions and tends to become unstable when the error paths are hit. It is very important that these error conditions are included in the stress scenarios.

 

Stress test need to be clear on:

1.)  What is the objective of the test? What for, why do you want to do it?

2.)  What is the key primitive/action that is targeted to?  And the code path it covers?

3.)  How it related to release criteria/customer impact? Duration of the runs?

4.)  Stability of the runs, and issues/bugs: AVs, leaks, deadlocks, or data integrity issues, or any other feature inconsistency that occurred during the stress?

5.)  Not often done, but it is also a good practice to report the performance patterns during the stress, thought the perf number obtained in stress is tainted by the instrumentation tools we used?

6.)  What are location/links to dumps, perf logs, trace files …, or simply offending call stack…

7.)  What is the clear context under which this test is conducted? Single user or multiple concurrent users?

8.)  Perf counters that record the stress run on resource utilization: CPU, Disk I/Os, memory consumption, network throughput/usage…

9.)  The environment parameter

10.)       What is the hardware used?s in which the results obtained? Large customer sets/data center

Blog at WordPress.com.