Triaged Tester

March 20, 2009

Test cases for Black box

Filed under: Black Box Testing,Test case,Tips — Triaged Tester @ 9:34 am
Tags: , , ,

For Black box testing, the test cases can be generated by using all or a combination of the below techniques.

Graph Based : Software testing begins by creating a graph of important objects and their relationships and then devising a series of tests that will cover the graph so that each objects and their relationships and then devising a series of tests that will cover the graph so that each object and relationship is exercised and error is uncovered.

Error Guesssing – Error Guessing comes with experience with the technology and the project. Error Guessing is the art of guessing where errors can be hidden. There are no specific tools and techniques for this, but you can write test cases depending on the situation. 

Boundary Value Analysis (BVA) is a test data selection technique (Functional Testing technique) where the extreme values are chosen. Boundary values include maximum, minimum, just inside/outside boundaries, typical values, and error values. The hope is that, if a system works correctly for these special values then it will work correctly for all values in between. 

Equivalence partitioning is a testing method that divides the input domain of a program into classes of data from which test cases can be derived.

Compasrision Testing – There are situations where independent versions of software be developed for critical applications, even when only a single version will be used in the delivered computer based system. It is these independent versions which form the basis of a black box testing technique called Comparison testing or back-to-back testing.

The Orthogonal Array Testing Strategy (OATS) is a systematic, statistical way of testing pair-wise interactions by deriving a suitable small set of test cases (from a large number of possibilities).

March 16, 2009

Test Activities during phases

Filed under: Black Box Testing,Checklist,General,Guidelines,Test Plan,Tips — Triaged Tester @ 8:51 am
Tags: ,

Test activities vary with the model and also on the type of project. So here is a generic list of items that needs to be done. You can always do it at any point when enough data is available.

1. Requirement Phase :

  • Invest in analysis at the begining of the project
  • Start developing the test set at the requirement analysis phase
  • The correctness, consistency and completeness of the requirements should be analysed.

2. Design Phase :

  • Analysis of design to check its completeness and consistency
  • Analysis of design to check whether it satisfies the requirements
  • Generation of test data based on design
  • Setting up of test bed

3. Programming/Coding Phase :

  • Check code for consistency with design
  • Perform system testing in an organized manner – Buddy testing, feature testing, integration testing, System testing etc
  • Use available tools
  • Apply stress to the program
  • Test one at a time
  • Measure test coovergae

4. Maintanence Phase :

  • Retest/Regress

March 10, 2009

Performance counter – 4

Filed under: Performance,Tips — Triaged Tester @ 1:24 pm
Tags: ,

Additional web counters  to monitor performance

ASP.NET -> Request Execution Time    Bear in mind that this is only the time it took to execute the mos recent request.
ASP.NET -> Requests Current    Total of waiting and executing requests.
ASP.NET -> Request Wait Time    If wait time is going up and CPU is low then opening up the threading options will probably help.  This could also indicate the need to additional hardware to hand the load.
ASP.NET -> Requests Queued     
ASP.NET Apps -> Pipeline Instance Count  Make sure this is for the Web Service App Only on thread at time can use a Pipelin instance.  So a sudden increase can indicate backend latency.
ASP.NET Apps -> Requests Executing  Make sure this is for the Web Service App  
ASP.NET Apps -> Requests Total  Make sure this is for the Web Service App  
ASP.NET Apps -> Requests/sec Make sure this is for the Web Service App How quickly are we handling requests?

March 6, 2009

Choosing what to automate

Filed under: Automation,Tips — Triaged Tester @ 1:22 pm
Tags: ,

Most, but not all, types of tests can be automated. Certain types of tests like user comprehension tests, tests that run only once, and tests that require constant human intervention are usually not worth the investment to automate. The following are examples of criteria that can be used to identify tests that are prime candidates for automation.

High Path Frequency – Automated testing can be used to verify the performance of application paths that are used with a high degree of frequency when the software is running in full production.  Examples include: creating customer records, invoicing and other high volume activities where software failures would occur frequently.

Critical Business Processes – In many situations, software applications can literally define or control the core of a company’s business. If the application fails, the company can face extreme disruptions in critical operations. Mission-critical processes are prime candidates for automated testing. Examples include: financial month-end closings, production planning, sales order entry and other core activities.  Any application with a high-degree of risk associated with a failure is a good candidate for test automation.

Repetitive Testing – If a testing procedure can be reused many times, it is also a prime candidate for automation. For example, common outline files can be created to establish a testing session, close a testing session and apply testing values. These automated modules can be used again and again without having to rebuild the test scripts. This modular approach saves time and money when compared to creating a new end-to-end script for each and every test.

Applications with a Long Life Span – If an application is planned to be in production for a long period of time, the greater the benefits are from automation

March 4, 2009

Performance counter – 3

Filed under: Performance,Tips — Triaged Tester @ 1:20 pm
Tags: ,

performance monitor for web server

Counter Name Collection Notes Comments
.NET CLR Loading :Bytes in Loader Heap Make sure this for aspnet_wp The number of bytes committed by the class loader across all AppDomains. This counter should reach a steady state. If this counter is continuously increasing, monitor the “Current Assemblies” counter. There may be too many assemblies loaded per AppDomain.
.NET CLR Memory: % Time in GC   Make sure this for aspnet_wp The percentage of time spent performing the last garbage collection. An average value of 5% or less would be considered healthy, but spikes larger than this are not uncommon. Note that all threads are suspended during a garbage collection
.NET CLR Memory -> # Gen 0 Collections Make sure this for aspnet_wp These three GC counters provide insight on the efficiancy of the GC.  Basically Gen 0 should be the highest.  Gen 1 should be at least 1/2 of Gen 0’s count and usually more like 10%.  Gen 2 should be the lowest.  It should be about 1/2 of Gen 1 and the less the better.  As we move up through the generations GC operations are more expensive so we want to watch and use these to determine if we need to examine allocation patterns.
.NET CLR Memory -> # Gen 1 Collections Make sure this for aspnet_wp  
.NET CLR Memory -> # Gen 2 Collections Make sure this for aspnet_wp  
.NET CLR Memory -> # Bytes in all Heaps Make sure this for aspnet_wp The number of bytes that are allocated currently by .NET.  This should have peaks and valleys.  A steady climb can indicate a memory leak or memory usage issue.
.NET CLR Exceptions -> # of Exceps Thrown Make sure this for aspnet_wp Exceptions are a very expensive thing yet very powerful thing.  They can be very helpful but should be an “exceptional” occurance.  Essentially the could should be low and not constantly climbing.  If there is an exception that is occuring on every call to some function that function should be analyzed to add validation logic to avoid the exception if possible.  If it has been identified that there are too many exceptions a debugger should be used to track down the type of exception that is causing the problem.
ASP.NET: Requests Queued   The number of requests currently queued. When running on IIS 5.0, there is a queue between inetinfo and aspnet_wp, and there is one queue for each virtual directory.
ASP.NET: Worker Processes Running   The current number of aspnet_wp worker processes. For a short period of time, a new worker process and the worker process that is being replaced may coexist. Although rare, sometimes processes deadlock while they are exiting. If you suspect this, consider using a tool to monitor the number of instances of the worker process. Alternatively, the Memory\Available Mbytes performance counter can be used, since these hanging processes will consume memory.  
ASP.NET: Application Restarts If this counter is going up you should also monitor: .NET CLR Loading -> Total AppDomains and Current AppDomains.  If current is growing line with Total then you know that you have and issue where the AppDomain is not getting unloaded on Application Restart and this can cause a leak. When the ASP.NET Application Restarts all of the assemblies have to be unloaded, reloaded, reJITed, and all of the application memory objects must be reloaded.  Restarts typicall occur when editing the Web.Config or some other change to application filies.  A restart every once in a while is not a problem, but if they happen frequently this can become a performance issue.
ASP.NET: Worker Process Restarts   This is related to the number of Application Restarts and has similar performance implications. However the cause is typically things like the process crashing. This can also occur when hitting memory, queue limit, or timeout thresholds.
Process: % Processor Time. Make sure this for aspnet_wp The percentage of time the threads of this process spend using the processors. Threshold: 70%. Values greater than this for extended periods of time indicate a need to purchase hardware or optimize your application.
Process:Thread Count Make sure this for aspnet_wp The number of threads active in this process. Thread count often increases when the load is too high.
Process -> Private Bytes Make sure this for aspnet_wp If the application is completely managed this should grow in parrellel to # Bytes in All Heaps.  If this counter is growing consitently and # Bytes in All Heaps is level that is an indication that there is an unmanaged mem leak.
Process -> Virtual Bytes Make sure this for aspnet_wp Most application run out of addressable memory before they run into any physical memory limitation so this counter is very imporant to monitor because if it goes above 1.2 GB the chances of an out of memory condition are VERY high.
Process -> Handle Count Make sure this for aspnet_wp The OS has a finite amount of handles for things like Files, Events and other OS objects so it is important in a web scenario to make sure that you are not leaking handles these.
Memory -> Available MBytes   Import when considering how often we are paging.  This count also can impact how frequently the GC needs to run
Web Service: Current Connections   A threshold for this counter is dependent upon many variables, such as the type of requests (ISAPI, CGI, static HTML, and so on), CPU utilization, and so on. A threshold should be developed through experience.
Web Service: Total Method Requests/sec   Used primarily as a metric for diagnosing performance issues. It can be interesting to compare this with “ASP.NET Applications\Requests/sec” and “Web Service\ISAPI Extension Requests/sec” in order to see the percentage of static pages served versus pages rendered by aspnet_isapi.dll.
Web Service: ISAPI Extension Requests/sec   Used primarily as a metric for diagnosing performance issues. It can be interesting to compare this with “ASP.NET Applications\Requests/sec” and “Web Service\Total Method Requests/sec.” Note that this includes requests to all ISAPI extensions, not just aspnet_isapi.dll.
Next Page »

Blog at