Wednesday, January 12, 2011

Cyclomatic Complexity

Cyclomatic complexity is a software metric that provides a quantitative measure of the logical complexity of a program. When used in the context of the basis path-testing method, the value computed for cyclomatic complexity defines the number of independent paths in the basis set of a program, and provides us with an upper bound for the number of tests that must be conducted to ensure that all statements have been executed at least once.
An independent path is any path through the program that introduces at least one new set of processing statements or a new condition.

Cyclomatic complexity has a foundation in graph theory and is computed in one of three ways:
  1. The number of regions corresponds to the cyclomatic complexity.

  2. Cyclomatic complexity, V (G), for a flowgraph G, is defined as
    V(G)=E-N+2,
    where
    E=Number of flowgraph edges
    N=Number of flowgraph nodes.
  3.  Cyclomatic complexity, V (G) for a flowgraph G, is also defined as

    V(G)=P+1,
    where
    P = Number of predicate nodes contained in flow graphs G. 
    The value V (G) provides us with an upper bound for the number of independent paths that comprise the basis set, and by implication, an upper bound on the number of tests that must be designed and executed to guarantee coverage of all program statements.

Performance , Load and Stress testing

     Performance testing. This type of testing determines or validates the speed, scalability,
     and/or   stability characteristics of the system or application under test.
     Performance is concerned  with achieving response times, throughput, and
     resource-utilization levels that meet the performance objectives for the project or product.  
     
     Load testing. Load testing is focused on determining or validating  
      performance characteristics of the system or application
      under test when subjected to workloads and load volumes anticipated
      during production operations. 
     
    Stress testing. This type of testing  is focused on determining or
     validating performance characteristics of the system or application under test when subjected
     to  conditions beyond those anticipated during production operations.
     Stress tests may also include tests focused on determining or validating performance
     characteristics of the system or application under test when subjected to other
     stressful conditions, such as limited memory, insufficient disk space, or server failure.
     These tests are designed to determine under what conditions an application will fail,
     how it will fail, and what indicators can be monitored to warn of an impending failure.

Jmeter Intro

Apache JMeter is  an open source Java desktop application used to do performance testing.
JMeter can simulate a heavy load on a server, network or object to test its strength or
to analyze overall performance under different load types.
It can also be used for regression testing.
           To run JMeter, click on JMeter executable file.Once JMeter is opened,we will see two options-
Test  Plan and Workbench.
   A test plan describes a series of steps JMeter will execute when run.
We can see two check boxes on   this page-
                 1.Run Thread groups consecutively (i.e. run groups one at a time)
                 2.Functional Test Mode (i.e. save response data and sampler data).

In performance testing process, we might want to test one thread group after other rather to test all the thread groups at one time. At that time 1st option is used.
With JMeter,we can also perform functional testing so there 2nd option comes in picture.

When we right click on test plan or click on edit + add, we can see different elements like threads(Users), config element, timer, pre processors, post processors, assertions, listener.

1.Thread group- In thread group,we can name the thread group with name option and also add comment. If for any reason,error is encountered,then what action is to be taken can be determined by selecting one of the four options - 1.continue 2.stop thread 3. stop test 4. stop test now
In thread properties,  we can determine
          (1).Number of threads(users)
          (2).Ramp-up Period(in seconds)- It tells JMeter how long to take to
               "ramp-up" ( or up and running) to the full number of threads chosen.
              e.g.- If thread number =10 and ramp-up period=100 sec,
              then JMeter will take 100/10 seconds to start each thread and total 100 sec.
          (3).Loop count - we can run the loop forever or for a specific count
          (4).Scheduler-
                (i) Start Time
                (ii)End Time
                (iii)Duration(seconds)
                (iv)Start Up Delay(seconds)

2. Logic Controller -It lets user customize the logic that JMeter uses to decide when to send requests. Logic Controllers can change the order of requests coming from their child elements.

3. Config Element - A config element can add to or modify requests,but can not send requests(exception is HTTP Proxy Server). A config element is accessible only from inside the tree branch where you place the element. It works closely with Sampler.

4. Pre-Processor Elements -These executes some action before a sampler request is made.
A pre-Processor is most often used to modify the settings of a Sample Request just before it runs, or to update variables that aren't extracted from response text.

5. Post Preprocessor Elements - A Post-Processor executes some action after a Sampler Request has been made. A Post-Processor is most often used to process the response data, often to extract values from it.

6. Timer - Timer is used to add delay between each request. By default, JMeter thread sends requests without any delay between each request. If delay is not introduced between threads,JMeter could overwhelm server by making too many requests in a short amount of time. If more than one timer is introduced to a thread group,then JMeter takes the sum of the timers and pauses for that amount of time before executing samplers.

7. Assertions - Assertions allow you to assert facts about responses received from the server being  tested. Using an assertion, you can essentially "test" that your application is returning the results you expect it to.It is a great way to check if response data satisfies the conditions.To view the assertion results, add an Assertion Listener to the Thread Group. Failed Assertions will also show up in the Tree View and Table Listeners, and will count towards the error percentage.

8. Listener - Listeners are a way to show results. Listeners can save data for later use. There are many listeners which essentially present  the data/result in different format.
e.g.-Graph result listener plots the response time on a graph,summary listener shows the summary and aggregate listener shows the aggregate.
      Listeners can be added anywhere in the test, including directly under the test plan. They will collect data only from elements at or below their level.

9. Sampler - Samplers tells JMeter to send request to the server and wait for the response.
Samplers perform the actual work of JMeter. Each sampler (except Test Action) generates one or more sample results. The sample results have various attributes (success/fail, elapsed time, data size etc) and can be viewed in the various listeners.

Thursday, January 6, 2011

Metrics in software development

Metric is a measurement used to compare two or more products,processes,or projects.
Software metrics help in-
  • Estimation of size and complexity of project
  • Tracking the progress of project
  • Analyzing the effectiveness of software progress and taking corrective action if necessary.
  • Measuring productivity of people.
1. Person Month Metric - The effort required for execution of project is measured in person months(PMs).
e.g.-If a person works for a month,effort is 1 PM.

2. Product Metrics - To estimating the size of project is very difficult task.One accepted method of estimating product size is using the metric KDSI(Kilo or Thousand delivered source instructions) or KLOC(Kilo Lines of code).
Based on KDSI,a project can be categorized as small,intermediate,medium,large or very large.

3. Productivity Metrics - The number of lines that can be written by a programmer in one hour can be used as a metric for measuring the productivity of programmer i.e. the Delivered Source Instructions (DSI) per hour.
Another metric used for productivity is the number of defects removed per hour by programmer.

4. Quality Metrics - To measure the quality of product,following metrics can be used -

  • Number of defects found per KDSI(Known as defect density)
  • Number of chages requested by customer after software is delivered.
  • MTBF(Mean Rime Between Failures) i.e. average time between failures.
  • MTTR(Mean Time To Repair) i.e. average time required to remove a defect after it is detected.

Wednesday, January 5, 2011

Testing Terminologies & definitions

Software Testing- Testing is the process of executing a program with the intent of finding errors.

Validation-Validation is to check whether the software meets customers expectations. (Building the right product).

Verification-Verification is to check whether the software conforms the specifications.It is done by development team to ensure that software is as per SRS document.(Building the product right ).

Test Case-Test case is sets of input parameters.Test cases describes how the test should be carried out.

Black Box Testing-Black box testing is the testing carried out to check functionality of the software without considering internal structure of code .

White Box Testing-White box testing is the testing carried out taking into the implementation details.

Software Requirements Specification-A document that captures the requirements of the proposed software.

Baseline- A measurement of where your processes are at any given point of time.It is used to compare one group at any given time to the same group at another point in time.

Benchmark- A measurement of where your process are compared directly to other companies or static model such as CMM.

Bug/Defect- A flaw in the software with potential to cause a failure.

Failure- Any deviation of a system that prevents it from accomplishing its mission or operating within specification.It is the manifestation of a defect.

Cyclomatic Complexity-  A technique using mathematical graph theory to describe the complexity of a software module.

Decision Table-Tables that list all possible conditions(inputs) & all possible actions(outputs).

Defect Age- A measurement that describes the period of time from the introduction of a defect until its discovery.

Exploratory Testing- A testing technique where the test design and execution are conducted concurrently.

Incident-Any unusual result of executing a test.

Latent Defect- An existing defect that has not yet caused a failure because the exact set of conditions has not been met.

Masked Defect - An existing defect that hasn't yet caused a failure because another defect has prevented that part of the code from being executed.

Milestone-A major checkpoint or a sub-goal indentified on the project or testing schedule.

Mutation Analysis- Purposely altering a program from its intended version in order to evaluate the ability of the test cases to detect the alteration.

Random Testing- Testing using data that is in the format of real data, but with all of the fields generated randomly.

Regression Testing-Retesting previously tested features to ensure that a change or bug fix has not affected them.

Requirement Traceability-Demonstrating that all requirements are covered by one or more test cases.

Scaffolding Code- Code that simulates the function of non-existent components(e.g.-stubs and drivers).

Smoke Testing-A test run to demonstrate that the basic functionality of a system exists and that a certain level of stability has been achieved.Frequently used as part of the entrance criteria to a level of test.

Use Case-A use-case describes a sequence of interactions between an external "actor" and a system,which results in the actor accomplishing a task that provides a benefit to someone.

Latency - Latency is a measure of responsiveness that represents the time it takes to complete the execution of a request. Latency may also represent the sum of several latencies or subtasks. 


Response time - Response time is a measure of how responsive an application or subsystem is to a client request.

Scalability - Scalability refers to an application’s ability to handle additional workload, without adversely affecting performance, by adding resources such as processor, memory, and storage capacity.

Throughput - Throughput is the number of units of work that can be handled per unit of time; for instance, requests per second, calls per day, hits per second, reports per year, etc.

Wednesday, December 29, 2010

Equivalence Partitioning & Boundary-value analysis

Equivalence Partitioning-
A good test case is a test case which has a reasonable probability of finding an error.But as we know,exhaustive input test of program is impossible and that is why we are limited to trying a small subset of all possible inputs.So we want to select the right subset which has highest possibility of finding the most errors. One of the possible way of it is equivalence partitioning.

The equivalence classes are identified by taking each input condition and partitionoing it into two or more groups.
e.g.-for a "X" condition,we can partition it in the valid equivalence class which represents all the valid inputs and invalid equivalence class which represents invalid inputs.


Boundary-Value analysis-
In Boundary value analysis,boundary conditions are on,above & below the edges of input  & output equivalence classes.
Unlike equivalence class,it requires one or more elements to be selected such that each edge of the equivalence class.

Monday, December 27, 2010

Walkthroughs

The Walkthrough is a set of procedures and error detection techniques for group code reading. The process of walkthrough is very much similar to code inspection. In walkthrough, generally 3-5 people participate-moderator,secretary who records all errors,tester,the person who will eventually  maintain the program and someone from different program.
In walkthrough,instead of reading the program/code, participants execute the test case mentally.These test cases are prepared by tester. The walthroughs should be followed by follow-ups.