State, State Graphs and Transition testing

State graphs, good & bad state graphs
State Graphs by nature are abstract models of behavior of the system. As such, while building state graph, we need to select relevant states, inputs, and transitions and thereby ignore irrelevant ones. In the software test design context, we need to deal with good as well as bad state graphs.

In order to judge whether state graph is good one, we can use following principles:

1. The total number of states in a given state graph is equal to product of the possibilities of factors that make up the state.
2. For every state and input, there is a unique transition to exactly one state and may be the same state itself
3. For every transition there is one specific output action.
4. For every state there is a sequence of inputs that will drive the system back to the same state

Bad state graphs can exhibit certain properties like the following:

1. No exit node which does not allow leaving the state graph. This situation violates the principle of requirement of at least one exit point. This is as depicted in the following Figure.
   
   
2. Transition does not permit entry into a particular node. This is as shown below:
   
   
3. No transition can be made from State A and State B and thus States A and B become non-reachable. This is as shown below:
   
   
4. 4. No unique transition for a the same input thereby violating the principle that there shall be unique transition for a specific input. This is as shown below:
   
   

The above ones are few examples of improper state graphs

Bugs in State Graphs
When testing an implementation against a State Graph, one shall study the following typical bugs (incorrect sequences of events, transitions, or actions):

• Unspecified transition (nothing happens with an event)
• Incorrect transition (the resultant state is incorrect)
• Unspecified or incorrect event
• Unspecified or incorrect action (wrong things happen as a result of a transition)
• Extra, unspecified or corrupt state
• Unreachable State (A state that no input sequence can reach
• Dead State ( A state that once entered cannot be exited)
• Sneak path (an event is accepted when it should not be)
• Trap door (the implementation accepts undefined events)

The Role of State Graphs in Software Testing
State Graphs provide framework for a model testing, where a State Graph is executed or simulated with event sequences as test cases, before starting the actual implementation phase. State Graphs specify system specification and support for testing the system implementation against the system specification. State testing is primarily a functional testing tool which is very effective when used in the early phases of design. With the help of State Graphs we can generate test cases based on following steps:

• Explicit mapping shall be done between the elements of the State Graph (states, events, actions, transitions, guards) and the elements of the implementation (e.g., classes, objects, attributes, messages, methods, expressions)
• Ensure that the current state of the State Graph underlying the implementation must be checkable, either by the runtime environment or by the implementation itself (built-in tests with, e.g., assertions and class invariants)

Test Design Strategies for State Graph based Testing
Test cases derived from State Graph are sequence of input events. In state graphs, exhaustive test case designing in order to execute every path at least once is impossible and unpractical. As such, test cases using state graph model can be designed using following notions or rules:

• All states coverage is achieved when each state of the State Graph is exercised at least once during testing. This is usually not a sufficient level of coverage, because behaviour faults are only accidentally found. If there is a bug in a transition between a specific state pair, it can be missed even if all states coverage is reached.
• All-events coverage: Each event of the State Graph is included in the test suite (is part of at least one test case)
• All-actions coverage: Each action is executed at least once
• All-transitions coverage: All-transitions coverage is achieved when the test executes every transition in the model at least once. This automatically entails also all states coverage. Reaching all transitions coverage doesn’t require that any specific sequence is executed, as long as all transitions are executed once. A bug that is revealed only when a specific sequence of transitions is executed is missed even in this coverage level. The coverage can be increased by requiring All n-transition coverage, meaning that all possible transition sequences of n or more transitions are included in the test suite.
• All n-transition sequences:
  • Every transition sequence generated by n events is exercised at least once
  • All transitions = all 1-transition sequences
  • All n-transition sequences implies (subsumes) all (n-1)-transition sequences
• All round-trip paths: every sequence of transitions beginning and ending in the same state is exercised at least once

What is State Transition Testing?
State Transition testing, a black box testing technique, in which outputs are triggered by changes to the input conditions or changes to 'state' of the system. In other words, tests are designed to execute valid and invalid state transitions

When to use?

• When we have sequence of events that occur and associated conditions that apply to those events
• When the proper handling of a particular event depends on the events and conditions that have occurred in the past
• It is used for real time systems with various states and transitions involved
  
  Deriving Test cases:
• Understand the various state and transition and mark each valid and invalid state
• Defining a sequence of an event that leads to an allowed test ending state
• Each one of those visited state and traversed transition should be noted down
• Steps 2 and 3 should be repeated until all states have been visited and all transitions traversed
• For test cases to have a good coverage, actual input values and the actual output values have to be generated
  
  Advantages:
• Allows testers to familiarise with the software design and enables them to design tests effectively.
• It also enables testers to cover the unplanned or invalid states.

Example:
A System's transition is represented as shown in the below diagram:

The tests are derived from the above state and transition and below are the possible scenarios that need to be tested.

Tests	Test 1	Test 2	Test 3
Start State	Off	On	On
Input	Switch ON	Switch Off	Switch off
Output	Light ON	Light Off	Fault
Finish State	ON	OFF	On

What is Static Testing?
Static Testing, a software testing technique in which the software is tested without executing the code. It has two parts as listed below:

• Review - Typically used to find and eliminate errors or ambiguities in documents such as requirements, design, test cases, etc.
• Static analysis - The code written by developers are analysed (usually by tools) for structural defects that may lead to defects.

Types of Reviews:
The types of reviews can be given by a simple diagram:

Static Analysis - By Tools:
Following are the types of defects found by the tools during static analysis:

• A variable with an undefined value
• Inconsistent interface between modules and components
• Variables that are declared but never used
• Unreachable code (or) Dead Code
• Programming standards violations
• Security vulnerabilities
• Syntax violations

What is Statistical Testing (ST)?
Statistical Testing makes use of statistical methods to determine the reliability of the program. Statistical testing focuses on how faulty programs can affect its operating conditions.

How to perform ST?

• Software is tested with the test data that statistically models the working environment.
• Failures are collated and analyzed.
• From the computed data, an estimate of program's failure rate is calculated.
• A Statistical method for testing the possible paths is computed by building an algebraic function.
• Statistical testing is a bootless activity as the intent is NOT to find defects.

You can use State Table to determine invalid system transitions. In a state Table, all the valid states are listed on the left side of the table, and the events that cause them on the top. Each cell represents the state system will move to when the corresponding event occurs.

For example, while in S1 state you enter a correct password you are taken to state S6 (Access Granted). Suppose if you have entered the wrong password at first attempt you will be taken to state S3 or 2nd Try. Likewise, you can determine all other states. Two invalid states are highlighted using this method which basically means, what happens when you are already logged into the application and you open another instance of flight reservation and enter valid or invalid passwords for the same agent.

Testability Tips
In order to design testability we need to build explicit state diagrams. Also, testability is easy if the State Graph is designed using only two states.

Also, good amount of effort shall be directed by programmers towards identifying what type of behaviors shall be considered to arrive at state graphs and what behaviors shall be ignored. If programmers ignore this and arrive at state graph that is quite comfortable to work with, then the very purpose of using state graphs as the basis for model based testing is beaten since in model based testing there is a need to identify relevant states, inputs, and transitions and ignoring irrelevant states, inputs, and transitions with a specific rationale behind it.

State graph is an advanced functional testing technique. Concepts of state graph help us in building state graph model from specifications in order to perform model based testing. It is essential a state graph shall be good. In order to ensure that the state graph we arrived at is good – it shall be verified to check whether state graph that is modelled from specifications is correct, complete and consistent enough for implementation of testing. State graph based testing do not support exhaustive testing. Also, it is very difficult to design state graph for complex and larger systems. As such state graphs are not good candidates for designing exhaustive testing and also, shall be considered for designing test cases of complex and larger systems.

Software Testability is a non-functional requirement that tell us about the ease with which we can test the software. It should be added in software so that testcases and test scripts can be executed thoroughly. Formally, “The extent to which a software system or its component enables the setting up of test criteria and performance of tests to conclude whether those criteria have been met or not”.

“Low Software Testability” in lot of circumstances degrades the software slowly as it may not be detected at once; the testers, unaware of the fact, may consider wrong reasons and to handle the problem may recommend numerous solutions like extending work hours, assigning more resources, expensive automation tools, risk based testing, need for better estimation and planning, etc. without the proper understanding the problem. This tends to aggravate the situation, as the real problem will remain unfocused.

Software Testability is a prerequisite for software development as any SDLC encompasses requirements gathering, analysis, design, coding, testing, implementation, and maintenance. Complete execution of the test scripts can only be ensured if the application that is being developed is significantly testable. Once decent test coverage is applied, most of the defects will be uncovered and fixed before the product goes live in market which, in turn will result in lesser issues being reported by the end users.

Published date : 03 Nov 2015 02:49PM