Fundamentals of Testing

By,

Smt.Pavani Priyadarshini,

Software Test Engineer

 

Introduction:
In this chapter, we will introduce you to the fundamentals of testing: Why Testing is needed, its limitations, objectives and purpose; the principle behind testing: the process that testers fallow: and some of the psychological factors that testers must consider in their work.

By reading this chapter we will gain an understanding of the fundamentals of testing and be able to describe those fundamentals. We will describe and illustrate how software defects or bugs can cause problem for people, the environment or a company. We will draw important distinctions between defects, their root cause and their effects. We will explain why testing is necessary to find these defects, how testing promotes quality, and how testing fits in to the quality assurance.

As we go through this section, watch the syllabus terms Bug, Defect, Error, Failure, Fault, Mistake, Quality, Risk and Exhaustive Testing.

We may be asking what is testing. And we will look more closely at the definition of testing. Let’s adopt a simple everyday life usage example: When we are testing something we are checking wither it is ok. We will need to refine that when we define software testing later on. Let’s start by considering why testing is needed. Testing is necessary because we all make mistakes. Some of those mistakes are unimportant but some of them are expensive or dangerous. We need to check everything and anything we produce because things can always go wrong - Humans make mistakes all the time - it is what we do best!

Because we should assume our work contains mistakes, we all need to check our own work. However, some mistakes come from bad assumptions and blind spots, so we might make the same mistakes when we check our own work as we made then we did it so we may not notice the flaws in what we have done ideally, we should get someone else to check our work – another person is more likely to spot the flaws.

What is testing?
Testing:
It is a process to test an application with the intension of finding error is called Testing
OR
The process consisting of all life cycle activities both static and dynamic, concerned with planning, preparation and evaluation of software products and related work products to determine that they satisfy specified requirements, to demonstrate that they are fit for purpose and detect detects.

Error alias Names:

• Mistake
• Bug
• Defect
• Flaw
• False
• Nucleated
• Fault etc.............

Testing is divided in to the following list of two ways:

• Manual Testing
• Automation Testing

Manual Testing:
It is a process to test an application by giving manually inputs observing manually outputs.

Automation Testing:
It is a process to test an application with the intension of finding errors with the help of tools.

Causes of software defects:
Error (Mistake):
A human action that produces an incorrect result is known as Error or Mistake.

Defect (Bug/Fault):
A flaw in a component or system that can cause the component or system to fail to perform its required function, Example: An incorrect statement or date definition. A defect, if encountered during execution, makes cause a failure of the component or system.

Failure:
Deviation of the component or system from its expected delivery, service or result to disc tub.

Testing and Quality:
Testing help us to measure the quality of software inters of the number of defects found, the test run, and system covered by the test. We can do this for both the functional attributes of the software (For example: Printing a report correctly) and for the non functional requirements and characteristics (For example printing a report quickly enough). Testing can give confidence in the quality of the system if it finds few or no defects, provided we are happy that the testing is sufficiently rigorous. Of course, a poor test may uncover few defects and leave us with a false sense of security.

A well designed test will uncover defects if they are present and so, if such a test passes, we will rightly be more confident in the software and be able to assert that the overall level of risk of using the system has been reduced. When testing does find defects, the quality of the software system increases when those defects are fixed, provided the fixes are carried out properly.

What is Quality?
The degree to which a component, system or process meets specified requirements and/or user or customer needs and expectations.

Projects aim to deliver software to specification. For the project to deliver what the customer needs required a correct specification. Additionally, the delivered system must meet the specification. This is known as a Validation (This is the right specification?) and verification (Is the system correct to specification?). Of course, as well as wanting the right software system built correctly, the customer wants the project to be within badger and timescale.

Definition of Software Testing:
Software testing definition breaks into two parts; the definition has some key phrases to remember. The definition starts with a description of testing as a process and then lists some objectives of the test process. First, let’s look at testing as a process:

Process:
Testing is a process rather than a single activity - these are a series of activities involved

All Life Cycle Activates:
Testing as a process that takes place throughout the software development life cycle. We saw earlier that the later in the life cycle we find bugs, the more expensive they are it fix, If we can find and fix requirements defects at the requirements stage, that must mark commercial sense.

We will build the right software, correctly and at a lower cost overall. So, the thought process of designing tests early in the life cycle can help to prevent defects from begging introduced into code. We sometimes refer to this as verifying the test basis via the test design. The test basis includes documents such as the requirements and design specifications.

Both Static and Dynamic:
As well as tests where the software code is executed to demonstrate the results of running tests (often called dynamic testing) we can also test and find defects without executing code. This is called as static testing. This testing includes reviewing of documents (including source code) and static analysis. This is a useful and cost effective way of testing

Planning:
Activities take place before and after test execution. We need to manage the testing; for example, we plan what we want to do; we control the test activities; we report the testing progress and the status of the software under test; and we finalize or close testing when a phase completes.

Preparation:
We need it choose what testing we will do, by selecting test conditions and designing test cases.

Evaluation:
As well as execution the tests, we must check the results and evaluate the software under test and the exit criteria, which help us decide whether we have finished testing and whether the software product has passed the tests.

Software products and related work products:
We do not just test code. We test the requirements and design specifications, and we test related documents such as operations, user and training materials static and dynamic testing are both needed to cover the range of products we need to test.

The second part of the definition covers the some of the objectives for testing the reasons why we do it:

Determine that (software products) satisfy specified requirements – Some of the testing we do is focused on checking products against the specification for the product; for example we review the design to see if it meets requirements, and then we might execute the code to check that it meets the design. If the product meets its specification, we can provide that information to help stakeholders judge the quality of the product and decide whether it is ready for use.

Demonstrate that (software products) are fit for purpose – This is slightly different to the point above; after all the specified requirements might be wrong or incomplete. Fit for purpose looks at whether the software does enough to help the users to carry out their tasks; we look at whether the software dose what the user might reasonably expect.

For example, we might look at who might purchase or use the software, and check that it does do what they expect; this might lead us to add a review of the marketing material to our static tests, to check that expectations of the software are properly set. One way of judging the quality of product is by how fit it is for its purpose.

Detect defects – We most often think of software testing as means of detecting faults or defects that in operational use will cause failures. Finding the defects helps us understand the risks associated with putting the software into operational use, and fixing the defects improves the quality of the products. However, identifying defects has another benefit. With root cause analysis, they also help us improve the development processes and make fewer mistakes in future work.

This is a suitable definition of testing for any level of testing, from component testing through to acceptance testing, provided that we remember to take the varying objectives of these different levels of testing into account. (In Chapter2 we’ll cover the different test levels, their objectives, and how they fit into the software development life cycle.)

We can clearly see now why the common perception of testing (that it only consists of running tests, i.e. executing the software) is not complete. This is one of the testing activities, but not all of the testing process.

Software test and driving test compared
We can see that the software test is very like a driving test in many ways, although of course it is not a perfect analogy! The driving examiner becomes the software tester. The driver being examined becomes the system or software under test, and you’ll see as we go through this book that the same approach broadly holds.

Planning and preparation – Both the examiner and the tester need a plan of action and need to prepare for the test, which is not exhaustive, but is representative and allows risk-based decisions about the outcome.

Static and dynamic – Both dynamic (driving the car or executing the software) and static (questions to the driver of review of the software) tests are useful.

Evaluation – The examiner and the tester must make objective evaluations, log the test outcome and report factual observations about the test.

Determine that they satisfy specified requirements – The examiner and tester both check against requirements to carryout particular tasks successfully.

Demonstrate that they are fit for purpose – The examiner and the tester are not evaluating for perfection but for meeting sufficient of the attributes required to pass the test.

Detect Defects – The examiner and tester both look for and long faults.

When can we meet out test objectives?
We can use both dynamic testing and static testing as a means for achieving similar test objectives. Both provide information to improve both the system to be tested, and the development and testing processes. We mentioned above that testing can have different goals and objectives, which often include:

• Finding Defects;
• Gaining confidence in and providing information about the level of quality;
• Preventing Defects.

Test Objective:
A reason of purpose for designing and executing a test

Testing Principles:
We have introduced a number of testing principles and brief explanations. These are listed in below, for you to read cover to remind yourself about them. These principles have been suggested over the past 40 years and offer general guidelines common for all testing

S.No.	Testing Principle Names
1	Testing shows presents of defects
2	Exhaustive testing is impossible
3	Early Testing
4	Defect Clustering
5	Pesticide Paradox
6	Testing id context dependent
7	Absence – of- errors fallacy

Testing shows presents of defects:
Testing can show that defects are present, but cannot prove that there are not defects. Testing reduces the probability of undiscovered defects remaining in the software but, even if no defects are found it is not a proof of correctness.

Exhaustive testing is impossible:
Testing everything (all combinations of inputs and preconditions) is not feasible except for trivial cases. Instead of exhaustive testing, risk analysis and priorities should be used to focus testing efforts.

Early Testing:
Testing activities should start as early as possible in the software or system development life cycle and should be focused on defined objectives.

Defect Clustering:
A small number of modules contain most of the defects discovered during pre – release testing or are responsible for the most operational failures

Pesticide Paradox:
If the same test is repeated over and over again, eventually the same set of test cases will no longer find any new defects. To overcome this pesticide paradox, the test cases need to be regularly reviewed and revised, and new and different tests need to be written to exercise different parts of the software of system to potentially find more defects.

Testing is context dependent:
Testing is done differently in different contexts. For example, safety – critical software is tested differently from an e-commerce site.

Absence – of – error fallacy:
Finding and fixing defects does not help if the system built is unusable and does not dultill the users needs and expectations.

Fundamentals of Test Process:
Testing is a process rather than a single activity. This process starts from test planning then designing test cases, preparing for execution and evaluating status till the test closure. So, we can divide the activities within the fundamental test process into the following basic steps:

Planning and Control

Analysis and Design

Implementation and Execution

Evaluating exit criteria and Reporting

Test Closure activities

Planning and Control:
Test planning has following major tasks:

1. To determine the scope and risks and identify the objectives of testing.
2. To determine the test approach.
3. To implement the test policy and/or the test strategy. (Test strategy is an outline that describes the testing portion of the software development cycle. It is created to inform PM, testers and developers about some key issues of the testing process. This includes the testing objectives, method of testing, total time and resources required for the project and the testing environments.).
4. To determine the required test resources like people, test environments, PCs, etc.
5. To schedule test analysis and design tasks, test implementation, execution and evaluation.
6. To determine the Exit criteria we need to set criteria such as Coverage criteria.(Coverage criteria are the percentage of statements in the software that must be executed during testing. This will help us track whether we are completing test activities correctly. They will show us which tasks and checks we must complete for a particular level of testing before we can say that testing is finished.)

Test control has the following major tasks:

1. To measure and analyze the results of reviews and testing.
2. To monitor and document progress, test coverage and exit criteria.
3. To provide information on testing.
4. To initiate corrective actions.
5. To make decisions.

Analysis and Design:
Test analysis and Test Design has the following major tasks:

1. To review the test basis. (The test basis is the information we need in order to start the test analysis and create our own test cases. Basically it’s a documentation on which test cases are based, such as requirements, design specifications, product risk analysis, architecture and interfaces. We can use the test basis documents to understand what the system should do once built.)
2. To identify test conditions.
3. To design the tests.
4. To evaluate testability of the requirements and system.
5. To design the test environment set-up and identify and required infrastructure and tools.

Implementation and Execution:
During test implementation and execution, we take the test conditions into test cases and procedures and other testware such as scripts for automation, the test environment and any other test infrastructure. (Test cases is a set of conditions under which a tester will determine whether an application is working correctly or not.)

Testware is a term for all utilities that serve in combination for testing a software like scripts, the test environment and any other test infrastructure for later reuse.

Test implementation has the following major task:

1. To develop and prioritize our test cases by using techniques and create test data for those tests. (In order to test a software application you need to enter some data for testing most of the features. Any such specifically identified data which is used in tests is known as test data.)
   We also write some instructions for carrying out the tests which is known as test procedures.
   We may also need to automate some tests using test harness and automated tests scripts. (A test harness is a collection of software and test data for testing a program unit by running it under different conditions and monitoring its behavior and outputs.)
   
2. To create test suites from the test cases for efficient test execution.
   (Test suite is a collection of test cases that are used to test a software program to show that it has some specified set of behaviours. A test suite often contains detailed instructions and information for each collection of test cases on the system configuration to be used during testing. Test suites are used to group similar test cases together.)
   
3. To implement and verify the environment.

Test execution has the following major task:

1. To execute test suites and individual test cases following the test procedures.
2. To re-execute the tests that previously failed in order to confirm a fix. This is known as confirmation testing or re-testing.
3. To log the outcome of the test execution and record the identities and versions of the software under tests. The test log is used for the audit trial. (A test log is nothing but, what are the test cases that we executed, in what order we executed, who executed that test cases and what is the status of the test case (pass/fail). These descriptions are documented and called as test log.).
4. To Compare actual results with expected results.
5. Where there are differences between actual and expected results, it report discrepancies as Incidents.

Evaluating Exit criteria and Reporting:
Based on the risk assessment of the project we will set the criteria for each test level against which we will measure the “enough testing”. These criteria vary from project to project and are known as exit criteria.

Exit criteria come into picture, when:

• Maximum test cases are executed with certain pass percentage.
• Bug rate falls below certain level.
• When achieved the deadlines.

Evaluating exit criteria has the following major tasks:

1. To check the test logs against the exit criteria specified in test planning.
2. To assess if more test are needed or if the exit criteria specified should be changed.
3. To write a test summary report for stakeholders.

Test Closure activities:
Test closure activities are done when software is delivered. The testing can be closed for the other reasons also like:

• When all the information has been gathered which are needed for the testing.
• When a project is cancelled.
• When some target is achieved.
• When a maintenance release or update is done.

Test closure activities have the following major tasks:

1. To check which planned deliverables are actually delivered and to ensure that all incident reports have been resolved.
2. To finalize and archive testware such as scripts, test environments, etc. for later reuse.
3. To handover the testware to the maintenance organization. They will give support to the software.
4. To evaluate how the testing went and learn lessons for future releases and projects.
   

Sample Exam Questions:
1. A company recently purchased a commercial off the shelf application to automate their bill paying process. They now plan to run an acceptance test against the package prior to putting into production. Which of the following is their most likely reason for testing?
A. To build confidence in the application
B. To defect bugs in the application
C. To gather evidence for lawsuit
D. To train the users
Ans: A

2. What is the another name of Bug ?
Ans: Defect

3. How many Testing Principles ?
Ans: 7

Published date : 18 Mar 2015 12:12PM