Originally posted on jan , 23 2009, Published again on Sept,18,2024
extracted completely from
http://testingsoftware.blogspot.com/2005/11/fundamentals-of-software-testing.html
Objectives of Testing
Finding of Errors - Primary Goal
Trying to prove that software does not work. Thus, indirectly verifying that
the software meets the requirements
Software Testing
Software testing is the process of testing the functionality and correctness of
software by running it. Software testing is usually performed for one of two
reasons:
(1) defect detection
(2) reliability or Process of executing a computer program and comparing the
actual behavior with the expected behavior
What is the goal of Software Testing?
* Demonstrate That Faults Are Not Present
* Find Errors
* Ensure That All The Functionality Is Implemented
* Ensure The Customer Will Be Able To Get His Work Done
Modes of Testing
* Static Static Analysis doesn¡¦t involve actual program execution. The code is
examined, it is tested without being executed Ex: - Reviews
* Dynamic In Dynamic, The code is executed. Ex:- Unit testing
Testing methods
* White box testing Use the control structure of the procedural design to
derive test cases.
* Black box testing Derive sets of input conditions that will fully exercise
the functional requirements for a program.
* Integration Assembling parts of a system
Verification and Validation
* Verification: Are we doing the job right? The set of activities that ensure
that software correctly implements a specific function. (i.e. The process of
determining whether or not products of a given phase of the software
development cycle fulfill the requirements established during previous phase).
Ex: - Technical reviews, quality & configuration audits, performance
monitoring, simulation, feasibility study, documentation review, database
review, algorithm analysis etc
* Validation: Are we doing the right job? The set of activities that ensure
that the software that has been built is traceable to customer requirements.(An
attempt to find errors by executing the program in a real environment ). Ex: -
Unit testing, system testing and installation testing etc
What's a 'test case'?
A test case is a document that describes an input, action, or event and an
expected response, to determine if a feature of an application is working
correctly. A test case should contain particulars such as test case identifier,
test case name, objective, test conditions/setup, input data requirements,
steps, and expected results
What is a software error ?
A mismatch between the program and its specification is an error in the program
if and only if the specifications exist and is correct.
Risk Driven Testing
What if there isn't enough time for thorough testing?
Use risk analysis to determine where testing should be focused. Since it's
rarely possible to test every possible aspect of an application, every possible
combination of events, every dependency, or everything that could go wrong,
risk analysis is appropriate for most software development projects. This
requires judgment skills, common sense, and experience.
Considerations can include:
- Which functionality is most important to the project's intended purpose?
- Which functionality is most visible to the user?
- Which aspects of the application are most important to the customer?
- Which parts of the code are most complex, and thus most subject to errors?
- What do the developers think are the highest-risk aspects of the application?
- What kinds of tests could easily cover multiple functionalities?
Whenever there's too much to do and not enough time to do it, we have to
prioritize so that at least the most important things get done. So
prioritization has received a lot of attention. The approach is called Risk
Driven Testing. Here's how you do it: Take the pieces of your system, whatever
you use - modules, functions, the section of the requirements - and rate each
piece on two variables, Impact and Likelihood.
Risk has two components: Impact and Likelihood
Impact
is what would happen if this piece somehow malfunctioned. Would it destroy the
customer database? Or would it just mean that the column headings in a report
didn't quite line up?
Likelihood
is an estimate of how probable it is that this piece would fail. Together,
Impact and Likelihood determine the Risk for the piece.
Test Planning
What is a test plan?
A software project test plan is a document that describes the objectives,
scope, approach, and focus of a software testing effort. The process of
preparing a test plan is a useful way to think through the efforts needed to
validate the acceptability of a software product.
Elements of test planning
* Establish objectives for each test phase
* Establish schedules for each test activity
* Determine the availability of tools, resources
* Establish the standards and procedures to be used for planning and conducting
the tests and reporting test results
* Set the criteria for test completion as well as for the success of each test
The Structured Approach to Testing
Test Planning
* Define what to test
* Identify Functions to be tested
* Test conditions
* Manual or Automated
* Prioritize to identify Most Important Tests
* Record Document References
Test Design
* Define how to test
* Identify Test Specifications
* Build detailed test scripts
* Quick Script generation
* Documents
Test Execution
* Define when to test
* Build test execution schedule
* Record test results
Bug Overview
What is a software error?
A mismatch between the program and its specification is an error in the Program
if and only if the specification exists and is correct.
Example: -
* The date on the report title is wrong
* The system hangs if more than 20 users try to commit at the same time
* The user interface is not standard across programs
Categories of Software errors
* User Interface errors
* Functionality errors
* Performance errors
* Output errors
* documentation errors
What Do You Do When You Find a Bug?
IF A BUG IS FOUND,
* alert the developers that a bug exists
* show them how to reproduce the bug
* ensure that if the developer fixes the bug it is fixed correctly and the fix
* didn't break anything else
* keep management apprised of the outstanding bugs and correction trends
Bug Writing Tips
Ideally you should be able to write bug report clearly enough for a developer
to reproduce and fix the problem, and another QA engineer to verify the fix
without them having to go back to you, the author, for more information.
To write a fully effective report you must :-
* Explain how to reproduce the problem
* Analyze the error so you can describe it in a minimum number of steps
* Write a report that is complete and easy to understand
Product Test Phase - Product Testing Cycle
Pre-Alpha
Pre-Alpha is the test period during which QA, Information Development and other
internal users make the product available for internal testing.
Alpha
Alpha is the test period during which the product is complete and usable in a
test environment but not necessarily bug-free. It is the final chance to get
verification from customers that the tradeoffs made in the final development
stage are coherent.
Entry to Alpha
* All features complete/testable (no urgent bugs or QA blockers)
* High bugs on primary platforms fixed/verified
* 50% of medium bugs on primary platforms fixed/verified
* All features tested on primary platforms
* Alpha sites ready for install
* Final product feature set Determined
Beta
Beta is the test period during which the product should be of "FCS
quality" (it is complete and usable in a production environment). The
purpose of the Beta ship and test period is to test the company's ability to
deliver and support the product (and not to test the product itself). Beta also
serves as a chance to get a final "vote of confidence" from a few
customers to help validate our own belief that the product is now ready for
volume shipment to all customers.
Entry to Beta
* At least 50% positive response from Alpha sites
* All customer bugs addressed via patches/drops in Alpha
* All bugs fixed/verified
* Bug fixes regression tested
* Bug fix rate exceeds find rate consistently for two weeks
* Beta sites ready for install
GM (Golden Master)
GM is the test period during which the product should require minimal work,
since everything was done prior to Beta. The only planned work should be to
revise part numbers and version numbers, prepare documentation for final
printing, and sanity testing of the final bits.
Entry to Golden Master
* Beta sites declare the product is ready to ship
* All customer bugs addressed via patches/drops in Beta
* All negative responses from sites tracked and evaluated
* Support declares the product is supportable/ready to ship
* Bug find rate is lower than fix rate and steadily decreasing
FCS (First Customer Ship)
FCS is the period that signifies entry into the final phase of a project. At
this point, the product is considered wholly complete and ready for purchase
and usage by the customers.
Entry to FCS
* Product tested for two weeks with no new urgent bugs
* Product team declares the product is ready to ship
================================