Unit III: Software Testing Process
When Testing should occur? Requirement Phase, Design Phase, Program (Build) Phase, Test
Phase, Installation Phase, Maintenance Phase, Testing activities, Test Plan, Test
Development, Test Execution, Results, Defect tracking, Reports
What is Testing?
Testing is the process of evaluating a system or its component(s) with the intent to find whether it satisfies the specified requirements or not. In simple words, testing is executing a system in order to identify any gaps, errors, or missing requirements in contrary to the actual requirements.
According to ANSI/IEEE 1059 standard, Testing can be defined as - A process of analyzing a software item to detect the differences between existing and required conditions (that is defects/errors/bugs) and to evaluate the features of the software item.
Who does Testing?
It depends on the process and the associated stakeholders of the project(s). In the IT industry, large companies have a team with responsibilities to evaluate the developed software in context of the given requirements. Moreover, developers also conduct testing which is called Unit Testing. In most cases, the following professionals are involved in testing a system within their respective capacities −
Software Tester
Software Developer
Project Lead/Manager
End User
Different companies have different designations for people who test the software on the basis of their experience and knowledge such as Software Tester, Software Quality Assurance Engineer, QA Analyst, etc.
It is not possible to test the software at any time during its cycle. The next two sections state when testing should be started and when to end it during the SDLC.
When to Start Testing?
An early start to testing reduces the cost and time to rework and produce error-free software that is delivered to the client. However in Software Development Life Cycle (SDLC), testing can be started from the Requirements Gathering phase and continued till the deployment of the software.
It also depends on the development model that is being used. For example, in the Waterfall model, formal testing is conducted in the testing phase; but in the incremental model, testing is performed at the end of every increment/iteration and the whole application is tested at the end.
Testing is done in different forms at every phase of SDLC −
During the requirement gathering phase, the analysis and verification of requirements are also considered as testing.
Reviewing the design in the design phase with the intent to improve the design is also considered as testing.
Testing performed by a developer on completion of the code is also categorized as testing.
When to Stop Testing?
It is difficult to determine when to stop testing, as testing is a never-ending process and no one can claim that a software is 100% tested. The following aspects are to be considered for stopping the testing process −
Testing Deadlines
Completion of test case execution
Completion of functional and code coverage to a certain point
Bug rate falls below a certain level and no high-priority bugs are identified
Management decision
Verification & Validation
These two terms are very confusing for most people, who use them interchangeably. The following table highlights the differences between verification and validation.
The software development process is normally long and tedious. But project managers and system analysts can leverage software development life cycles to outline, design, develop, test, and eventually deploy information systems or software products with greater regularity, efficiency, and overall quality.
What is System Development Life Cycle?
A system development life cycle or SDLC is essentially a project management model. It defines different stages that are necessary to bring a project from its initial idea or conception all the way to deployment and later maintenance.
System Development Life Cycle US Guide
In this guide, we’ll break down everything you need to know about the system development life cycle, including all of its stages. We’ll also go over the roles of system analysts and the benefits your project might see by adopting SDLC.
7 Stages of the System Development Life Cycle
There are seven primary stages of the modern system development life cycle. Here’s a brief breakdown:
Planning Stage
Feasibility or Requirements of Analysis Stage
Design and Prototyping Stage
Software Development Stage
Software Testing Stage
Implementation and Integration
Operations and Maintenance Stage
Now let’s take a closer look at each stage individually.
Planning Stage
Before we even begin with the planning stage, the best tip we can give you is to take time and acquire proper understanding of app development life cycle.
The planning stage (also called the feasibility stage) is exactly what it sounds like: the phase in which developers will plan for the upcoming project.
It helps to define the problem and scope of any existing systems, as well as determine the objectives for their new systems.
By developing an effective outline for the upcoming development cycle, they'll theoretically catch problems before they affect development.
And help to secure the funding and resources they need to make their plan happen.
Perhaps most importantly, the planning stage sets the project schedule, which can be of key importance if development is for a commercial product that must be sent to market by a certain time.
Analysis Stage
The analysis stage includes gathering all the specific details required for a new system as well as determining the first ideas for prototypes.
Developers may:
Define any prototype system requirements
Evaluate alternatives to existing prototypes
Perform research and analysis to determine the needs of end-users
Furthermore, developers will often create a software requirement specification or SRS document.
This includes all the specifications for software, hardware, and network requirements for the system they plan to build. This will prevent them from overdrawing funding or resources when working at the same place as other development teams.
Design Stage
The design stage is a necessary precursor to the main developer stage.
Developers will first outline the details for the overall application, alongside specific aspects, such as its:
User interfaces
System interfaces
Network and network requirements
Databases
They’ll typically turn the SRS document they created into a more logical structure that can later be implemented in a programming language. Operation, training, and maintenance plans will all be drawn up so that developers know what they need to do throughout every stage of the cycle moving forward.
Once complete, development managers will prepare a design document to be referenced throughout the next phases of the SDLC.
Development Stage
The development stage is the part where developers actually write code and build the application according to the earlier design documents and outlined specifications.
This is where Static Application Security Testing or SAST tools come into play.
Product program code is built per the design document specifications. In theory, all of the prior planning and outlined should make the actual development phase relatively straightforward.
Developers will follow any coding guidelines as defined by the organization and utilize different tools such as compilers, debuggers, and interpreters.
Programming languages can include staples such as C++, PHP, and more. Developers will choose the right programming code to use based on the project specifications and requirements.
Testing Stage
Building software is not the end.
Now it must be tested to make sure that there aren’t any bugs and that the end-user experience will not negatively be affected at any point.
During the testing stage, developers will go over their software with a fine-tooth comb, noting any bugs or defects that need to be tracked, fixed, and later retested.
t’s important that the software overall ends up meeting the quality standards that were previously defined in the SRS document.
Depending on the skill of the developers, the complexity of the software, and the requirements for the end-user, testing can either be an extremely short phase or take a very long time. Take a look at our top 10 best practices for software testing projects for more information.
Implementation and Integration Stage
After testing, the overall design for the software will come together. Different modules or designs will be integrated into the primary source code through developer efforts, usually by leveraging training environments to detect further errors or defects.
The information system will be integrated into its environment and eventually installed. After passing this stage, the software is theoretically ready for market and may be provided to any end-users.
Maintenance Stage
The SDLC doesn’t end when software reaches the market. Developers must now move into a maintenance mode and begin practicing any activities required to handle issues reported by end-users.
Furthermore, developers are responsible for implementing any changes that the software might need after deployment.
This can include handling residual bugs that were not able to be patched before launch or resolving new issues that crop up due to user reports. Larger systems may require longer maintenance stages compared to smaller systems.
Role of System Analyst
An SDLC’s system analyst is, in some ways, an overseer for the entire system. They should be totally aware of the system and all its moving parts and can help guide the project by giving appropriate directions.
The system analyst should be:
An expert in any technical skills required for the project
A good communicator to help command his or her team to success
A good planner so that development tasks can be carried out on time at each phase of the development cycle
Thus, systems analysts should have an even mix of interpersonal, technical, management, and analytical skills altogether. They’re versatile professionals that can make or break an SDLC.
Their responsibilities are quite diverse and important for the eventual success of a given project. Systems analysts will often be expected to:
️Gather facts and information
Make command decisions about which bugs to prioritize or what features to cut
Suggest alternative solutions
Draw specifications that can be easily understood by both users and programmers
Implement logical systems while keeping modularity for later integration
Be able to evaluate and modify the resulting system as is required by project goals
Help to plan out the requirements and goals of the project by defining and understanding user requirements
What is a Test Case?
A Test Case is a set of actions executed to verify a particular feature or functionality of your software application. A Test Case contains test steps, test data, precondition, postcondition developed for specific test scenario to verify any requirement. The test case includes specific variables or conditions, using which a testing engineer can compare expected and actual results to determine whether a software product is functioning as per the requirements of the customer.
In this Test Case tutorial, you will learn:
Test Scenario Vs Test Case
Test scenarios are rather vague and cover a wide range of possibilities. Testing is all about being very specific.
For a Test Scenario: Check Login Functionality there many possible test cases are:
Test Case 1: Check results on entering valid User Id & Password
Test Case 2: Check results on entering Invalid User ID & Password
Test Case 3: Check response when a User ID is Empty & Login Button is pressed, and many more
This is nothing but a Test Case.
Test Case Video
Click here if the video is not accessible
The format of Standard Test Cases
Below is a format of a standard login Test cases example.
This entire table may be created in Word, Excel or any other Test management tool. That’s all to Test Case Design
How to Write Test Cases in Manual Testing
Let’s create a Test Case for the scenario: Check Login Functionality
Step 1) A simple test case to explain the scenario would be
Step 2) Test the Data.
In order to execute the test case, you would need Test Data. Adding it below
Identifying test data can be time-consuming and may sometimes require creating test data afresh. The reason it needs to be documented.
Step 3) Perform actions.
In order to execute a test case, a tester needs to perform a specific set of actions on the AUT. This is documented as below:
Many times the Test Steps are not simple as above, hence they need documentation. Also, the author of the test case may leave the organization or go on a vacation or is sick and off duty or is very busy with other critical tasks. A recently hire may be asked to execute the test case. Documented steps will help him and also facilitate reviews by other stakeholders.
Step 4) Check behavior of the AUT.
The goal of test cases in software testing is to check behavior of the AUT for an expected result. This needs to be documented as below
During test execution time, the tester will check expected results against actual results and assign a pass or fail status
Step 5) That apart your test case -may have a field like,
Pre – Condition which specifies things that must be in place before the test can run. For our test case, a pre-condition would be to have a browser installed to have access to the site under test. A test case may also include Post – Conditions which specifies anything that applies after the test case completes. For our test case, a postcondition would be time & date of login is stored in the database
Best Practice for writing good Test Case.
Test Case Best Practice
1. Test Cases need to be simple and transparent:
Create test cases that are as simple as possible. They must be clear and concise as the author of the test case may not execute them.
Use assertive language like go to the home page, enter data, click on this and so on. This makes the understanding the test steps easy and tests execution faster.
2. Create Test Case with End User in Mind
The ultimate goal of any software project is to create test cases that meet customer requirements and is easy to use and operate. A tester must create test cases keeping in mind the end user perspective
3. Avoid test case repetition.
Do not repeat test cases. If a test case is needed for executing some other test case, call the test case by its test case id in the pre-condition column
4. Do not Assume
Do not assume functionality and features of your software application while preparing test case. Stick to the Specification Documents.
5. Ensure 100% Coverage
Make sure you write test cases to check all software requirements mentioned in the specification document. Use Traceability Matrix to ensure no functions/conditions is left untested.
6. Test Cases must be identifiable.
Name the test case id such that they are identified easily while tracking defects or identifying a software requirement at a later stage.
7. Implement Testing Techniques
It’s not possible to check every possible condition in your software application. Software Testing techniques help you select a few test cases with the maximum possibility of finding a defect.
Boundary Value Analysis (BVA): As the name suggests it’s the technique that defines the testing of boundaries for a specified range of values.
Equivalence Partition (EP): This technique partitions the range into equal parts/groups that tend to have the same behavior.
State Transition Technique: This method is used when software behavior changes from one state to another following particular action.
Error Guessing Technique: This is guessing/anticipating the error that may arise while doing manual testing. This is not a formal method and takes advantages of a tester’s experience with the application
8. Self-cleaning
The test case you create must return the Test Environment to the pre-test state and should not render the test environment unusable. This is especially true for configuration testing.
9. Repeatable and self-standing
The test case should generate the same results every time no matter who tests it
10. Peer Review.
After creating test cases, get them reviewed by your colleagues. Your peers can uncover defects in your test case design, which you may easily miss.
While drafting a test case to include the following information
The description of what requirement is being tested
The explanation of how the system will be tested
The test setup like a version of an application under test, software, data files, operating system, hardware, security access, physical or logical date, time of day, prerequisites such as other tests and any other setup information pertinent to the requirements being tested
Inputs and outputs or actions and expected results
Any proofs or attachments
Use active case language
Test Case should not be more than 15 steps
An automated test script is commented with inputs, purpose and expected results
The setup offers an alternative to pre-requisite tests
With other tests, it should be an incorrect business scenario order
Test Case Management Tools
Test management tools are the automation tools that help to manage and maintain the Test Cases. Main Features of a test case management tool are
For documenting Test Cases: With tools, you can expedite Test Case creation with use of templates
Execute the Test Case and Record the results: Test Case can be executed through the tools and results obtained can be easily recorded.
Automate the Defect Tracking: Failed tests are automatically linked to the bug tracker, which in turn can be assigned to the developers and can be tracked by email notifications.
Traceability: Requirements, Test cases, Execution of Test cases are all interlinked through the tools, and each case can be traced to each other to check test coverage.
Protecting Test Cases: Test cases should be reusable and should be protected from being lost or corrupted due to poor version control. Test Case Management Tools offer features like
Naming and numbering conventions
Versioning
Read-only storage
Controlled access
Off-site backup
Popular Test Management tools are: Quality Center and JIRA
Defect Tracking
July 15, 2020
"In engineering, defect tracking is the process of tracking the logged defects in a product from beginning to closure (by inspection, testing, or recording feedback from customers), and making new versions of the product that fix the defects"
In software engineering, the engineers have to face numerous of defects in a build. The more complex is the software the more complex are the defects to detect. The difficulty the engineers face is to manage, evaluate and prioritize the defects. A proper defect tracking system has to be set up so that these large and difficult defects can be analyzed and tracked to meet the required solution. A defect tracking system makes the work easier for the managers.
Defect tracking can be tedious, yet if we compare the defects then it can also help the testers so that the work done is efficient and improvised. A defect is not a bug because bugs are actually outside of your control whereas, defects have a definite solution and they can be resolved. A defect tracking system tools are different from bug tracking tools. People often confuse these two terms because of their similar functions and purposes.
Objectives of Defect Tracking-
A defect tracker has revolutionized the software industry as it provides a faster method of correcting the defects-
A defect tracker keeps a track of the all defects. So that the management may not miss any defect from correction.
Due to these tracked defects, the most correct methods are adopted and preventive measures are taken to avoid further defects.
It saves the time of the engineers thus helping in the fast delivery of the work. Also it enhances the efficient work delivered.
Why Defect Tracking is Necessary?
A defect tracker keeps record of all the defects so that none of the errors is missed for evaluation process.
Monitors the bugs that had already been resolved, there by saves time.
To ensures that right defects are being worked on.
Defect tracking Issues
Defect respiratory uses Ad Hoc Spreadsheet-
This is the spreadsheet method where the project participants access the file in which they update or read the changes done in the program. Awareness will be helpful in the reduction of the defects. It is useful when the number of defects is relatively small. Where it comes to a large number of defects, this method is non-functional.Insufficient issue description-
If the analyst cannot understand the nature of the defect then it becomes difficult to solve out the problem. In such cases, it is advisable to contact the defect originator. The missing information of the defects will make the current work stop. Until the defect is rectified, no further work can be carried out.Incorrect old defects-
Inappropriate analysis of the old defects may introduce new defects. It can be problematic, if the old defects are not resolved. This may cause difficulty in making the correction in the current/new defects.Poor prioritization of defects-
Sometimes, the poor prioritization of the defects may cause havoc. A serious problem may be overlooked and a problem which can be rectified at a later stage can be solved out earlier, due to which more new defects may be raised.
est Report
Test Report is a document which contains a summary of all test activities and final test results of a testing project. Test report is an assessment of how well the Testing is performed. Based on the test report, stakeholders can evaluate the quality of the tested product and make a decision on the software release.
For example, if the test report informs that there are many defects remaining in the product, stakeholders can delay the release until all the defects are fixed.
Test Report Example
Why Test Report?
The following scenario will show you why we do need the Test Report
Do you know the root cause of this problem? Why does the website still has defects even when your Team has already tested it?
The problem is you ignored the reporting & evaluation phase in Test Management. The boss has no information to evaluate the quality of this website. They just trusted what you said and released the website without knowing its testing performance.
The typical benefits of a test report include:
To answer this, you must know –
What does a test report contain?
Project Information
All information of the project such as the project name, product name, and version should be described in the test report. For example, the information of Guru99Bank project will be as follows
Test Objective
As mentioned in Test Planning tutorial, Test Report should include the objective of each round of testing, such as Unit Test, Performance Test, System Test …Etc.
Test Summary
This section includes the summary of testing activity in general. Information detailed here includes
The number of test cases executed
The numbers of test cases pass
The numbers of test cases fail
Pass percentage
Fail percentage
Comments
This information should be displayed visually by using color indicator, graph, and highlighted table.
Defect
One of the most important information in Test Report is defect. The report should contain following information
Total number of bugs
Status of bugs (open, closed, responding)
Number of bugs open, resolved, closed
Breakdown by severity and priority
No comments:
Post a Comment