You’ve made changes to your large and complex system, and you’ve completed the lifecycle of your project, from specification through to development and testing. Your code is ready to be released to the customer for User Acceptance Testing (UAT). You’re asking yourself: “Why do I need to regression test?”.
How do you know that your changes have not caused a significant issue with the core functionality? That’s the importance of regression testing.
What is regression testing and what are the benefits?
Regression testing is the process of running functional and non-functional tests to ensure that changes to the codebase have not impacted the existing core functionality. Large enterprise systems often have many large and interconnecting modules, which means that the impact of a change is masked until a regression test is completed. Completing a regression test makes sure that the code won’t cause any issues with the core functionality of the system.
If you don’t have regression testing in your business, you’re at risk of having major bugs that slip through the net and can cause irrecoverable data issues, such as incorrect accounting records or malformed customer data. You may not have access to the source data to correct these, so it’s important that you can find any issues before release.
When you should regression test
A regression test should be carried out whenever the underlying source code of a system has been changed, which may include:
- New functionality added to an existing feature
- A new feature or piece of functionality
- The codebase is changed to fix bugs
- The system is optimised to improve performance
- Patches are made to fix high priority issues
- Configuration values are changed
- Which areas of the system would need regression testing? (We’ll discuss more on this later)
- What level of impact would an unidentified bug cause?
- How much time needs to be committed?
- What testing have you already completed?
You’ll need to measure what impact regression testing has, and how much resource you want to commit to it. For example, if a project includes a major change to accounting procedures, it’s crucial to regression test everything prior to release. If you are working in an agile team, you may choose to regression test at the end of a sprint rather than every unit of work because the overall risk is smaller.
Automated regression testing
Regression tests are a set of repeatable cases that can be ran by anyone to ensure core functionality continues to work. Automated regression testing is an obvious contender, and it is… for the right type of test case.
As mentioned in our previous blog “Manual vs Automated Testing: What’s the Difference?”, automated testing has its benefits, but also several flaws, including that the test case must be not rely on the cognitive abilities of a human. Products that work well with automated regression testing include APIs and supporting services that a user would not directly interact with.
A lot of regression test cases require testing the flow that an end user may take through the system, therefore relying on using a user interface. Many systems and applications are created using technologies that make automated UI testing impractical. For systems and applications in this group manual regression testing is the only option. Manual testing also allows for slight variation in the data that is input, increasing the chance of picking up issues when unexpected data is used. This is useful for regression test as it reduces the likelihood of software being developed to fit the happy path tests.
How to regression test
There are several methods you could use to run a regression test, depending on your risk level. full regression test. This is when you load all regression test cases for the entire system into a test plan and run each test to ensure every single test case passes. This can be useful to catch every error that could occur but can also be very time-consuming and takes a lot of manual effort.
Regression test all
As you embed regression testing into your processes and create a wider range of regression test cases, your approach will likely evolve. What started as a 30-minute manual test plan, when only the core areas of the system were tested, can quickly become 2-hours of testing if you were to run all the test cases. This is where regression test selection and prioritisation of test cases proves useful.
Regression test selection
This is when the most appropriate test cases are selected based on the changes. For example, if the client module of a travel system has been modified, you may select only the tests that include functionality in the client module. However, the risk of continually regression testing this way is that another module, for example bookings, may use the functionality in the client module, and this may not have been considered. This could have been broken by the change and remain unidentified.
Test case prioritisation
This is when test cases are chosen based on business impact, critical & frequently used functionalities. Selection of test cases based on priority will greatly reduce the regression test suite but potentially miss the finer details of some functionality.
How do I write a regression test plan?
Rome was not built in a day, and neither are regression test plans! A good regression test should be able to be run by anyone, including those unfamiliar with the business. Building up documentation can take time, if you don’t already have it. First, identify and document the areas of the system that would have the biggest impact if the functionality was broken. Here are three things to consider when writing regression test cases:
1. How should the functionality work, and how is this used as part of the test? (The test description)
Give the user an idea of what should be expected when using the functionality and how it fits into the wider process.
For example:
2. What are the steps that should be taken to test that area of the functionality? (The test steps)
Describe the exact inputs and steps that a user should take to successfully complete the test. The data may be valid or invalid depending on whether you want to test a process that should be blocked or allowed to continue. For many systems the inputs may be time specific, so we often title the inputs “Example Data” to show the user that they must use valid data when completing the test.
For example:
3. What do you expect to happen after the test steps have been followed? (The expected results)
Document the expect behaviour that the user will see after completing the test steps to validate the functionality. This may include more than one expected behaviour if multiple actions happen as a result of the test steps.
For example:
As you undertake new developments you will build in writing regression tests as part of the software development lifecycle. Often the content of developer tests can be modified to form a regression test for that area of functionality.
If you’re looking for a test management platform to manage your regression testing, Qucate uses dynamic test run templates to speed up the process and improve the quality of your software.
Head to our Qucate page for more information, or feel free to contact us.
