There are many different types of essential tests that a Quality Engineer (QE) needs to consider when testing software. I’ve chosen a few to talk about in this blog, because they account for the most common (and avoidable) issues that I’ve seen arise throughout my career in software development. Not to mention, that I’ve been guilty of a few of these myself!
Boundary Analysis Testing
Boundary Analysis is the process where the extreme ends of a valid set of values are tested. To demonstrate, let’s consider a real-life example:
An online store is selling widgets. These widgets retail at £1.99 each, however, the customer should receive a scaling discount for bulk-buy orders:
As a QE, you will decide on a range of values to test. This will make sure that the discount is applied and the total price is correct.
As well as testing some random values, you should test the values that lie on the boundary of each range. This will make sure that the developer did not make a mistake, such as using ‘less than’ instead of ‘less than or equal to’.
Therefore, your test plan should include the following tests:
Capacity Testing is where you will test that each input value to an application or process cannot exceed it’s logical or physical capacity.
That’s a bit of a mouthful, so let’s break it down.
Capacity Testing - User Interface
Are the inputs within a user interface (UI) properly validated and constrained, so that the application doesn’t break when the user does something unexpected? For example, if a user is asked to input their age, does the application fall-over when they enter an age of 1000? If they are asked to enter their name, does the input field limit the number of characters they can enter, and is that limit reasonable?
Let’s consider our previous example of an online store. If a user is asked to enter the quantity that they wish to order, then here are some examples of capacity tests that a QE might perform:
Data Type Capacity Testing
Data storage, whether physical or in-memory, is fundamental to any software application. Each piece of data that is stored will have an associated data type. Common data types, such as numeric and alphanumeric, are often defined early in the software development lifecycle (SDLC). If you change these data types once the application is in production then it can be extremely problematic! You need to consider whether the data types that have been used are sensible.
As an example, imagine that our online shop orders are stored in a database table named ‘Orders’. It might look like this:
The ID column in our example has an Integer data type, which commonly has a maximum capacity of 2.1 billion. As a QE, you should ask yourself:
“Is an Integer a sensible choice?”
“Has the development team considered what happens if our widget store becomes mega-successful, and grows into the next Amazon?”
“What if we merge with a competitor and need to bulk-import hundreds of millions of previous orders?”
Handling dates correctly can be a significant challenge in software development, and a good test plan will often consist of multiple tests relating specifically to dates.
Here a few common problem areas that you should look out for.
Dates can be inclusive or exclusive
When testing date-based calculations, determining the correct result often requires you to understand the domain in which they will be used.
As an example, let’s consider a common scenario from the travel industry:
A customer wants to book a 3-night holiday in a cottage that starts on 1st August. The application that processes the request must check that the cottage is available for the entire holiday i.e. there is not an existing booking at that time
To determine the range of dates to check, the software developer could be tempted to use the formula: Start Date + Nights. This would give us a date range to check of 1st August – 4th August.
Is this correct? Well, it depends whether the application should allow back to back bookings or not.
In this case, a customer renting a cottage for 3 nights from 1st August, would spend their last night on 3rd August, and leave the cottage on 4th August. If the 4th August is booked by another customer, should this holiday be allowed? It’s a business decision.
The software developer who coded this behaviour may have made a set of assumptions. You, as a QE, may make different assumptions. The expected outcome should be tested and recorded in your test plan.
Dates can have a time element!
If you fail to consider the time element of a date then it’s likely to cause edge-case bugs in an application, especially when comparing one date to another.
For example, consider the requirement: “The application should list all orders, created on or prior to a user-specified date”
If the ‘Order Created Date’ does not contain a time portion, this formula would produce the correct result:
Order Created Date <= [USER-SPECIFIED DATE]
However, to account for the possibility of a time element in the Order Created Date field, we should instead use:
Order Created Date Time < ([USER-SPECIFIED DATE] +1)
Note the subtle different in the comparison operator, as well as the value. This is to account for the time element in the order date, and will ensure that any orders placed on the given date are picked up.
Formats and Time zones vary
When you’re testing an application that stores or processes dates, you should ask yourself:
- Will the application be used across multiple time zones?
- If so, are we storing dates in a format that allows us to account for this?
- Are we displaying dates in the local format?
- Do we need to consider factors such as daylight-saving Time zones?
Ambiguous Business Requirements
Let’s consider this business requirement for the online widget shop:
“If the customer has placed an order in the previous year, they should automatically receive a 10% discount on their purchase”
Is the business requirement clear? Are these risks and assumptions about what behaviour is expected? Let’s look more into it:
- What is the definition of “placed an order”? Does this mean the customer has paid for an order during the qualifying period? What if they then cancelled that order?
- What does “previous year” mean? Should we look back 365-days, or 12-months? What if it was a leap year? Or perhaps it means “any time in the previous calendar year” i.e. 1st Jan 2021 – 31 Dec 2021
- Is the discount off the entire order, including the delivery fee?
- Is there a limit on the number of discounts a customer can receive?
As the QE, you should challenge an ambiguous requirement such as this, and then document the intended behaviour in the test plan. Then it can be tested and retested on multiple test cycles. Ideally, the business requirement would have been written as follows:
“If the customer has at least one previous order, which was paid for during the previous calendar year and not refunded, they should automatically receive a 10% discount on their order items at checkout. The discount can be applied to multiple orders, however, only one discount can be applied per order, and this should always be the discount that gives the customer the greatest reduction on the price of their order.”
At Koderly, we use Qucate to plan, coordinate and execute our software testing process. Qucate allows you to create fully audited test plans for compliance, visibility, and transparency, whilst improving quality and test coverage. If you’re looking for a test management platform, head to our Qucate page for more information, and start your 30-day free trial today!
Thanks for reading!