Software Product Quality

Without quality, product delivery is unsustainable. Poor quality is like an avalanche. At first, it’s small, unnoticeable. But then it accumulates, and we reach a point where we’ve lost control.

Without quality, product delivery is unsustainable.

Poor quality is like an avalanche. At first, it’s small, unnoticeable. But then it accumulates, and we reach a point where we’ve lost control. We can’t deliver sustainably anymore.

Many people (mistakenly) believe that high quality is expensive. But actually, poor quality is much more expensive! (Here, I'm referring to the total product lifetime cost, rather than the initial cost).

The cost of poor quality includes:

• Software that doesn’t work correctly, where the user doesn’t trust the software; not too good for our reputation

• Software that is too slow, the user is frustrated by waiting for operations to complete. Furthermore, the software has inefficient resource consumption, thus increasing our resource costs

• Reaching a point where the development team is spending most of their time on bug fixes (rather than new functionality), when any change is so expensive and so risky, reaching a point where development stagnates, and we lose in the market

We will start our journey into quality through a theoretical introduction regarding software quality. After all, we can’t achieve quality if we don’t know the characteristics of quality.

Software Quality

There are eight quality characteristics defined by ISO/IEC 25010:

• Functionality - Does the product function correctly? Does the functionality meet user needs?

• Performance - Does the product provide fast response times and use resources efficiently?

• Compatibility - Can the system exchange information and co-exist with other systems?

• Usability - Are users able to use the product efficiently and effectively? Are they satisfied with the product?

• Reliability - Is the system available, fault-tolerant, and recoverable?

• Security - Is the system protected through authentication and authorization?

• Maintainability - Is the system easy and safe to modify? Is it designed in a modular way? Is it testable?

• Portability - Can the system be transferred from one technology or environment to another?

The following is a more detailed representation of the quality characteristics and sub-characteristics:

Functional Suitability

Does the product function correctly? Does the functionality meet user needs?

Challenge: Functionality doesn’t meet user needs

When we create products disconnected from users, when we try to guess what the users need, and when we don’t get feedback, there’s a high probability that the product won’t meet user needs. Sure, the developers could implement everything per specifications, and the product functions correctly, but if it doesn’t meet user needs, it’s a failure.

The solution is to work in agile, which means working iteratively, releasing the product to users frequently, and adapting the product based on user feedback. This way, we have greater assurance that the product meets user needs. It’s essential to gain a deeper understanding of the use cases and the domain model. This is where Use Case Driven Development and Domain Driven Design can help.

Challenge: Incorrect functionality.

Let’s say that a team does not have any test automation or has poor test automation and only needs to rely on manual testing. In the beginning, everything’s fine; the manual tester can keep up with the development team and test everything after the team has developed it. But as the product accumulates functionality, more and more time is needed just for manual regression testing. And as the feature base increases, there are more and more regression bugs. Eventually, manual regression testing gets “cut“ to cover just “important“ scenarios, products are released with bugs, and the clients aren’t too happy.

The solution is to adopt test automation, to have a sustainable safety net against regression bugs. QA Engineers would then focus on new features and exploratory testing rather than manual regression testing. But how good is our test suite? Low Code Coverage means that we have a low-quality test suite, but high Code Coverage doesn’t necessarily mean we have a high-quality test suite - because it covers only whether code was executed, but not whether the behavior of the code has adequate assertions. This is where Mutation Testing is able to detect test suite quality issues which were not detected by Code Coverage.

Usability

Are users able to use the product efficiently and effectively? Are they satisfied with the product?

Challenge: Product is hard to learn and hard to use

Suppose the product wasn’t designed with the end user in mind but rather from the product owner's or developers' perspective. When we give the product to the user, the user experiences a knee-jerk reaction; they feel as if they’ve been displaced into foreign territory, they’re lost. They waste time trying to find where to click. Something simple takes too much to do. The user is frustrated! They want to return to the “old“ way of doing things - email, phone, MS Excel. Also, the solution isn’t to write a 100-page user manual either - if it takes too long to explain, if it’s not intuitive, then it’s too hard to use!

The solution is to understand the user persona and their goals, to be in “their shoes“. It’s not about designing what’s usable or pretty for us, but rather what is usable for the user! Design task-based applications; don’t just think in terms of CRUD. Think about the user, their goals, and what is of primary and secondary importance to them. Don’t engage the UX Designer just to convert the user story into visuals; they should be part of the product design. Design Sprints and Rapid Prototyping with Figma can be very useful to get feedback on the initial concept. During development, we need to get user feedback and observe (and collect statistics) regarding which functionalities are used and which ones aren’t and how long it takes to perform operations.

Maintainability

Is the system easy and safe to modify? Is it designed in a modular way? Is it testable?

Challenge: Expensive maintenance costs and change is risky

The biggest cost in software development is maintenance costs. The first 100m sprint is cheap and easy, but how about the marathon? If we don’t have adequate technical practices, we won’t see experience maintenance problems at the beginning, but as the product grows, that’s when change takes so much time; any change is painful and risky.

There are two root causes of maintenance costs. The first root cause is a lack of test automation - so introducing any change means the cost of manual regression testing and the cost of bug fixing regression bugs, and change is unsafe. The second root cause is poor internal code quality, for example, poor architecture and unreadable code, which means it takes more time for the developer to find the place to make the change, to read existing code, and make the change.

The solution is to ensure that we have a high-quality test automation suite and a regression safety net when we make changes. Furthermore, we need to have a high internal quality of our system - the system structure, the overall architecture, the modules, all the way down to classes and methods - to ensure that it is readable so that changes take less time.

Performance

Does the product provide fast response times and use resources efficiently?

Everything was fine at the beginning. But now, as we have more users and more data, the system has slowed down. Users are complaining that the system is too slow! Everyone’s frustrated.

What are the most common performance problems? What are their root causes? What can we do to make our system more performant? We will be digging deeper into this in a future article, so make sure to subscribe.

Reliability

Is the system available, fault tolerant, and recoverable?

The system is down again! In business, time is money - and this downtime is causing businesses to lose money. When one (peripheral) part of the system stopped working, now the core parts of the system stopped working too! What about recovery from failure? We may examine this topic in future articles.

Security

Is the system protected through authentication and authorization?

Security is “forgotten“ until it’s too late. It’s too late when a hacker steals all the data and causes reputation loss. Is our system protected? Are we controlling who can access our system and who can perform which actions on our system? We may examine this topic in future articles.

We want to hear from you

Which quality challenges have you seen in the products you’ve worked on? What are the biggest problems regarding quality? What are you seeing as challenges in the teams you’ve worked with? Which quality characteristics should we dig deeper into during our next articles?

Leave a comment below; I’m looking forward to hearing from you!