How to put quality back into QA

However, fixing bugs or implementing changes at such a late stage introduces risks associated with re-development and may mean you can’t release to production on time.  In a world of Continuous Deployment, an organization’s ability to deliver quickly is now essential to its success, particularly in ecommerce.

What if there was another approach with a focus on quality before anyone even starts coding?  An approach that promotes a cross-functional team and aims at preventing bugs rather than fixing them?

In this article, we will look at some ideas for “baking-in” quality to your app, to reduce risk and re-work, increase team understanding, and build confidence in deployment.

.

Quality is a team responsibility

A big hurdle for many teams is assuming quality is the responsibility of the QA team.  Even in agile projects, QA is often treated as a siloed activity after the development is done.

Eliminating barriers between business, developer and QA is fundamental to the success of a quality first approach.  In an ideal scenario, QA would not even be considered a separate entity, but rather an intrinsic, overarching aspect of the wider development process.

Consider for example the number of people invited to your typical agile ceremonies (daily, refinement, planning, etc).  Including too many people could put off some people from speaking, risking misunderstanding and knowledge not being shared.  Smaller meetings, and teams in general, encourage participation from all team members.

Similarly, frequent status checks of in-progress development can be useful for managing the expectations of stakeholders.  For example, if a feature turns out to be more complex than originally estimated, a quick shoulder check can help identify blockers and react to unexpected changes, even if that does mean excluding it from the next production release.

Now that you have open communication, what steps can you take to encourage collaboration and work towards team responsibility to quality?

Define quality early

Defining your quality strategy early can help the delivery team identify risks, increase their understanding of how to test the app, and provide opportunities for assessing success objectively.

However, your quality definition doesn’t have to just be about the functional aspects of the app, such as user friendliness or device compatibility.  Ecommerce platforms using a DevOps model could, for example, include the DORA metrics in their quality definition, to help achieve a performant system.

There are several perspectives to quality, and it can be daunting trying to understand and define your quality aspects.

However, a good starting point is RiskStorming.  There are already many excellent resources available for learning about this game, so we won’t cover them again here.  However, in the context of defining quality, it is a useful tool that starts by reaching an agreement on the most important quality aspects of a system. This way the team has a common understanding, and agreed expectations on how they and their customers should perceive “quality”.

Given the subjectivity of quality, ideally the session should involve people of different disciplines within your project, but at least representatives from business and development.  This way both business and technical aspects are equally considered.  Nevertheless, as your customer is ultimately the most capable judge of quality, they should be involved too.

Test your quality expectations

Now that you have an agreed definition, consider what risks or barriers may prevent you and your customers from perceiving that quality.  Once identified, you can then think about which testing activities are appropriate to mitigate them.

An ecommerce platform may perhaps consider speed as an important measure of quality, where daily price or stock updates are essential to providing an accurate user experience.  In that context, look at whatever process is responsible for updating that data, and test for bottlenecks and risks to invalid or inaccurate data.

Similarly, if your platform is predominantly used on mobile devices, connectivity issues may impact performance for the end user.  Delivery teams should leverage real usage data to understand the limitations of their user devices and, for example, test the system using a throttled network connection, to simulate as best as possible the user experience.

Regardless of whether you use exploratory testing charters or traditional test cases, documenting how you will assess your app as a team is both a good exercise in collaboration, and a way to increase common understanding of a feature.  It also serves as an agreement to the scope of the testing, to better understand when you can consider the feature ready to ship.

Continual quality assessment

At any time during the development of system feature, ask yourself “what is the current quality status?”.  If you’re not able to answer confidently, you’re missing a feedback loop that could lead to issues later in development, when it may be too late to react.

Enabling fast feedback loops means implementing mechanisms that allow the delivery team to test at any stage of development.  Testing in this context however does not necessarily mean having a working version of the system, rather exposing whatever information is available for critique.  Even involving the delivery team as early on as discovery meetings can be beneficial, where their system knowledge can be used to reveal technical dependencies early on and help determine the feasibility of a feature.

The Holistic Testing Model is an excellent example of how we can assess quality during development and after release.

Later in development, tools that allow hands-on testing to start before pushing code to a live environment can simplify bug fixing.  For example, I have used AWS Amplify as a platform for accessing a useable preview of a webapp whilst any changes are still in code review.  This means any frontend bugs can be found and fixed in the code review itself and removes any re-development risk had that bug been found after being deployed.  Similarly, asking a backend developer to pass me a development compilation of a webservice to run locally enables me to start testing an endpoint’s success or failure responses, and report issues when they’re easier to fix.

Including the customer in that feedback loop is also important, and recently we started sending ours a survey after the sprint demo. Whilst they were already involved in defining requirements, receiving their comments about the demo itself allows us to gauge their understanding of what was presented, and to help us improve our presentation skills.

In any case, trying to measure quality in the traditional sense of “goodness” could prove to be a wasted effort.  Instead, when defining your quality aspects, think about what evidence is needed to help you discuss and assess its status at any given time.

A recipe for quality

Quality is hard to define, and measuring it is even more so.  Nevertheless, any software development project looking to take quality seriously should start considering what that Q in QA really means for them.

Set yourself up for a successful delivery by:

1. Shifting your mindset towards team responsibility to quality
2. Collaborating to define and understand your quality expectations
3. Testing and assessing those expectations throughout the development lifecycle

It’s a simple recipe, but one with complex and nuanced ingredients.  Hopefully this article has given you some ideas on how to get started.  Happy baking!