Quality Assurance in Real-life Projects

In this blog post, we would like to address the topic of software QA methodology. On the one hand, we recognize there is a demand for clear hands-on QA testing instruction for those who are looking to dive deeper into the discipline. On the other hand, we feel that this topic needs to be brought into the light from more angles, covering both technical and business aspects, and showing various ways in which the two interact.

Hopefully, this can help businesses and agencies align their workflows more accurately, leading to faster and more efficient project development cycles.

The goals of the current guide include the following:

• Outline the scope of quality assurance (QA) vs quality control (QC)
• Explain the role of a QA tester within the project development life cycle
• Describe QA/QC testing methodology that helps ensure projects are delivered both effectively and on time.

The guidelines listed below are derived from multiple real-world website and web application development & testing projects. However, the same principles apply to other domains of software development, making them applicable to quality assurance specialists’ needs across the board.

QA / QC scope

What is quality control?

QC is defect detection and rejection. An integral part of QC tester’s job is identifying problems and s.c. “bug reporting” (see below). This process is what we refer to as defect detection.

What is quality assurance?

QA is a way of preventing mistakes and defects during development. A QA tester’s job is to take measures in order to help avoid potential problems. This process is what we refer to as defect prevention.

QA/QC focus within the project development lifecycle

Portrayed below are the main software development project phases with the description of QA/QC efforts necessary at each stage. Note that test strategy may alter depending on the given project specifics.

Overall, both defect detection and defect prevention take place at certain project phases and play an important role in a project development cycle.

DEFECT DETECTION

Defect detection, also known as quality control (QC) is the part of the software testing life cycle during which testing engineers go through the active project (whether live or in development), identifying and reporting defects. Here the same logic, as outlined in the QA methodology section above, applies: 

• Test the functional output first
• Then go after data representation, design, and cross-browser compatibility testing.

Types of bugs to search for

What is considered a bug? A bug in software development is either 

• an issue that prevents the project from being delivered, or
• an issue that prevents the client from running their business as intended.

Listed below are the main types of issues that need to be addressed through various types of testing. Aim to find, describe and report these as accurately as possible.

Content 

What to pay attention to 

• Deviations in textual or visual copy from the expected result.

What to look for

• UI elements 
• Notifications
• CTAs
• Other textual / design elements’ matching the intended design/page purpose
• Spelling errors
• Capitalization & punctuation errors
• Untranslated content 
• Demo & test content: both text and images
• Missing bits of page content: both text and images.

Visual 

What to pay attention to 

• Deviations of site visual layout from what was confirmed in the initial design designs or style guide.

What to look for

• Pixel-perfect revision of gaps between elements
• Pixel-perfect revision of responsive design elements
• Deviations in color, size, positioning, margins, the order of elements 
• Visual inconsistencies throughout the UI
• Incorrect or missing on-hover effects/state of selected elements
• Content that is misaligned or doesn’t fit the area in some other way
• Missing designs elements e.g. particular background or background colors
• Revision of all active elements such as links, buttons, input fields, error messages to ensure correct representation.

Cross-browser

What to pay attention to 

• The website being displayed differently in different browsers/browser versions, to the extent that is harmful to the business.

What to look for

• Unexpected visual artifacts that appear
• Functional elements being unavailable
• Performance issues, such as longer load time.

Cross-device

What to pay attention to 

• Device-specific behavior deviations. 

What to look for

• Elements displaying/behaving differently across different OS
• Elements displaying / behaving differently across phone / tablet / desktop.

Functional

What to pay attention to 

• Behavior that doesn’t work as intended, or doesn’t work at all.

Note: When reporting functional bugs, a software test engineer has to describe the problem and specific steps to recreate the faulty behavior.

What to look for

• Incorrect product filtering
• Incorrect element enumeration
• 404 pages
• Broken or missing product gallery images
• Product data inconsistency across different pages (e.g., incorrect prices)
• Broken Email sign-up functionality
• Broken Search functionality
• Broken Product order functionality
• Broken Checkout functionality
• Broken Login functionality
• Data inconsistency between back-end and front-end (e.g., VAT tax rates, special prices, etc.)

Performance

What to pay attention to 

• Slow page load / response.

Note: A general rule is no operation can take longer than 2-3 seconds to execute. Longer feedback time is almost surely a sign of an underlying technological problem.

What to look for

• First-page load (try incognito, remove all cache)
• Search results
• Product filtering
• Pagination
• Large images and banners (these cannot exceed 600-700 Kb)
• Listing page first loading
• Adding product to cart
• Placing an order
• Logging in and logging out of customer account

Structure of a bug report

Listed below are the main elements of a bug report with descriptions. Following these guidelines will help testing specialists consistently provide their teams with clear and uniform bug reports.

Bug report title

The title should describe the problem in a clear and concise manner. It has to make it evident whether the bug is functional, visual, cross-browser, cross-device, content, or performance-based.

Examples:

• Customer cannot log in
• Unsubscribe link does not work
• Missing translations in the FAQ.

Steps to recreate the issue

This section has to provide step-by-step instructions on how to arrive at the point where the issue can be experienced.

What to keep in mind:

• Make sure to indicate the relevant data inputs
• Make sure to list browser and device versions for cross-browser and cross-device issues
• Provide a clear indication of the incorrect behavior that needs to be recreated.

Expected results

This section of the bug report describes the resulting behavior after the issue is fixed. The goal of this is to provide developers with a clear understanding of what is required of them in order to complete the task.

The expected result can be described in a form of a link to design templates, specific data output, unblocking of a user journey, or particular feature availability.

Testing details

This section of a bug report has to contain the additional information necessary to understand the scope of the issue.

Environment

Details of the testing environment, describing device type, device model, browser details, OS. 

Screenshots 

Screenshots of a bug with arrows and other visual elements to highlight the problematic area.

Video (optional)

Adding a screen-capture video can be helpful to quickly demonstrate the specific sequence of steps necessary to recreate the issue.

Logs (optional)

These can and should be added if they really help understand the problem. Make sure to only add the console logs that have appeared in the course of the described malfunction.

Bug reporting workflow

Bug reporting vs failing a task

In the context of bug reporting workflow, QA/QC efforts can be divided into two broad categories.

Free-form testing

In this mode, a testing specialist has a module, website, or application to explore. Reporting issues here involves creating as many bug reports as necessary, covering every discovered issue.

Testing a specific task to validate someone else’s work

In this mode a testing specialist has a couple more choices to make:

1. If everything works as planned, the task can be marked as “Passed QA”
2. If the task has issues, there are two additional options
   1. The task is marked as “Failed QA” with additional information describing the reasons. This option is used when the key expected outcome is not met. Use it to make sure the critical errors don’t make it into the next project phase.
   2. The task is marked as “Passed QA”, with the addition of extra bug reports to document things that need to be further fixed. This option is used when the key business objective of the task is completed, yet there are some additional issues that were discovered during the testing process. Use it to make sure that the non-critical issues don’t block the entire project.

One bug report per issue

A general rule of thumb is that the tester creates a separate bug report per each discovered issue. This is advisable in order to paint a clear picture of the current state and progress of the project and to accurately estimate the efforts required to move it closer towards completion.

Naturally, there are exceptions to this. For instance, minor bugs related to the same issue can and should be grouped together to avoid redundant clutter. 

Example: You could easily combine under the same task issues related to border thickness, sign-up button font, and hover effect of a newsletter subscription module.

DEFECT PREVENTION

Defect prevention, also known as quality assurance (QA) is the part of the software testing life cycle that takes place either prior to or at the same time with the other development activities. With QA we generally want to take a step back in the project delivery process and start by figuring out how we can prevent errors – before any development work begins.

Task definition clarity

As a QA specialist, it is important that you:

• understand the expected result of the task, 
• have access to all relevant project materials (e.g., design templates)
• understand how a given feature is managed via the back-end, as well as its expected front-end representation.

A well-defined QA task is the one that provides sufficient data to a QA specialist regarding what they need to test and how.

A well-defined dev task is the one that clearly outlines the requirements and describes a specific approach to the solution.

Consider the following attributes of a well-defined task:

• Actionable task name
• Background is provided
• Clear task structure including task description, concrete to-do, and precise “how to test” section
• Short and precise, easy to understand
• Edge cases are covered 
• Scope of this task is limited and relations to other tasks are outlined

Creating “How to test” recommendations

What is a test case?

A test case is a specification of the inputs, execution conditions, testing procedures, and expected results. These enable a single test to be executed in order to achieve a particular software testing objective.

Or, in short, a test case is a unit of testing for a specific task.

What is a test scenario?

A test scenario is an array of multiple test cases.

In the scope of a given task’s description “How to test” is one of the most important sections, since it 

1. helps the developer to understand the task
2. serves as a quick reference for the entire QA process.

Consider the following attributes of a well-written “How-to-test” case:

• Test data is clearly defined
• All tasks, details, and descriptions are as specific as possible
• All recommendations are presented in a clear step-by-step manner
  • Remember that the test case is the first step for test AUTOMATION, thus it has to be clear not only to a developer but also to a robot
• It includes a clear description of an expected result.

How to prepare a UAT checklist

User acceptance testing (UAT) checklist a document that QAs assemble to guide the UAT test planning and execution. The goal is to compile the entire project scope into one document, made up of testable project features.

What does the UAT checklist have to include?

• Business-level customer-facing features, e.g.:
  • “User is able to sign up for the newsletter”
  • “There is a search input in the header that expands on click and shows suggestions after the user starts typing.”
  • “Home page layout matches the confirmed designs.”
• Devices and browsers that support them

How to use the UAT checklist?

Go through every feature on the list. If the feature behaves as intended, mark it as DONE. Otherwise – prepare a defect report, wait until fixed, then re-test. Repeat the process until all features are marked as DONE.

After every feature on the list is checked off, the document can be presented to the client to serve as proof of the scope quality. Beyond this point, it is up to the client whether or not they choose to have specific features re-tested. 

At this stage, the project can be considered delivered, and ready for Go-Live.

UAT checklist compilation timeframe

The checklist can be created as early as mid-project, and gradually populated as the work progresses. A good practice is to have the checklist structure finalized by at least 90% around 4-6 weeks before the project delivery. 

Every item on the list is expected to be marked as DONE by the time the project is presented to the client for revision.

Test strategy for missing content 

It is of utmost importance that website content is kept in check and up to date. Unfortunately, all too often this is overlooked when focusing heavily on feature and functionality development. 

We suggest setting up a missing content TO-DO list to help the QA tester identify and keep track of the project elements where the content is missing or needs an update. This document is also useful for communicating this information to the other team members.

Below are a few examples of what should and should NOT be included in the missing content TO-DO list.

• Dummy content: YES
• Hardcoded filter values: YES
• Identical information (placeholders, descriptions, etc.) in international site versions: YES
• The client is preparing to launch a Black Friday campaign, but the banners are for Back To School: YES
• Missing layered navigation on PLP that is not transferred from the design template: NO, it should be added to the bug list
• Missing filters in search overlay: NO, this too is a bug – a missing feature, not missing content

Purpose 

The purpose is to provide an up-to-date list of UI elements with the content currently missing. Below are some components to look out for when QA testing an eCommerce website:

• Slider images 
• Bestseller items
• Promotional texts
• FAQ
• About Us pages
• Any other areas temporarily filled with “Lorem Ipsum”, sample content, or untranslated content.

Remember that the document is called a TO-DO list for a reason. Thus, make sure to keep it up-to-date by regularly adding current issues and checking off items that have been fixed.

Motivation 

Still not convinced it is a really important part of a QA test strategy? Let’s put things in perspective. Many great projects are delayed because content creation is cast as an “easy” task, deprioritized, and put off “till later”. However, apart from being a time-consuming process, content creation often has to go through several layers of validation by various teams.

In general, it is not uncommon that content creation and validation take roughly the same time as the project development itself.

Looking to take your website to new heights with pro-level Quality assurance? Let us help you! Feel free to get in touch to see how we can assist your business growth.

Related articles:

How to Properly Set Up Layered Navigation for Your Store

Share on: