Функциональные баги

Functional bugs are related to the functionality of a piece of software. Examples: A button doesn’t submit the form, the search doesn’t react to the user input, the app crashes. Every time you perform an action and the website/app doesn’t respond as you expected, it might be a functional issue. Our limited information about our customers' products and lack of knowledge of their implementations make it difficult to determine whether an observed behavior is intentional or actually a bug. Making educated guesses based on experience and analyzing the product behavior by testing different scenarios can help you to come up with answer.

How to determine if app behavior is a functional bug:

Try to figure out if a feature is designed a particular way or if it is actually broken. Test it by itself and in combination with other features to spot potential differences.
Think about what the customer’s intentions might have been and consider that the product might just work how it has been implemented.
Find evidence that something is not working as it should. Support your claim.
Example: A webshop functionality works differently than in other webshops you know. That doesn’t mean the functionality is broken. The client can implement their product however they want.
Example: You claim a form field is not validated and it’s a bug. Is there any indication that they intended the field to be validated? Provide evidence by showing that the field is validated in some cases but not in others. If you don’t provide evidence, it’s an unproven claim.
A visual or a content problem becomes a functional problem when it hinders a functionality and thus should be reported as a functional bug.
A piece of functionality consistently works the same way in different scenarios and without obvious problems? Then it’s probably intended (not a bug) and you just suggest a change (usability suggestion).

Severity Assessment

When judging the functional severity level of a bug, several factors have to be considered: functional impact, workaround possible vs. showstopper, extent of the problem, potential and notable loss of sales, and comparison to other bugs of the same severity.

A straightforward approach is to look at the bug's functional impact. Think about how severe it is that a function is not available. Subordinate functionality will not block users from pursuing their goal – other than broken main functionality. Ask yourself how relevant that piece of functionality is in the context of the whole product.

Consider if you can reach your goal via an alternative route or option or if a piece of functionality remains unavailable. When you intuitively and easily find a way around a bug, this so-called workaround enables you to still reach your goal. A bug with a workaround receives a lower severity level than an equivalent bug without a workaround. Finally, when there is no workaround for broken main functionality, it is a showstopper.

The question of how many people, products, or items are affected by a functional problem is a determining factor for the extent of a problem. Does, for example, the "Add to Cart" button not react on all product detail pages of a webshop or only on a particular one? Is a small group of users concerned by the problem, or everyone?

Estimating a potential loss of sales is a secondary approach as you often can only assume how people might react to a bug. Nevertheless, take into account how high a potential loss is. It makes a world of difference if the price of a product differs by cents or hundreds of dollars.

After all, you can also compare your bug to those that the same team leader already approved in this test in order to find out if your severity level is appropriate.

We have three severity levels for functional bugs:

LOW:

Minimal impact on the usage of the product.
The product shows unintended behavior, but the general usage is not affected.
Few users, products, or items are concerned.
A feature/piece of functionality is broken or unavailable, but an easy workaround solves the problem.

HIGH:

Serious impact on the usage of the product, but the main functionality is intact.
A large number of users, products, or items is concerned.
Non-trivial functionality is broken or unavailable and no workaround exists.
Important functionality is broken or unavailable but a workaround exists (hence not a showstopper).

CRITICAL:

The bug prevents the core functionality of the app or website.
A showstopper prevents the user from continuing with a main process, e.g. the checkout process.
The bug causes a potential and notable loss of sales for the company running the app or website.

Common assessments

We have a list of specific cases that have fixed severity levels. The above assessment scheme does not apply to cases on that list. As the list will be updated over time, please check it regularly.

Take me to the Bug Assessment Sheet!

Reduced severity for specific environments

Due to the fact that some browser and operating system versions are outdated and their market share has decreased, bugs that only occur on environments of low relevance receive a lower severity level assessment. These environments are:

IE 10 and below
iOS 10 and below
Android 5.0 and below

Severities are reduced the following way:

Critical turns into High (exception: severity stays Critical in Rapid tests)
High turns into Low
Low remains Low

Edge case bugs

Edge case bugs occur when a piece of functionality is used in a unusual way. The functionality is not broken when used with typical data and typical user actions. Here are some examples:

Instant actions, such as minimizing an app after clicking on a button
Repeatedly doing the same thing, e.g. opening and closing menus
Any bug that only occurs after an uncommon set of actions

Each case must be evaluated separately. Edge case bugs that are relevant for our customers are forwarded as Low bugs. Irrelevant cases, which account for most edge case bugs, will be rejected.

Forced bugs

Forcing a bug by nontypical behavior or special conditions is generally out of scopeas such bugs are not relevant for our customers. Nontypical behavior does not reflect normal user behavior. Examples of nontypical behavior or special conditions:

Tapping on multiple elements at the same time
Random button pressing
Fast-paced clicking on a button several times
Reducing the size of your window to non-typical sizes
Full RAM or internal memory leading to unexpected behavior
Using unofficial, beta, or modified OS versions