Bug Reporting – Art and Advocacy
This article highlights the essence and traits of finding bugs. It leaps to redefine the art, a tester should inculcate while finding a bug. It enumerates various artifacts in reporting a bug. Whereas, also voices the advocacy on the bugs that has been reported. The basic amenity of a tester being to fight for the bug until it is fixed.
As testers, we all agree to the fact that the basic aim of the Tester is to decipher bugs. Whenever a build appears for testing, the primary objective is to find out as many bugs as possible from every corner of the application. To accomplish this task as perfection, we perform testing from various perspectives. We strain the application before us through various kinds of strainers like boundary value analysis, validation checks, verification checks, GUI, interoperability, integration tests, functional – business concepts checking, backend testing (like using SQL commands into db or injections), security tests, and many more. This makes us to drill deep into the application as well as the business.
We would agree to the fact that Bug Awareness is of no use until it is well documented. Here comes the role of BUG REPORTS. The bug reports are our primary work product. This is what people outside the testing group notices. These reports play an important role in the Software Development Life Cycle – in various phases as they are referenced by testers, developers, managers, top shots and not to forget the clients who these days demand for the test reports. So, the Bug Reports are remembered the most.
Once the bugs are reported by the testers and submitted to the developers to work upon, we often see some kinds of confrontations – there are humiliations which testers face sometimes, there are cold wars – nonetheless the discussions take the shape of mini quarrels – but at times testers and developers still say the same thing or they are correct but the depiction of their understanding are different and that makes all the differences. In such a situation, we come to a stand-apart that the best tester is not the one who finds most of the bugs or the one who embarrasses most programmers but is the one who gets most of the bugs fixed.
Bug Reporting – An Art:
The first aim of the Bug Report is to let the programmer see the failure. The Bug Report gives the detailed descriptions so that the programmers can make the Bug fail for them. In case, the Bug Report does not accomplish this mission, there can be back flows from the development team saying – not a bug, cannot reproduce and many other reasons.
Hence it is important that the BUG REPORT be prepared by the testers with utmost proficiency and specificity. It should basically describe the famous 3 What’s, well described as:
What we did:
- Module, Page/Window – names that we navigate to
- Test data entered and selected
- Buttons and the order of clicking
What we saw:
- GUI Flaws
- Missing or No Validations
- Error messages
- Incorrect Navigations
What we expected to see:
- GUI Flaw: give screenshots with highlight
- Incorrect message – give correct language, message
- Validations – give correct validations
- Error messages – justify with screenshots
- Navigations – mention the actual pages
Pointers to effective reporting can be well derived from above three What’s. These are:
1. BUG DESCRIPTION should be clearly identifiable – a bug description is a short statement that briefly describes what exactly a problem is. Might be a problem required 5-6 steps to be produced, but this statement should clearly identify what exactly a problem is. Problem might be a server error. But description should be clear saying Server Error occurs while saving a new record in the Add Contact window.
2. Bug should be reported after building a proper context – PRE-CONDITIONS for reproducing the bug should be defined so as to reach the exact point where bug can be reproduced. For example: If a server error appears while editing a record in the contacts list, then it should be well defined as a pre-condition to create a new contact and save successfully. Double click this created contact from the contacts list to open the contact details – make changes and hit save button.
3. STEPS should be clear with short and meaningful sentences – nobody would wish to study the entire paragraph of long complex words and sentences. Make your report step wise by numbering 1,2,3…Make each sentence small and clear. Only write those findings or observations which are necessary for this respective bug. Writing facts that are already known or something which does not help in reproducing a bug makes the report unnecessarily complex and lengthy.
4. Cite examples wherever necessary – combination of values, test data: Most of the times it happens that the bug can be reproduced only with a specific set of data or values. Hence, instead of writing ambiguous statement like enter an invalid phone number and hit save…one should mention the data/value entered….like enter the phone number as 012aaa@$%.- and save.
5. Give references to specifications – If any bug arises that is a contradictive to the SRS or any functional document of the project for that matter then it is always proactive to mention the section, page number for reference. For example: Refer page 14 of SRS section 2-14.
6. Report without passing any kind of judgment in the bug descriptions – the bug report should not be judgmental in any case as this leads to controversy and gives an impression of bossy. Remember, a tester should always be polite so as to keep his bug up and meaningful. Being judgmental makes developers think as though testers know more than them and as a result gives birth to a psychological adversity. To avoid this, we can use the word suggestion – and discuss with the developers or team lead about this. We can also refer to some application or some module or some page in the same application to strengthen our point.
7. Assign severity and priority – SEVERITY is the state or quality of being severe. Severity tells us HOW BAD the BUG is. It defines the importance of BUG from FUNCTIONALITY point of view and implies adherence to rigorous standards or high principles. Severity levels can be defined as follows:
Urgent/Show – stopper: Like system crash or error message forcing to close the window, System stops working totally or partially. A major area of the users system is affected by the incident and It is significant to business processes.
Medium/Workaround: When a problem is required in the specs but tester can go on with testing. It affects a more isolated piece of functionality. It occurs only at one or two customers or is intermittent.
Low: Failures that are unlikely to occur in normal use. Problems do not impact use of the product in any substantive way. Have no or very low impact to business processes
State exact error messages.
PRIORITY means something Deserves Prior Attention. It represents the importance of a bug from Customer point of view. Voices precedence established by urgency and it is associated with scheduling a bug Priority Levels can be defined as follows:
High: This has a major impact on the customer. This must be fixed immediately.
Medium: This has a major impact on the customer. The problem should be fixed before release of the current version in development or a patch must be issued if possible.
Low: This has a minor impact on the customer. The flaw should be fixed if there is time, but it can be deferred until the next release.
8. Provide Screenshots – This is the best approach. For any error say object references, server error, GUI issues, message prompts and any other errors that we can see – should always be saved as a screenshot and be attached to the bug for the proof. It helps the developers understand the issue more specifically.
This is an interesting part of the forum as every now and then we need to fight for each and every bug of ours to prove that the bug really is a bug or the bug really needs to be fixed as it is impacting the application. In the course, we often hear following comments from the programmers / give such comments to programmers:
Scenario 1: "Works for me. So what goes wrong?" – Developers often say that the bugs are not reproducible. They say that this is working fine at their system. In such a case a tester needs to patiently hear and see what exactly the developer means by his statements. He needs to find out where the difference of opinions and understanding lies. It is not always that we have understood the system right. It is quite possible that what we say is wrong and can be rightly done by them.
Scenario 2: “So then I tried . . ." – It is often seen, that with the due pressure on the tester, in the course of finding and reporting bugs, he forgets that the tests need to be performed on the stable condition of the application whereby the application shows a consistent behavior. A tester enter a phone number as special characters – impact of saving this can be a crash or overflow error…even without checking this he also enters the name as 150 characters and save them altogether – again this data can also give some error. Sometimes, the system gives some error and over that we continue to work further on the system until it crashes – then we report a bug. So, in such a case further actions worsens the problem.
Scenario 3: "That’s funny, it did it a moment ago." – There are circumstances when programs that fail once a week, or fail once in a blue moon, or never fail when you try them in front of the programmer but always fail when you have a deadline coming up. So, it is important to keep snapshots, test data, databases trace, xml traces, etc for that matter.
Scenario 4: “And then it went wrong” – When the tester himself is not clear about the steps he has performed or the data he has entered and approximately reports down the bug – certainly the bug might get irreproducible. This can lead us and system nowhere or in sea of problems – we can neither close the bug nor can we describe the bug and worst even the tester cannot reproduce the bug or fight for the cause of the bug!
It is important that we report everything that we feel, we do, what is the impact and measure this impact in terms of severity and priority. A tester is the catalyst of any team – he makes up the team on one hand and breaks up the application on the other. It is important to note all the issues big and small in the application with due course of understanding in terms of business and application – thereby be a face value through strong BUG REPORT as a proof and status of bugs updated at all stages in Software Development Life Cycle. Happy Bugging!!
– Priyanka Prakash
About the Author:
The author – Priyanka Prakash is an eminent engineer in computer science. She has over 5yrs of experience into software testing. She has worked with several IT companies of high reputation in numerous domains. She is currently working with a CMMI Level 5 and PCMM organization in Gurgaon.