Home

Showing posts with label Exploratory Testing. Show all posts
Showing posts with label Exploratory Testing. Show all posts

Tuesday, April 26, 2022

A Novice's Tale: Learning the Ropes of Game Testing

As I delve into the world of game testing, I'm discovering the many layers that make this field so intriguing. My journey has led me to test games on platforms like the Nintendo Switch and Steam, each offering unique challenges and learning opportunities. A significant part of my testing experience has been with the charming world of "Animal Crossing: New Horizons" on the Nintendo Switch. The game's delightful characters and immersive environment have provided a rich backdrop for honing my testing skills.

Game testing is far from just leisurely play; it's a critical examination of a game's every aspect to ensure a smooth player experience. For example, during my sessions with "Animal Crossing: New Horizons," I encountered a peculiar issue where my character began to move sluggishly without any input from the controller. This odd behaviour led me to investigate the controller's calibration, revealing it was off-center at rest—a potential compatibility issue between the game and the hardware. Such findings highlight the importance of thorough testing to catch issues that could hinder gameplay enjoyment.

Courtesy: Nintendo

The game testing process is methodical, beginning with drafting a test blueprint. This blueprint considers recent game changes, new supported environments or hardware, potential test scenarios, and any feature modifications. Following the blueprint, the test strategy is executed, which includes setting up the testing environment, identifying and documenting issues, verifying fixes, and ensuring comprehensive coverage through repeated testing. Afterward, the test plan is revisited to confirm that the fixes hold up to our quality standards.

Exploratory testing is a dynamic part of game testing that relies on discovery and intuition rather than a set script. While testing "Animal Crossing: New Horizons," I often relied on my gaming instincts to guide me through the game's vibrant world, uncovering issues that structured testing might miss.

In addition to exploratory testing, other essential testing types ensure a game's quality, such as functionality testing, compatibility testing, progression testing, and regression testing. Each plays a vital role in the game testing process.

Game testing is a complex field that blends technical skills with an understanding of the player experience. It's a challenging yet rewarding endeavour that significantly contributes to creating games that resonate with players worldwide. As I continue my journey in game testing, I eagerly anticipate the learning experiences ahead and the chance to contribute to crafting enjoyable and engaging gaming experiences.

Wednesday, March 7, 2012

A Simple Solution of A Common Problem


Problem:

A new aspiring tester in the team was given the test cases and asked him to execute the test cases. This new aspiring tester was just completed his degree in computer engineering. He was one of the brilliant students in his batch. Everybody including the management of the company was expecting him to do well in his job. So this new tester started executing the test cases for the first time and then for the next time and so on. All test cases were passed and no bug was logged.  

After a month the tester was terminated from the project.  As per management the new aspiring tester couldn't become a tester because he didn't have the flame that a tester must have. The Tester was expelled from the team. Now he was sitting on the bench and thinking what went wrong.

How many of you have seen this problem? Actually I have seen a few and I had been also thinking what went wrong in this case.

Understanding the Problem:

Recently I was reading a clinical tale by Oliver Sacks. The tale was about a patient who was congenitally blind with cerebral palsy. The conditions are more pathetic than we can think of – she was suffering of spasticity and athetosis, i.e., involuntary movements of both hands, which was added due to failure of the eyes.

After initial tests and checkup it was found that there was no sensory ‘deficit’. Her hands would seem to have the potential of being perfectly good hands—and yet they were not. They were functionless—’useless’—because she had never used them. Had being protected and looked after since birth prevented her from the normal exploratory use of the hands which all infants learn in the first months of life.

Can you correlate the problem with the case mentioned above?

The real problem is that new aspiring tester was being pampered by the management itself. Instead of really doing the testing he was asked to follow the test cases which are most probably written by someone else. Initially, following the test cases made his task easy. He used to execute the test cases and finishes the task early. He couldn’t learn how to test from this exercise and we all know the result. The tester was put on bench (Literally killed the aspiring tester)

Solution:

A new aspiring tester in the team was given the application and asked him to test it for all the functional and usability issues. This new aspiring tester was just completed his degree in computer engineering. He was one of the brilliant students in his batch. Everybody including the management of the company was expecting he would do great in his profession. So this new tester who was hesitant initially started with exploring the application. Think about the infant who reaches for the breast when feeling hungry. Similarly the new aspiring tester took some time but he did some testing and logged few good issues.

Benefits from this Solution:
  • Confident and satisfied tester
  • Better Testing
  • More No. of bugs
  • A lively job
  • Better Self Education
  • Happy Management
What do you think? Does it seem a right solution to you?

Reference: "The Man Who Mistook His Wife For A Hat" by Oliver Sacks

Wednesday, October 19, 2011

Search by POST method–A small research

This post is based on a small research I have done on Mozilla Firefox Search Plug-in.

Searchbar

The research started with the study of cookies stored by Mozilla Firefox and moved towards building the Mozilla Firefox Search Plug-in for my blog. My research is not finished yet but still I would like to share an opportunity I found testing these Search Plug-ins.

Testing Nook

While exploring these search plug-ins I noted that every search plug-in is using GET method to send the search query to Search Engine. It seems fine to me as it is just a search so using POST method to send the request doesn’t make a sense to me, but then I thought other way – What would happen if POST method is used to send the request to server. I got curious about this and start to edit the properties of these plug-ins. I have tested multiple search plugins to see how corresponding search engines handles the request sent by POST method and found these three different behaviours:

Google: Google was first search plug-in I tested. When I sent search request to Google using POST Method, I got an error page like this:

Error 405 (Method Not Allowed)!!1 2011-10-18 20-08-21

It seems appropriate. Google doesn’t serve any search request being send by POST Method but then a question is raised - does a non technical user really understand what is POST method? For eg.: My wife doesn’t know this terminology and she will definitely call me for help if she get such error from the Google.

(See the Error Code 405. I have not seen this error earlier. So my research also helped me to find a new error message Winking smile)

Yahoo: The next search engine was Yahoo. Initially I thought that Yahoo will give the same result as given by Google but it proved me wrong. To my surprise Yahoo has given the result for the search query I sent using POST method.

Hello - Yahoo! Search Results 2011-10-18 20-11-15

Bing: The third major search plugin which I tested was Bing – A Microsoft Product. I love my job because it surprises me and it surprises me daily. Till now I have tested two major search engines Google and Yahoo dealing with the request sent using POST method and both of them have given me different results. Once again I was guessing the result before testing. Based on my two previous results I was thinking that Bing should either display the error message or it should display the search result. I have not thought of any other result but surprisingly, Bing has given some unexpected result:

Bing 2011-10-18 20-20-39

Instead of displaying search results or any error, it has opened Bing Search Engine.

I have tested for few other search plugins for different search engine but result are more of the same as these three. Now, question is which search engine has served the user’s request properly: Google, Yahoo or Bing. Think as a tester, Think as a developer and Think as an end user.

Post me your answers with the reason in Comments section, we can have a good discussion on it.

Saturday, November 13, 2010

A Feature which have become Bouquet of Bugs


Recently, I was testing an online shopping site – just for fun, nothing official. There I saw a feature called as “Tell A Friend”.  I found this feature useful in first glance. During shopping, if you see any product which you find usable for your friend and you think your friends might be interested, just forward info about the product in the mailbox of your friend. You just have to write name & email address of your friend. I think this is great idea to make new customers.
So, I thought to take my hands on this feature but I was highly disappointed when I used this feature. In hurry to introduce the new feature, the site managers kept a bouquet of bugs on the site. Let’s come with me to see the dissection of this feature with me.
Oh… I forgot to tell you the store name. It’s Zappos.com. First, some introduction with Zappos:
Zappos.com is an online shoe and clothing shop. Since its founding in 1999, Zappos has grown to be the largest online shoe store. Zappos did "almost nothing" in sales for 1999, but grossed over USD$800 million in merchandise sales in 2007 and grossed over $1 billion in 2009.
On Zappos.com I selected a product to see its description. The product seemed good to me and I was sure one of my friends would be interested in this product. I decided to forward the details of the product to my friend. So I clicked on “Tell A Friend” button available on the product page.

So I was on “Tell A Friend” Window.

After seeing this window I thought to play with this.
I clicked on Send button to see what would happen if values were not filled. On clicking send I saw two error messages. The error messages were:
  • ·         You  didn’t specify an email to send to
  • ·         You need to supply an email

Displaying an error message is not a formality. They could phrase the error messages in much better way.  Also they displayed the error messages in reverse order of the fields on the form. In the above screenshot you can see that the sender’s email address field appears before receiver’s email address. The error message should also display in the same order. This is not a rule but this is always expected.
From error messages it was clear that sender’s name is not mandatory here. Just one questions from all my readers – How many of you know email addresses of your all friends?
As a friend, I know all my friends by their name and not by their email addresses. So, I believe Sender’s name should be mandatory here.
After this I checked for the validations on the fields.
I filled numerals values in sender’s name, email and receiver’s email address fields.

Clicked on send button. The message had been sent.

As you can see there is no validation on sender’s name, email & receiver’s email address. The message had been sent. I didn’t know which poor devil was going to get this mail.
I have doubt that some professional developer has developed this window. It seems any school going kid has done the job. Even they are also aware of these validations.
Now let’s see the max length validation.
I used Perl Clip for generating the string of one million characters and pasted the string in each field.

No surprise, each field has accepted one million characters. Now I wanted to see what would happen if I click on Send button. So I did the job.
I had doubt that this message would ever go and guess what, I was right.

If they had put max length validation for each field, you have not to see this error. From here, I was sure that I was going to get many bugs in this feature.
This time I thought to select the checkbox to send a personalize message with the product details. So I clicked on check box. It displayed one more field along with Captcha verification.

I clicked on send button again without filling up any data in any field.

As you can see, It had given the same error messages. I was surprised why it didn’t show any error message for personalize message or Captcha. On looking down I saw that marked checkbox was unmarked. So you see on page refresh the data was lost.
Again, I filled up the complete form

and clicked Send button. See, what I got here.
 

Now, this was the biggest joke. The Captcha verification had been provided but application was unable to check the Captcha. Then, what it was doing here? Moreover, instead of fixing the error, the application was suggesting me to send the message without personalize message.
While sending the product details I wanted to tell my friend that why I have chosen this product for him but I couldn’t do that.
As application was unable to send a personalize message, it had also unmarked the checkbox itself, so I decided to send the product details without personalize message. I clicked on send button again.
What?????
It was still looking for Captcha.

To check behind the curtains I clicked on checkbox.
OMG!!!!

Text message was still there. Captcha also had been refreshed.  It means I couldn’t send product details until I remove the personalize message and Captcha text.
One funny thing I noted here: When Captcha text was incorrect, it has given the right error message but when correct Captcha text was entered earlier it was unable to process.
I was unable to understand whether they are making my job easy or tough. I could send direct mail to my friend much faster rather than using this add-on.
Next, I opened the “Tell A Friend” in Mozilla tab as you can see in the following image.

I wanted to see on clicking Cancel button whether it would close the single tab or all tabs. So I clicked on Cancel button. I was expecting tab to be closed but nothing happened.
So, I filled the data in the field and click on Send button. The message was sent.

I clicked on Close button but as usual nothing happened.
Finally I decided to close the window myself and never tried to open it again. I was frightened with “Tell A Friend”.
Before finishing, I just have one request – Just try to reduce the tasks of a user instead of multiplying them. They have lots of things to do in life. Before releasing, please make it sure that application doesn’t have such kinds of stupid bugs.

Tuesday, April 27, 2010

Navigating the Quirks: A Light-Hearted Look at Gmail's User Experience Blunders

I would like to dedicate this post to Santhosh Tuppad. It was Santhosh's post and the comments on the post which gave me idea to write this post. The post is going to heart break some Google fans but I think now they should accept that Google is also fallible.

Mission: To display the stupid mistakes to Google devotees in Gmail Registration form.
Time: 0.5 Hrs

How should I change the language here?
Scenario: I am a common user and I don’t have internet access at home so I go to cyber café to use the internet services. On such a public computer, I opened the GMail and found the following screen:



I don’t know Hindi so I am unable to read the content on the screen. How could I change the language to English?

Perhaps, Google forgot to provide the Change Language dropdown here.

GMail Help – Most Stupid Mistake
Scenario: I am using the GMail in Hindi and clicked on ‘Can’t access your account?’ link (Written in Hindi Translation) and redirected to following page:

 Somebody must tell Google that upper right side should be towards upper side of the page and not at bottom of the page.

Now I have doubt on Google’s direction sense. How could I believe on Google Earth now?

Where is the play captcha image?
I am wondering where the play captcha image for this captcha is.

 Play Captcha image: The wheelchair icon with captcha, on clicking you'll be presented with an interesting voice version of the captcha. This is generally used by physically disabled people.

Where is the Refresh button with Captcha image?
Scenario: I am unable to understand the letters in the captcha image. What should I do here?

Should I submit the incorrect word and try again for the new captcha, there is not guarantee that letters in new captcha will be readable

Or Should I refresh the page which will cause the lost of information that I already filled up.

Sorry, we are unable to present username suggestions at this time.

See the screenshot below:


There could be various reasons for this message:
1. GMail server might be busy with any female server :D
2. Its lunch time. :D
3. GMail server may be using the loo :D

Actually, GMail server suggests username on the basis of First name and Last Name entered by user and username can be in English Language only. Here, I have entered Hindi letters and he couldn’t process the request so instead of displaying its inability to suggest username, it has given this error message.

Incorrect Hindi Writing
Neither I had time to search them all nor do I have any spellchecker for Hindi Language so I have mentioned only one mistake in the screenshot. There might be more mistakes in Hindi Spellings.

 The word should be written as ऊपर and not उपर


Displaying Error message without wrong Input

When I clicked on Check availability button, it displayed me a captcha with a field to enter the characters in the captcha and an error message which says
The characters you entered didn't match the word verification. Please try again.

When I didn't enter any word then why they are saying that I have entered incorrect word. This message should appear if I enter the incorrect characters.


Relevant Information is not available on Time

I have filled up the password with the characters which are not allowed by GMail. It displayed me the following error after submitting the form:

The following characters are allowed in your password: a-z, A-Z, 0-9, and common punctuation characters.

If password characters were not valid then why didn’t it displayed the error when I tabbed from the choose password field to next field i.e. re-enter password.

If it has displayed the error at right time I could save a lot of time.

I hope GMail server is not allergic to the user name which I have selected.

In the screenshot, see the distance between the field and error message. It seems that GMail server is allergic to the selected username and that’s why the error message going far away every time I click check availability button.



I am not sure but the reason might be: Whenever user clicks on ‘Check Availability’ button, it generates same error message. To avoid the duplication of error messages, the developer had hid the repeating error message.

The funny thing is when these kind of stupid mistakes are mentioned by any passionate tester who is doing it without any greed, few people got offended. According to them “No one will die and no two countries will go to war.” Believe me if we tester start to keep this point in mind, we don’t need UI / Look and feel testing then.

It took just 30 minutes to find the above issues and I am sure if any tester devotes his some time he can find more issues in Google products.

Trust the Google but with open eyes.

Tuesday, November 10, 2009

Lessons Learned from Pradeep Soundararajan

Chapter 2 - Kids can do Boundary Value Analysis!!

My niece is 5 years old. She doesn't know what is software testing, in fact she doesn't know what is software but still she can do Boundary Value Analysis. Amazing na.... although I have never asked her to do so but still I am sure that she can do it.

Surely, you will be surprised how I am so sure? Actually, she can add the numbers, subtract the numbers and that is exactly what most of us do on the name of Boundary Value Analysis.

I was also following the same approach until Pradeep Sir has not mentioned it in one of his workshop. The question is - Are we really doing analysis?

The Wikipedia says:
Analysis is the process of breaking a complex topic or substance into smaller parts to gain a better understanding of it.

As definition says Analysis is just not adding or subtracting 1. It seems that we have changed the definition of analysis. If what we are doing is analysis then perhaps the software testers at NASA are doing the same :) They are just adding and subtracting the numbers. If that so, any body who can do addition and subtraction can join panel of Software Testers in NASA.

Boundary value analysis is a software testing design technique in which tests are designed to include representatives of boundary values. Values on the edge of an equivalence partition or at the smallest value on either side of an edge. The values could be either input or output ranges of a software component.

The definition doesn't mention +1/-1 approach. It also doesn't tell us that BVA can be be applied on input fields only which we generally do.

It is said that most of the bugs reside on the boundaries. Don't know who made this statement. Might be a tester has made the statement when he found most of the bugs at the boundaries when he was testing an application.

But is he right?

Might be he is right but did he really applied the +1/-1 approach. I don't think so. Check out the total bugs logged by you and then see how many bugs you have found by +1/-1 approach. I am sure the percentage would be very low.

Perhaps we are misunderstanding the concept of Boundary Value Analysis or Might be we need to rename the +1/-1 approach. How about calling it “Kids Approach”.

Think about it tell then let me try to find out what is Boundary Value Analysis?

Wednesday, November 4, 2009

Lessons Learned from Pradeep Soundararajan

Chapter 1 - A Great Learning Opportunity

It was just like an ordinary day. I am working in the office doing tests, having discussions with developers & meetings etc. I was not on my desk for last two hours due to such a meeting. When I returned back I saw a message from Pradeep Sir. I was little bit amazed as I don't receive messages from him regularly so i pinged him back immediately and what he said has changed the ordinary day into special one.


Pradeep Sir was coming to Noida for a corporate workshop and he offered me to attend the workshop for free (not completely free). I was more than happy but two question were in my mind :) (Question should always be there :))

1. Why Pradeep Sir has offered me to attend his workshop on less amount?

2. What would I need to pay to attend the workshop?


But next day I got the reply of my each question.


I don't remember the exact wordings which he used but they are some thing like : Mohit, money is not a big thing. I have sufficient money, so you need not pay anything. Last time you have paid from your own pockets to attend my workshop so this is your reward from me. What I really want from you to learn, learn and learn more and try to help the testing community from your learning and this would be your fee.


I promised him to do so but couldn't do anything yet what I have promised. If I have done so then you would definitely read this article at least 15 days ago but I hope Pradeep Sir know what were the reasons for being late.


Anyway, the two days workshop from Pradeep Sir was one of the best experience.


Don't know what is best... Attending the workshop on Exploratory Testing from the country's best Exploratory Tester, Watching him doing testing, Having discussion with him over lunch or having a vodka shot after the workshop.


The workshop has started with the introduction of Pradeep Sir and ends with SBTM (Session based Test Management). In between we have spent 17 hrs learning the testing skills from one of India's best tester.


Only one blog post can not comprise the whole learning session and it would not be justified to do so I am going to post them in parts one by one. Wait for my next blog post which will discuss “Kids can do Boundary Value Analysis!!”.

Thursday, November 6, 2008

Exploratory Testing - An Introduction

Exploratory Testing - The term coined by Cem Kaner in 1983. He defines exploratory testing (ET) as
A style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the quality of his/her work by treating test - related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parrallel throughout the project.
The definition is too explicit. In simple words we can say that:
Exploratory Testing is an approach of software testing that can be described as simultaneous learning, test design and test execution.
Exploratory testing is not pre-planned as in case of test cases based testing (i.e. scripted testing) but it is done with a point of view exploring features of the product and to find bugs which were not logged during scripted testing. Every Software Tester knowingly or unknowingly does it. In exploratory testing, next result is influenced by the result of last test.

Jon Bach has defined the difference between scripted testing and exploratory testing with a simple and interesting example in his article. He says:

The difference between scripted testing and exploratory testing is as simple as describing
the game “20 Questions.” In this game, one person thinks of an object (an animal, vegetable, or mineral) and another person is allowed 20 yes or no questions to discern what the object might be. If the guesser can’t determine what the object is by the 20th question, they lose the game.

The important aspect of the game is that the guesser asks one question at a time, and an answer is given before the guesser decides what their next question will be.

This is exploratory testing in action. The game would not work if a scripted testing paradigm was used, where the guesser had to submit their 20 questions in advance and could not adapt their questions to each answer.

Unlike scripted testing, exploratory testing relies on this adaptation. Like the “20 Questions” guesser, the tester is free to design and execute their next test idea based on the results of the previous test. To test wisely, the tester can draw from anything in their cognition – experience, heuristics, biases, observations, conjectures, notions, and hunches, and on and on – but culling through all of that cognition, choosing that next test, playing that hunch and applying that experience – takes skill.
The outcome of an exploratory testing session is a set of notes about the product, failures found, and a concise record of how the product was tested. When practiced by trained testers, it yields consistently valuable and auditable results.

Benefits of Exploratory Testing:
  1. The main advantage of exploratory testing is that less preparation is needed to test and bugs are found faster.
  2. Exploratory testing helps tester to understand the functionality of the product thoroughly; hence covering the most important part of requirement understanding.
  3. Exploratory testing is especially useful in complex testing situations, when little is known about the product
  4. As in case of exploratory testing, we write and execute the test cases simultaneously. It helps in collecting result oriented test scripts and shading of load of unnecessary test cases which do not yield and result.