Home

Showing posts with label Testing. Show all posts
Showing posts with label Testing. Show all posts

Wednesday, October 19, 2011

Search by POST method–A small research

This post is based on a small research I have done on Mozilla Firefox Search Plug-in.

Searchbar

The research started with the study of cookies stored by Mozilla Firefox and moved towards building the Mozilla Firefox Search Plug-in for my blog. My research is not finished yet but still I would like to share an opportunity I found testing these Search Plug-ins.

Testing Nook

While exploring these search plug-ins I noted that every search plug-in is using GET method to send the search query to Search Engine. It seems fine to me as it is just a search so using POST method to send the request doesn’t make a sense to me, but then I thought other way – What would happen if POST method is used to send the request to server. I got curious about this and start to edit the properties of these plug-ins. I have tested multiple search plugins to see how corresponding search engines handles the request sent by POST method and found these three different behaviours:

Google: Google was first search plug-in I tested. When I sent search request to Google using POST Method, I got an error page like this:

Error 405 (Method Not Allowed)!!1 2011-10-18 20-08-21

It seems appropriate. Google doesn’t serve any search request being send by POST Method but then a question is raised - does a non technical user really understand what is POST method? For eg.: My wife doesn’t know this terminology and she will definitely call me for help if she get such error from the Google.

(See the Error Code 405. I have not seen this error earlier. So my research also helped me to find a new error message Winking smile)

Yahoo: The next search engine was Yahoo. Initially I thought that Yahoo will give the same result as given by Google but it proved me wrong. To my surprise Yahoo has given the result for the search query I sent using POST method.

Hello - Yahoo! Search Results 2011-10-18 20-11-15

Bing: The third major search plugin which I tested was Bing – A Microsoft Product. I love my job because it surprises me and it surprises me daily. Till now I have tested two major search engines Google and Yahoo dealing with the request sent using POST method and both of them have given me different results. Once again I was guessing the result before testing. Based on my two previous results I was thinking that Bing should either display the error message or it should display the search result. I have not thought of any other result but surprisingly, Bing has given some unexpected result:

Bing 2011-10-18 20-20-39

Instead of displaying search results or any error, it has opened Bing Search Engine.

I have tested for few other search plugins for different search engine but result are more of the same as these three. Now, question is which search engine has served the user’s request properly: Google, Yahoo or Bing. Think as a tester, Think as a developer and Think as an end user.

Post me your answers with the reason in Comments section, we can have a good discussion on it.

Saturday, November 13, 2010

A Feature which have become Bouquet of Bugs


Recently, I was testing an online shopping site – just for fun, nothing official. There I saw a feature called as “Tell A Friend”.  I found this feature useful in first glance. During shopping, if you see any product which you find usable for your friend and you think your friends might be interested, just forward info about the product in the mailbox of your friend. You just have to write name & email address of your friend. I think this is great idea to make new customers.
So, I thought to take my hands on this feature but I was highly disappointed when I used this feature. In hurry to introduce the new feature, the site managers kept a bouquet of bugs on the site. Let’s come with me to see the dissection of this feature with me.
Oh… I forgot to tell you the store name. It’s Zappos.com. First, some introduction with Zappos:
Zappos.com is an online shoe and clothing shop. Since its founding in 1999, Zappos has grown to be the largest online shoe store. Zappos did "almost nothing" in sales for 1999, but grossed over USD$800 million in merchandise sales in 2007 and grossed over $1 billion in 2009.
On Zappos.com I selected a product to see its description. The product seemed good to me and I was sure one of my friends would be interested in this product. I decided to forward the details of the product to my friend. So I clicked on “Tell A Friend” button available on the product page.

So I was on “Tell A Friend” Window.

After seeing this window I thought to play with this.
I clicked on Send button to see what would happen if values were not filled. On clicking send I saw two error messages. The error messages were:
  • ·         You  didn’t specify an email to send to
  • ·         You need to supply an email

Displaying an error message is not a formality. They could phrase the error messages in much better way.  Also they displayed the error messages in reverse order of the fields on the form. In the above screenshot you can see that the sender’s email address field appears before receiver’s email address. The error message should also display in the same order. This is not a rule but this is always expected.
From error messages it was clear that sender’s name is not mandatory here. Just one questions from all my readers – How many of you know email addresses of your all friends?
As a friend, I know all my friends by their name and not by their email addresses. So, I believe Sender’s name should be mandatory here.
After this I checked for the validations on the fields.
I filled numerals values in sender’s name, email and receiver’s email address fields.

Clicked on send button. The message had been sent.

As you can see there is no validation on sender’s name, email & receiver’s email address. The message had been sent. I didn’t know which poor devil was going to get this mail.
I have doubt that some professional developer has developed this window. It seems any school going kid has done the job. Even they are also aware of these validations.
Now let’s see the max length validation.
I used Perl Clip for generating the string of one million characters and pasted the string in each field.

No surprise, each field has accepted one million characters. Now I wanted to see what would happen if I click on Send button. So I did the job.
I had doubt that this message would ever go and guess what, I was right.

If they had put max length validation for each field, you have not to see this error. From here, I was sure that I was going to get many bugs in this feature.
This time I thought to select the checkbox to send a personalize message with the product details. So I clicked on check box. It displayed one more field along with Captcha verification.

I clicked on send button again without filling up any data in any field.

As you can see, It had given the same error messages. I was surprised why it didn’t show any error message for personalize message or Captcha. On looking down I saw that marked checkbox was unmarked. So you see on page refresh the data was lost.
Again, I filled up the complete form

and clicked Send button. See, what I got here.
 

Now, this was the biggest joke. The Captcha verification had been provided but application was unable to check the Captcha. Then, what it was doing here? Moreover, instead of fixing the error, the application was suggesting me to send the message without personalize message.
While sending the product details I wanted to tell my friend that why I have chosen this product for him but I couldn’t do that.
As application was unable to send a personalize message, it had also unmarked the checkbox itself, so I decided to send the product details without personalize message. I clicked on send button again.
What?????
It was still looking for Captcha.

To check behind the curtains I clicked on checkbox.
OMG!!!!

Text message was still there. Captcha also had been refreshed.  It means I couldn’t send product details until I remove the personalize message and Captcha text.
One funny thing I noted here: When Captcha text was incorrect, it has given the right error message but when correct Captcha text was entered earlier it was unable to process.
I was unable to understand whether they are making my job easy or tough. I could send direct mail to my friend much faster rather than using this add-on.
Next, I opened the “Tell A Friend” in Mozilla tab as you can see in the following image.

I wanted to see on clicking Cancel button whether it would close the single tab or all tabs. So I clicked on Cancel button. I was expecting tab to be closed but nothing happened.
So, I filled the data in the field and click on Send button. The message was sent.

I clicked on Close button but as usual nothing happened.
Finally I decided to close the window myself and never tried to open it again. I was frightened with “Tell A Friend”.
Before finishing, I just have one request – Just try to reduce the tasks of a user instead of multiplying them. They have lots of things to do in life. Before releasing, please make it sure that application doesn’t have such kinds of stupid bugs.

Monday, June 14, 2010

I am an Impersonator

I am a tester for some time now. A tester plays different roles for his job so do I and believe me I am happy about that. It seems that I am living life of an Impersonator. Some of you might be thinking what is an Impersonator?
An impersonator is someone who imitates or copies the behavior or actions of another.
Now when you know what is impersonator, you must be thinking why I am so happy about it? Actually I don’t think that any other career than testing can give me opportunity to play such different roles and this is not an exaggeration. When an actor can proud on his different characters played in different movies then why can’t I.

Different roles which I play

I am an Advisor: An Advisor is normally a person with more and deeper knowledge in a specific area i.e. a specialist. As a tester I generally play the role of an advisor. I give them suggestions from my experience with different products during development, I suggest them how they can make software better. I suggest them during bug fixing. Even I suggest them how to display an error message. What if they don’t follow every time what I suggested, sometimes even they don’t consider it but they always know I am doing best to help them and they are always keen to know my thoughts. That is sufficient for me.
I am a Police Officer: A Police officer is generally charged with the apprehension of criminals and the prevention and detection of crime, and the maintenance of public order
In Beautiful Testing, Linda Wilkinson has said:
Are testers policemen? Not usually. They can’t “arrest” anyone for breaking a law, even if the world of software development could figure out what those laws should be, and most do not have the authority to keep something from moving to production, regardless of the generic goodness (or badness) of the software.
According to me, a tester also wears a cap of a policeman. As a policeman, I find the culprit who breaks the laws of an application and put it behind the bar. The culprit in this case is a bug. I find the bug and log it in the bug sheet.
I am a Detective: A detective is any licensed or unlicensed person who solves crimes, including historical crimes, or looks into records. My job allows me to play the role of a detective.
For being a good tester, it is important to have good detective skills. A good tester is not who finds an issue, a good tester finds the issue and the reason behind that issue and here I need my Detective Skills to find the reason behind any issue. Sometimes I fail to find the reason but I always try. It helps me to know the product better than anyone else.
I am a Lawyer: A lawyer is a person who is learned in law and licensed to practice the law. As a detective I found the evidences and found the culprit, as a policeman I have put the culprit behind the bars but what if the culprit is escaped due to poor advocacy. To win a case we need strong prosecution against the culprit otherwise the defense lawyer can make him free. Similarly, sometimes we need strong prosecution skills against a bug. I need to describe the impact of the bug to the developer. In Software Testing it is called as Bug Advocacy.
I am a Critic: A critic is a person who offers reasoned judgment or analysis, value judgment, interpretation, or observation. Like a critic, I analyze a product according to my knowledge and share my views with the stakeholders about the product. I don't bother about what they want to hear actually, I just give them an honest opinion.
I am an End User: An end-user as the person who uses a product. This is the most important role which I play. It is an end user who is going to use the product finally so it is important to understand an end user for the success of the product. I test any product with the perspective of an end user and discuss my views with the development team and other stack holders so that they can improve the product accordingly.
I am a Reporter: A reporter is a type of journalist who researches and presents information in certain types of mass media.
Any news is breaking news if it is most significant story of the moment. It could be a story that is simply of wide interest to viewers and has little impact. As a reporter, I report the entire bug with all the evidences as soon as I find them. Late reporting can cause benefit to other reporters. Once an issue is found, I immediately report it.
I am a Salesman: A Salesman represents a person or company on the selling end of the deal. A real Salesman is that who sells the product to a customer when he needed it least. Sometimes I need to sell the bugs. I have found the bug, I have reported about it, On the basis of my bug advocacy it is also proved that the bug is critical and must be fixed but still I need to make the developers to fix it in current release. In that case, I use my Salesman skill to convince them to fix the bug immediately. It’s like selling a bug.
I am a Tester: and in the last but above all I am a tester. I need to change my role according to the situations but it is only because of the need of my Job. My basic job is to test the product. It is versatility of my job which allows me to wear the different caps at different time.

Am I missing any role? I might be....do tell me....... :)

Sunday, April 11, 2010

Bulls and Cows

There is an old proverb - Necessity is the mother of invention. When a scientist invents something there is always a need. The Need is the final motive of any invention.

What is Need?
Need is a condition or situation in which something is required. In short need is requirement.

But what if the need is wrongly interpreted, Is the invention possible in that case? Might be possible but it would not be right invention; it would not actually do what we needed.

Now we change the above proverb according to IT Industries.

Requirement is the mother of every product.

Now what if requirement is wrongly interpreted, is the product possible in that case? Might be possible but it would not be right product because it would not do actually what stakeholders needed.

To provide a quality product to client, it is very important to understand the needs of the client. The purpose of testing can't be fulfilled if tester doesn't understand the requirement properly.

Why the requirement is mis-interpreted?

There may be various reasons such as:
• The clients/ users actually don’t know themselves what they really want or what they need?
• When it is impossible to know who yours users will be and it is quite often if we don’t consider the narrowly defined situation such as an EHR application.
• In general Requirements are not detailed enough to understand the exact needs of the stakeholders.

How the situation can be avoided?

The best solution is questioning. Questioning can be used as a tool to understand the exact requirement of a client. More you ask questions, more you clear about the product.

To depict the power of questioning and how it helps to understand the requirements of a client, we can take example of a game called “Bulls and Cows

What is Bulls & Cows? - In childhood, you might play this game. This is also known as Cows and Bulls or Pigs and Bulls or Bulls and Cleots. This is a code breaking game played between two players.

How the Game is played?
- On a sheet of paper, one player writes a 4-digit secret number. The digits must be all different. Then, the second player tries to guess this number. If the matching digits are on their right positions, they are "bulls", if on different positions, they are "cows". Example:

Secret number: 9374
Opponent's try: 4873 and asks “Is this correct?”
Answer: 1 bull and 2 cows. (The bull is "7"; the cows are "3" and "4".)

On the basis of answer, the opponent tries again until he finds the secret number. By asking a single Question again and again, Opponents finds the correct number.
When a single question can help to find a number then imagine how many questions you can ask in a requirements document. Asking Questions is a skill and it comes with lots of practice and time. But if you know the skill of questioning, there is no way to mis-interpret the requirements.

My Suggestion:
If you find problems to understand the requirements then you should practice “Bulls & Cows”. If there is nobody to play with you, don't panic we have an open source version of “Bulls” and “Cows” called 4 digits.
The game's objective is to guess a four-digit number in 8 times using as less time as possible.

Monday, May 18, 2009

Black Box Testing

A testing technique in which tester doesn't need to know the internal structure of the code to test an application. The main objective of Black Box Testing is to test the functionality of the application as a whole.

The Black Box testing considers the application as black box so explicitly it doesn't need the knowledge of internal structure or code. While designing test cases, tester takes external perspective of the application and based on the perspective the test cases are derived which can be functional and non functional.

The Black Box testing techniques is applicable to all levels of testinng : Unit, Integration, System and Acceptance Testing.

Advantage of Black Box Testing:

  • More Effective when used for large systems

  • As the tester and developer are independent of each other, test is balanced and unprejudiced

  • Tester can be non-technical.

  • There is no need of having detailed functional knowledge of system to the tester.

  • Testing helps to identify the vagueness and contradiction in functional specifications.

  • Test cases can be designed as soon as the functional specifications are complete

Disadvantage of Black Box Testing:

  • Test cases are tough and challenging to design, without having clear functional specifications

  • It is difficult to identify tricky inputs, if the test cases are not developed based on specifications.

  • It is difficult to identify all possible inputs in limited testing time. So writing test cases is slow and difficult

  • Chances of leaving many program paths untested

  • Chances of repeating the tests that are already done by programmer.

Black Box Testing Techniques: Typical Black Box testing techniques are as follows:

  • Error Guessing

  • Equivalence Partitioning

  • Boundary Value Analysis

Error Guessing: Error Guessing is a testing technique which is based upon the tester's past experience, knowledge and intution to predict where the bugs can be found in application under test. Some areas to guess are: empty or null strings, zero instances, occurrences, blank or null characters in strings, Negative numbers etc.

Equivalence Partitioning: Equivalence Partitioning is a Black Box testing technique in which input domain data is divided into different equivalence data classes. This technique tries to define test case that uncovers classes of errors, thereby reducing the total number of test cases that must be developed.

For E.g.: Consider you are testing a date field which can accept date from 01-01-2001 to 31-12-2010 . In this case there is no use to write 3652 test cases for all valid inputs plus test cases for invalid inputs.

By using Equivalence Partitioning, we can divide all the inputs into three classes:

  1. Dates before 01/01/2001 - Input data class with all values below lower limit. i.e. any value below 01/01/2001, as a invalid input data test case.

  2. Dates from 01/01/2001 to 31/12/2009 - One input data class with all valid inputs. Choose a single value from 01/01/2001 to 31/12/2009 as a valid test case. One test case for valid input data is sufficient because If you select other values between 1 and 1000 then result is going to be same.

  3. Dates after 31/12/2009 - Input data class with all values upper to higher limit. i.e. any value after 31/12/2010, as a invalid input data test case.

By using Equivalence Partitioning, we have categorize all the test cases into three classes. Now we can choose an input value from each class to design our test cases. As you can see by using Equivalence Partitioning, we have covered maximum condition from fewest test cases.

Boundary Value Analysis (BVA): From Practice and Experience, it has been prove that most of the errors are found at the boundaries of input and output ranges of an application component that result in application faults. Boundary value analysis assists with the design of test cases that will exercise these boundaries in an attempt to uncover faults in the software during the testing process.

As Equivalence Partitioning, BVA is also used to design test cases where test cases are selected at the boundary of the input ranges.

For eg: In the above example test cases for date field accepting dates between 01/01/2001 to 31/12/2010 using BVA will be as follows:

  1. Test cases with test data exactly as the input boundaries of input domain i.e. values 01/01/2001 and 31/12/2010.

  2. Test cases with test data with values just below the boundaries of input domains i.e. 31/12/2000 and 30/12/2010.

  3. Test cases with test data with values just above the boundaries of input domains i.e. 02/01/2001and 01/01/2011.

After determining the necessary test cases with equivalence partitioning and subsequent boundary value analysis, it is necessary to define the combinations of the test cases when there are multiple inputs to an application component.

Sunday, November 2, 2008

Do you practice Epistemology?

The important parts of testing don't take place in the computer or on your desk. It's in a Tester's brain.

Few days ago, I was going through a lesson in "Lessons Learned in Software Testing" by Cem Kaner, James Bach & Bret Pettichord where authors said that testing is applied epistemology. Now what the hell is this Epistemology ? - don't need to mention the term was bounced over my head. So I thought to google about the term.

The first definition which I found on wikipedia says
"Epistemology or theory of knowledge is a branch of philosophy concerned with the nature and scope of knowlege."
In simple words Epistemology is the study of how we know, what we know. It tries to answer the basic question: what distinguishes true knowledge from false knowledge ?

Now the question is what is Knowledge?

The classical definition, described but not ultimately endorsed by 'Plato' has it that in order for there to be knowledge at least three criteria be fulfilled; that in order to count as knowledge, a statement must be justified, true and believed.

If above definition is considered, it means if someone knows X, that means that he or she
  1. believes that X is true
  2. X is, in fact, true and
  3. the person who claims to know X can give a justification for thinking that X is true.
but most philosopher says that these conditions are not sufficient. They say justification can be given by appealing to intuition, reasoning or sense experience. Despite the fact that intuition is a common phenomenon, they do not consider as a form of knowledge because there seems to be little way to determine whether it does, in fact, provide knowledge as opposed simply to lucky guesses. So most philosophers consider experience as the basis of knowledge.

When there is too much debate how Epistemology helps in testing? Actually, the basic skills of epistemology are:
  • Ability to ask questions
  • Ability to observe
  • Ability to describe
  • Ability to think critically about what you know
  • Ability to recognize and manage bias
  • Ability to keep thinking despite already knowing.
  • Ability to analyze someone else's thinking.
  • Ability to form and test conjectures.
Oh My God! Did you notice to become a good tester you should also have these skills. Now I know what Cem Kaner, James Bach & Bret Pettichord wanted to say.

All good testers practice Epistemology. Do You?


References:
1. Wikipedia
2. Software Testing - A Practical Approach (Writer - Unknown to me)
3. Lessons Learned in Software Testing - Cem Kaner, James Bach & Bret Pettichord

Saturday, October 25, 2008

Hello Testing Testing...... but what is Testing?

The first definition of Testing which I mugged was
"Testing is the process of executing a program with the intent of finding defects in the program."
This was the time when I was looking for a job. Whenever any interviewer asked me "Okay Mohit, what is testing?" - I just vomit out the same. During my job I came to know Testing is not only finding defects, it is much more than that and then i started to study the writings of Testing experts. The experts has left the treasure of their knowledge on net, more you dig more you find. There are many ways to define testing properly. My favorites are -

Testing is:
"A technical investigation of the product under test conducted to provide stakeholders with quaility related Information." - CEM KANER
James Bach reworded the definition. According to him, Testing is
"Questioning a product in order to evaluate it."
There is no difference in the two definition except prior is explicit and later is shorter.

The most interesting definition which I found is also given by James Bach. He says

"Testing is the INFINITE process of comparing the INVISIBLE to AMBIGUOUS so as to avoid UNTHINKABLE happening to the ANONYMOUS."
Cem Kaner, James Bach and Bret Pettichord says in their book "Lessons Learned in Software Testing" that Testing is done to find the information. Now I know What they really mean.

Everyone can define testing in his context but it is up to you what do you think? You can adopt anyone of the above or if you want, you can make your own. Now I define testing as

"The process to know your product better."
What is your definition of Testing? 

Wednesday, October 22, 2008

There's a little bit of TESTING in everybody's daily life

This is my first post and therefore I was thinking what should I write which must be interesting and read worthy. I don't know whether I will be succeed or not but let me try:

Okay, Today I was going from market when I saw a hoarding of SAIL (Steel Authority of India) which says "There's a little bit of SAIL in everybody's life" - quite impressive. I was thinking about the slogan and feelings behind it when I notice that it is also true for "Testing". So I decided to use it as topic of my first post, although I have changed it "a little bit" according to my subject.

Now, who will deny or dissatisfy with this phrase - nobody. If there is someone then let me give you some examples -


  • Its Diwali season - although market is down but you (at least we Indians) are doing shopping, buying cloths, new utensils etc. as per our rituals. So when you go to buy a shirt, you definitely check the color, size, brand and price etc. as per your need. You try the shirt in the trial room to see how you look in the shirt or should I say You test the shirt in the trial room to see how you look in the shirt. what's that - It is testing.
  • Okay, take another example - When a bulb is fused in your room, you go to market to buy the bulb. When shopkeeper gives the bulb, you say "Please check it" (why do you not take it without checking -because you believe in testing). If it lightens you take bulb and then handover the money.
  • One more - when your mother cooks food for you she always taste it (testing again...) whether food is as per your taste or not and if not then she make changes accordingly and deliver you the food as per your taste.
So these are the only three examples which I have given here and you may have hundreds of example too because we all believe in Testing and "There's a little bit of TESTING in everybody's daily life".