Rapid Software Testing: Workshop by James Bach – Day 1

This was not an ordinary day, at least for me. You don’t get an opportunity daily to attend a workshop on Rapid Software Testing by James Bach. It’s like learning significance of zero from Aryabhatt, the famous mathematician who invented zero. Thanks to Moolya for providing an opportunity to attend the workshop.

I couldn’t sleep properly previous night and keep checking the time after each 20-30 minutes and finally woke up at 5 AM in the morning which is very rare for me. By the time I reached at the venue, it was 7:40 AM and few Moolyavans were already there. At sharp 9 AM the workshop was started and no prize for guessing that it started with thought provoking exercises in James Bach style. Truly said, the exercises picked by James for testers are heart of the workshop. I loved every single exercise and learned what I need to work on to become a credible tester.

There were many things taught by James in workshop and one of them is Sympathetic Testing which took my attention– An approach to test the application to get familiar with it, appreciating the efforts of the developer and ignoring the bugs temporarily unless something is really serious before approaching to real testing. In evening, when I was recalling the things, I realized that I was doing Sympathetic testing for every project without knowing the name. Although, my approach differs to some extent what explained by James. When I get a new application to test I talk to developer and start a tour to learn the application. During the application tour, I note down all probable bugs on my notepad to replicate them later. If I don’t find any blocker I give my feedback to developer about the application without mentioning any bugs. Once I learn application, I start aggressive form of testing. While analysing Sympathetic testing I concluded that it is special form of Sanity Testing. To confirm I must need to talk to James.

Overall, the day was amazing and full of learning. Now, I am desperately waiting for second day of the workshop.

Thanks James Bach

Thanks Moolya.

Why This Kolaveri?

One of the most innovative discovery of 20th Century is the world wide web and by the starting of 21st century, means in the era we are living, it has become life line of most of the business and non business activities. Statistics says that number of internet users  in 2012 has increased by more than 500% from the users in year 2000. If I take my example, I was not using internet in 2000 but now, most of the time, I am connected to internet. There are multiple things which I do regularly on web:

• Earning - My job is totally dependent on Internet.
• Mails/Chat – All business and personal communications
• Connecting with friends through different Social Networking
• News
• Entertainment – Music, Movies, Games
• Study and blogging
• Shopping
• Travel planning and bookings
• Bank Transactions
• etc.

The list is long and I am sure most of you are also using the web for same or different purposes. So, We live in a world which runs over cloud. Imagine a day when you wake up in the morning to find out that you can’t use the web and the reason is neither technical nor electric but physical and you can’t do anything to fix this. What will you do?

I have a story before coming to the main point.

Once Upon a time: Thousands year ago, there was a blind king named Dhritarashtra who is father of Kauravas, his hundred sons. Then, the famous war of Mahabharata between Pandavas and Kauravas started at Kurukshetra. As the king was blind he couldn’t go to war but being a father and a king he wanted to get every news from the war field. Dhritrarashtra had an advisor and charioteer named Sanjaya, who had the gift of seeing events at a distance granted by the sage Vyasa, he narrated to Dhritarshtra the action in the climactic battle of Kurukshetra. So, with the help of Sanjaya, Dhritarshtra could get update of every single second of war field.

That was a different time but need of Sanjaya is still there. Looking at the dependency on web, it could be quite difficult to survive without Sanjaya but having Sanjaya as assistant could be quite costly so we need an alternative in form of web accessibility.

Web Accessibility: For me, Web accessibility is about making the web possible for everybody – people with all abilities and disabilities. There are multiple kinds of disabilities which Web accessibility should address:

• Visual
• Motor/Mobility
• Auditory
• Cognitive/Intellectual
• Seizures

So, we have an option to survive provided web accessibility but unfortunately, most of the websites and web software available have accessibility barriers which make it difficult or impossible for people with special abilities or disabilities to use the Web.

Why This Kolaveri? – Developing an accessible website is not a rocket science; the only reason which has built up this barrier is Ignorance. Most business people claim that their sites aren't used by people with disabilities, so they don't need to pay attention to web accessibility but this is most ridiculous claim ever made and there is no basis of this claim. When constitution of our country gives the right of equality then who are me, you and anyone else to take that away.

The other crucial reason is lack of awareness, the same reason which prompted me to write this blog post.

Here is few statistics data which I collected from various websites. This data should be an eye opener for all those business people who only trust numbers. Below are the numbers of incidence of disabilities recorded by government agencies in different countries

1. New Zealand, 20% of the population (Statistics New Zealand, 2001)
2. United States, 19.3% of the population (US Census Bureau, 2000)
3. United Kingdom, 18% of the population (National Statistics, 2001)
4. Australia, 17% of the population (Australian Bureau of Statistics, 2003)
5. Canada, 12.1% of the population (Statistics Canada, 2001)
6. European Union, across the 15 EU countries in 2001, 19.3% of the population was hampered by physical or mental health problem, illness or disability, with 9.3% severely hampered. (Eurostat, 2003)
7. One in every two Americans of 65 and above has a disability.
8. Nearly 30% of all families in the U.S. are affected by a member who has a disability.
9. People with disabilities boast a population of 54 million and have a discretionary income of more than $175 billion – that’s nearly twice the buying power of the teen market

Why Stop This Kolaveri?: If the statistics data provided could not change your mind, I have other reasons for you to stop this Kolaveri.

• Social Responsibility: Humanity is one of the qualities which keep us human above anyone else. As a human being we should make sure to provide a better and independent life to those who need it. For web users, we can do it by providing the accessibility on our website.
• Grow your Business: More users mean more business and providing web accessibility can bring more users on your website.
• Save your Ass: If above two reasons are not good enough; that only means you are one of the tough nuts to crack and we have one fool proof reason for you: Most of the countries has some standards to regulate the web accessibility. If you are working from such country – better go for web accessibility rather than getting screwed your ass someday.

A single blog post is not sufficient for this topic so I will be publishing some more blog posts on accessibility testing in coming days. Please let me know your feedback and suggestions on the same by yours valuable comments. Don’t forget to share if you liked it.

Come, Let’s make web a place accessible to all.

References:

• http://www.internetworldstats.com/stats.htm
• http://www.usability.com.au/resources/statistics.cfm
• www.wright.edu/web/access

QTP Challenge

Earlier, This blog post was supposed to explain how QTP identifies the objects but when, for curiosity, I typed in Google, I found 352 and more sites explaining the same. Some of them are so good that I didn’t find it useful to explain the same thing once again. It seems everybody knows how QTP identifies the objects. Still, if you have any concerned question, let me know by posting your questions in comment section.

So, when the initial idea was dropped, I thought to put a challenge on Object Identification by QTP. The challenge is very simple or at least it seems very simple but none of the QTP engineers I know could tell me the correct answer. This might be due to multiple reasons like:

• They were too busy with their other daily work challenges
• They didn’t find the challenge interesting enough
• They didn’t want to spend time on something irrelevant of their daily work
• They didn’t have tendency to explore about the new things
• They didn’t have enough courage to take the challenges
• etc. 

If anyone of you have one of the same reasons, this post is not for you. You can leave the post here and can continue working on something else. Rest of you, please continue with the challenge given below.

Challenge:

Their is a login page which have two fields: Login and Password. I have made some small changes in the code so that both objects have the same properties. Once, it is done I have added both these objects in the object repository and set the same properties for both of them. Now, the situation is:

• Both of the objects have the same properties (except Logical Name which is not required for object identification)
• Ordinal Identifier is set to none
• Smart Identification is also disabled

Now, when I highlight, the QTP identifies the object correctly. The question here is how QTP is recognizing the objects correctly when every property of the two objects are same. Shouldn’t QTP display an error of existing of more than one object with same properties.

Download the Login Page and Object Repository for the reference.

Let me and others know if you have the answer or Please explore if you want to find out the answer yourself otherwise wait till someone post the correct answer. Please post your questions and answers as comments to this post.

(Credits to http://www.authpro.com/ for the login page)

Automation Architecture using QTP

For last few months, I have been working on QTP. In these months I have automated few applications with QTP and now I think the time has come to share my learning on QTP with others.  From now on, I will write the blog posts on QTP on regular basis.

Today, I am starting with Automation Architecture (framework) using QTP. Automation Architecture can be well understood by the following diagram:

AUT: AUT means “Application under Test” i.e. the application which is being automated in this case.

Object Repository: As name suggests it is repository of the objects from AUT. QTP recognizes the objects in the application through this object Repository. We will see how QTP recognizes the objects in the next blog post.

General User Library: General User Library aka “GUL” is the function library which has the common functions written in it. These functions can be used with any AUT.

Business User Library: Business User Library aka “BUL” is the function library which has Application specific functions.

Test Data: Test Data is the data required for testing. It could be anything like data table, excel, XML or CSV file.

Environment Variables: The global variables which are used by the Driver Script.

Recovery Scenarios: Recovery Scenarios are written to manage the unexpected error during execution.

Driver Scripts: This is main script which drives the automation. This is where the whole logic is written.

Test Execution Report: This report displays the script execution result either pass or fail.

I hope the above description will help you to understand the Automation Architecture using QTP. Let me know if there is any question on the same.
========================================================================================
Update on 12-06-2012: The title of the post and image is updated after getting the review from Lakshminarasimha M. and Sadhanandhan B. Thanks for their valuable feedback.

Reviewing qTrace: An Heuristic Approach

Few days back I came to know about a tool called qTrace from QASymphony. I am sure most of you must have heard about this tool but if you are one of those who missed it for some reason then this blog post is definitely for you. This post can also help you if you are testing qTrace or any other similar tool.

When I came to know about the qTrace I found it interesting and thought to try it. I downloaded the free trial version from here and installed it on my machine. The trial version of qTrace is valid up to 30 days so I always knew that I would definitely get  some time to review qTrace.

To know the tool better I started with the Touring Heuristic FCCCUTSVIDS by Michael D Kelly. The following report is based on my findings by using FCCCUTSVIDS heuristic on qTrace. This report gives the idea what could be tested in qTrace or any other similar application.

Features: I started exploring qTrace for the features. Few of the features which I found include:

• qTrace captures the screen of the selected application. User can take the single picture or can record the whole session of testing. qTrace automatically records the testing steps.
• User can also record the notes while testing the application. It’s a very good feature which can be used smartly while doing the exploratory testing
• Add the Environment Information of the bug into bug report without any effort. qTrace records the environment information and avails it on a single click.
• User can edit the captured screenshots and annotate them with important information.
• Submit the bug reports directly to associated bug Tracker. qTrace can associate with major bug tracking tools like Bugzilla, Jira, HP Quality Centre etc.
• Email the bug reports to the concerned person.
• Bug Reports can be saved in famous formats like Word, PDF & JPG format

Complexity: qTrace is very user friendly and easy to use. The most complex features which I would like to test thoroughly are as follows:

• Association with different Bug Trackers.
• To handle Large size Bug Reports can be a challenge for qTrace. It’s a region where we can find the issues
• qTrace can be used on three different machines with same licence.
• Emailing the bug reports
• Recording the testing steps with important notes.

Claims: The best place to find the claims about a product is to search the product site or help documents so I used the both of them. Following are the claims which I found about the product on the product site and help documents:

Claims	My Views
qTrace is a software testing and reporting tool that makes quality control simpler, faster, and more seamless. With qTrace you can easily record defects, edit details, and share results.	Agree with most of the claims. Not Agree with the Quality Control claim.
qTrace is for anyone looking to improve defect reporting, boost productivity, and speed quality control cycles to help bring products to market more quickly.	Agree with improve defect reporting claim. Partially Agree with the boost productivity claims and not agree with the Quality Control claim.
qTrace helps you and your team be more productive. It streamlines defect documentation while seamlessly meshing with your existing tools and processes.	Partially Agree
Automatically documents every screen and every step of a defect or test case	Agree
Capture all defect information to document an issue and submit a complete bug report directly to Jira, Bugzilla, HP QC, Team Foundation Server, VersionOne or qTest.	Agree
Seamlessly integrates with leading defect tracking tools	Partially Agree
Simplify defect reporting	Agree
Point-and-click to start recordings	Agree
Publish reports with screen shots, user actions and system specifics	Agree
qTrace is Simple and Intuitive to Use	Agree
qTrace Makes Documenting Defects a Breeze	Agree
qTrace Complements Your Existing Tools	Agree
qTrace Eliminates the Drudgery of Testing	Not Agree
qTrace Helps Your Team Be More Productive	Partially Agree

Configuration: I could find only following two ways to change the configuration of qTrace.

• Settings in the Application
• qTrace.exe.config file in Program Files

Let me know if there is any other way to change the configuration.

Users: I could imagine the following users (&groups) who could use the qTrace or qTrace outcome directly or indirectly.

• Software Tester
• Software Developer
• Test Manager/ Project Manager
• IT Admin
• Top Management
• Client

The scenarios for each user is described below in Scenario section

Testability: Testability tour helped me to find out the testable feature of the qTrace. Major testable features are listed below:

• Capturing the Screenshots
• Automatic recording of the testing steps
• Adding the notes
• Submitting the bug report to Bug Tracker
• Emailing the bug report
• Configuration Settings
• Opening/Editing the .trace file
• Annotating the reports
• Association with different Bug Tracking Tools
• Adding/Editing/Deleting/Moving the Steps
• Saving Bug Report in different formats

Scenarios: I found the following scenarios for the users I found during the User Tour.

• Software Tester: A software tester can use qTrace for testing the application. During testing he would like to use it for recording the testing steps, taking the notes, taking the screenshot of the bugs, submitting the bugs, emailing the bugs etc.
• Software Developer: A Software Developer is the one who is going to fix the bug so it is very important that recorded bugs are well described including the steps, screenshots and environment. All these information will help a developer to fix the bugs quickly. So while testing the qTrace it is necessary to know the Developer’s perspective on the bug reports.
• Test Manager/Project Manager: Test Managers/Project Managers are the one who are more concerned about the outcome and productivity. There is no doubt qTrace helps testers to generate descriptive bug reports with all the supporting screenshots and reproducible steps so overall it will definitely help their productivity with the quality of the bug reports
• IT Admin: You must be wondering how qTrace can be used IT Admin. The IT Admin is one who takes care of your machine and application installed on it. From IT Admin’s perspective following features should be available with the qTrace:
  • IT Admin may want to associate the qTest with the in-house bug tracking tool or any other commercial bug tracking tool being used in the organization. There should be an option to associate the qTrace with the in-house bug Tracking Tools.
  • Concurrent Licence similar to QTP Concurrent Licence. Managing 200 or 300 licences could be a tedious job. Having the concept of Concurrent licence will ease his job.Right now, maximum three users can use qTrace with same credentials and Licence
  • Every BT has size limit for the files being attached. IT Admin would like to Define the Maximum File Size of the attachments as per the limitation of associated Bug Tracking Tool and Email Client
• Top Management: The top management is always worried about the money. In qTrace reference they must want to know the cost about the tool. So, for there reference qTrace is not the expensive tool. The cost is very reasonable and can easily bear by the management.
• Client: Some clients want to see the issues, especially, if testing is outsourced. A good bug report gives a good impression about the testers and their organization and qTrace can help testers to generate a very good bug report.

Variability: The Variability tour is about the things which we can change in the application. Following are the things which I found testable based on the Variability Tour.

• Default Save Location
• Attachment format for email and BT
• Image Quality
• Annotation Font, Size and Color
• Defect Tracker
• Environment

Interoperability: qTrace interacts or might interact with the following applications:

• Windows Environment
• Microsoft Outlook
• Configured Bug Tracker

Let me know If I am missing any application.

Data: What could be the input data for an image grabber tool? I could find the following data which can be an input for qTrace.

• Recording Environment
• Captured Images ( for annotation of captured images)
• Input Data for eg: Text, Numeral etc. for Notes, Title, Bug Description
• Bug Reports (which will be send to Bug Tracker and Email)

Structure: Structure Tour couldn’t help me much in this context as I have only exe file of the application.

My Recommendation: Although I just had a tour of qTrace, actual testing is still pending but after touring it I can confirm one thing: This tool is a boon for software testers. Bug Reporting is an area of testing  process where a tester spends a lot of time, especially, if we are talking about good bug reports. qTrace saves a lot of bug reporting time. It prepares well descriptive bug reports for the testers which can be submitted with small changes in it. Sometime, no change is required. I would like to add this tool in my testing armour.

Things I would like to have in qTrace someday:

• Association with In-house bug tracking tool. Also, Mantis BT is one of most used bug tracking tools. I am wondering why qTrace is not allowed to associate with the Mantis BT.
• Video Recording of the bugs

Please comment if you have any question or any suggestion.