? QA Design Gurus: April 2015

Apr 30, 2015

End to End Testing

What is End to End Testing
End-to-end testing is a technique used to test whether the flow of an application right from start to finish is behaving as expected.
It is also called as Gray box testing as it is combination of both White box testing and Black box testing.

Apr 29, 2015

Choosing right Automation Tool and Framework for Agile Process

In last topic I  have discussed about major QA challenges in Agile Process. Now in this read up I am going to share about the factors to be considered while choosing automation tool in agile process.

Based on the challenges mentioned in previous post, QA start thinking of automation. If we have proper automation,we can overcome most of the challenges. Also if the chosen automation tool does not meet QA requirements then it would be really painful and instead of we achieving solution for problems, we will be just spending time in fixing the automation scripts
issues or just keep the scripts aside and do manual testing. So finding the right agile automation tool is therefore incredibly important if agile teams are to have a bug-free version of the software with each iteration.

Now another biggest challenge is which tool you have to choose for automation and on what parameters.

QA challenges in Agile process:

In this fast changing world, software projects using agile methodology for the delivery are becoming common. Agile software development is all about developing working increments of the software and releasing it. Agile testing is a software testing practice that follows the principles of Agile Software Development. In this read up I have covered some of the major challenges the agile testing team is facing
  1. Constantly changing requirements - Although agile testing welcomes last minute changes to the application, however, at times it becomes difficult for the testers to accommodate the changes, especially with the deadline is approaching and when it cannot be postponed. 

Apr 28, 2015

Localization & Internationalization Testing

What is Localization?
Localization refers to the adaptation of a product, application or document content to meet the language, cultural and other requirements of a specific target market (a locale).
Localization is sometimes written as L10N, where 10 is the number of characters between 'l' and 'n'

What is Localization Testing?
Localization testing is testing process to validate whether application is capable enough for using in a particular location or country. 

It can be related to:
Numeric, date and time formats
Use of currency
Keyboard usage
Collation and sorting
Symbols, icons and colors
Text and graphics containing references to objects

Apr 13, 2015

How we tested a critical feature of cloud platform with zero customer defects


We got a critical(architecture level change ) feature for testing. We followed a different approach to testing this feature and we achieved zero customer defects.

Why the different approach?

Our Cloud platform is integrated with many other cloud products.We use an integrated environment for regular testing. If we directly test on the regular integrated environment then all other product teams may get disturbed if we have any critical bugs. Instead of wasting other team members time we followed a different approach.

Steps followed:


1) Dev Testing

QA provided automation tests to Developer. Developer ran all automation tests on dev environment

and fixed the defects. Provided a good stable build to QA.

2) Regular QA build testing

QA did a full round of testing on regular QA build. Ran all automation suites like Manual tests, API tests, UI tests.. etc

Verified all existing defects.

3) Testing on Cluster setup

QA did a full round of testing on Cluster setup. Ran all test suites on cluster setup.

4) Testing on Customer environment 

We simulated a customer environment. The customer who is going to get benefited more with this feature. We executed customer-specific tests.

5) Testing on Integrated environment

QA did a full round of testing. Executed all automation tests. Did not find any integration defects

6) Testing on Stage-prod

QA did sanity test of all features.


Finally, we released the feature and no defects reported by customers.



Apr 11, 2015

Re Run Failed Tests automatically |TestNG



We are using TestNG framework to manage and execute UI Tests. UI Tests might failed due to different reasons like connection timed out/browser hang/Network Problem...etc. This kind of issues is not related to Application under Test. In this case, we should re run the tests once again.
TestNG is providing default feature called Retry Analyzer which will re execute test case in case of failure.  TestNG Provides interface called IRetryAnalyzer with method called Retry. We need to override this method as per our requirement.
Example:
1| Assume you have a class file with 10 Methods [@Test]
2| Lets say one got failed due to Network issue
3| The class file, RetryAnalyzer will make the failed test-case to re-run with desired count; But, it display errors for all the test failures during re-run. [Explained in this post]
4| you are ought to add a Listener class, RetryTestListener; it will print the failure only once.
5| If the test got passed in the final count, failure exception won't be thrown in the console.
Test Class
@BeforeTest
public void setUp()
 {
System.out.println("This is Setup Method");
 }

@Test(retryAnalyzer=RetryAnalyzer.class)
public void test01() throws Exception
 {
        // Assert.fail();
 Assert.assertEquals("Failed", "Passed");
 }
RetryAnalyzer.java
import org.apache.log4j.Logger;
import org.apache.log4j.PropertyConfigurator;
import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;

public class RetryAnalyzer implements IRetryAnalyzer  {
private int count = 0;
private int maxCount = 4; // set your count to re-run test
protected Logger log;
private static Logger testbaseLog;

static {
    PropertyConfigurator.configure("test-config/log4j.properties");
    testbaseLog = Logger.getLogger("TestclassName");
}

public RetryAnalyzer()
{
    testbaseLog.trace( " ModeledRetryAnalyzer constructor " + this.getClass().getName() );
    log = Logger.getLogger("TestclassName");
}

@Override
public boolean retry(ITestResult result) {
    testbaseLog.trace("running retry logic for  '"
            + result.getName()
            + "' on class " + this.getClass().getName() );
        if(count < maxCount) {                    
                count++;                                   
                return true;
        }
        return false;
}
}

The Test Class is having one test method with RetryAnalyzer class which will invoke RetryAnalyzer  whenever the test fails and RetryAnalyzer class re executes the same test method automatically.

Apr 10, 2015

Purpose of Test Strategy

Strategy is - “An idea that guides the design."


Do I need ideas to guide my test design? Yes, if I want to do it well. Actually, in my terms, every tester has a test strategy. The questions are: how well does it work, is the tester aware of it, and can the tester describe it? Is it necessary to document it?
A Test Strategy document outlines the "big picture" - basically it lays out all the big blocks of testing. A very succinct example would be "We will do unit testing, followed by integration and system testing. After those are successful, we move to performance testing, followed by UAT."
Some companies include the “Test Approach” or “Strategy” inside the Test Plan, which is fine and it is usually the case for small projects. However, for larger projects, there is one Test Strategy document and different number of Test Plans for each phase or level of testing.


Here are few reasons for writing test strategy
  • The number one reason of writing a test strategy document is to "have" an approved document, which includes a written testing methodology, tools to be used and defines exit criteria.
  • The test strategy document tells us how the software product will be tested.
  • The creation of a test strategy document presents an opportunity to review the test plan with the project team.
  • The test strategy document describes the roles, responsibilities, and the resources required for the test and schedule constraints.
  • When we create a test strategy document, we have to put into writing any testing issues requiring resolution (and usually this means additional negotiation at the project management level).
  • The test strategy is decided first, before lower level decisions are made on the test plan, test design, and other testing issues.
  • Finally, having a test strategy does satisfy one important step in the software testing process. 

Apr 7, 2015

Art of writing a good bug report

Why good bug report?
One of the important deliverable in software testing is bug report. Writing a good bug report is an important skill of a software tester. If your bug report is effective, chances are higher that it won't be ignored by the product owner and developer. So fixing a bug depends on how effectively you report it.

What are the qualities of a good software bug report?
A bug report should have the following:

A clear title:
A good title helps reduce duplicate issues and accurately summarize the issue.
Include the specific name of the components involved in your bug in the title.

Steps to reproduce:
You should clearly mention the steps to reproduce for the bug. It should contain all the details of components involved with environment details.
It should contain information as below-
Product: In which product you found this bug.
Component: These are the major sub modules of the product.
Version: The product version.
Platform: Mention the hardware platform where you found this bug.
Operating system: Mention all operating systems where you found the bug.
Priority:When bug should be fixed? Priority tells the importance of the bug.
Severity:This describes the impact of the bug.
Summary:This portion contains a brief summary of the bug. Make sure your summary is reflecting what the problem is and where it is.

Apr 3, 2015

How do we measure product/project's health? - Importance of BugReports

Is it based on the no of bugs existing in your product? If so, how can we prepare our product to make it bug free or releasable.

Do you collect bug reports for you project? Then how often do you do?

Bug reports are important for any software development. It is a good practice to collect bug reports. In our company, this is one of the good practices we follow, collecting bug reports.

Collecting bug reports helps both development and quality teams to track down and fix bugs in product. These reports improves the quality of final product. They helps to prepare product launch. Until all the P0, P1 and P2 defects are fixed, the project is not ready to ship.

The analysis of the data collected helps in understanding more about a particular set of problems or tests. Also helps in understanding and reporting the overall level of quality.

Finally, defect reports, when analyzed over a product/project and even across projects, give information that can lead to development and test process improvements.


How to write JavaScript in Ant tool

Below are the some examples lets you understand to write Java Script in ANT Tool.

Ex: To Calculate how much time its taken to run the target .

Step1: Calculate Start Time before executing target

Get the System Date and convert into Simple Date Format

<script language="javascript">
<![CDATA[

           importClass(java.text.SimpleDateFormat);
         

           fmt = new SimpleDateFormat("MM/dd/YYYY HH:mm:ss a");

           self.getProject().setProperty('startdateformat', fmt.format(new Date().getTime()));

       ]]>

</script>
<echo> time is ${startdateformat}</echo>
<echo> time is ${startTime}</echo>


Step 2.  Invoke Target. Ex:
<parallel threadcount="3">
<echo message="UseCase1" />
<echo message="UseCase2" />
<echo message="UseCase3" />
</parallel>

Step 3. Now Calculate End Time and Elapsed Time.
<script language="javascript">

<![CDATA[

importClass(java.text.SimpleDateFormat);
   fmt = new SimpleDateFormat("MM/dd/YYYY HH:mm:ss a");
self.getProject().setProperty('enddateformat', fmt.format(new Date().getTime()));
 project.setProperty('elapsedTime', new Date().getTime() - startTime)  ]]> </script>  
<echo>End Date format : ${enddateformat} ms</echo>
<echo>End time: ${endTime} ms</echo>


Here Elapsed Time Calcuated in Milli Seconds. Inorder to convert milliseconds to Seconds use below snippit in Ant.

<math result="result" operand1="${elapsedTime}" operation="/" operand2="1000" datatype="double"/>
<echo>result is ${result}</echo>  




Apr 2, 2015

How to invoke RESTful webservice from ANT script

We got a requirement like we need to send automation results to Cloud Platform www.rollbase.com.

We used HTTP Post ANT task to send automation results to www.rollbase.com.

We can invoke REST service from ANT script using POST ANT task.

Example:

    <property name="test.val" value="here's my test value"/>
    <property name="test.val2" value="second test value"/>
    <post to="http://wwwj.cs.unc.edu:8888/tang/servlet/tangGetPostServlet"
        verbose="true">
        <prop name="prop1" value="val1 ${test.val}"/>
        <prop name="prop2" value="val1 value 2"/>
        <prop name="prop3" value="val got some spaces %funky ^$* chars"/>
        <prop name="prop4" value="&amp; do an ampersand like this &amp;amp; or
        Ant will whine"/>
        <prop name="thanks" value="dude, thanks for the echo server!"/>
        <prop name="test.val"/>
        ${test.val2}
    </post>


Prerequisites:
  1. Copy ant-contrib-0.3.jar to the lib directory of your Ant installation. If you want to use one of the tasks in your own project, add the lines
    <taskdef resource="net/sf/antcontrib/antcontrib.properties"/>
    
    to your build file.
  2. Keep ant-contrib-0.3.jar in a separate location. You now have to tell Ant explicitly where to find it (say in /usr/share/java/lib):
    <taskdef resource="net/sf/antcontrib/antcontrib.properties">
      <classpath>
        <pathelement location="/usr/share/java/lib/ant-contrib-0.3.jar"/>
      </classpath>
    </taskdef>