Continuous Integration for QA

The Agile practice has completely changed the QA approach. Sprint iterations brought new practices to deliver the software quickly and with trusted quality. With the given limited time and dynamically  changing environment challenges the QA. Continuous integration practice  is the blessing in disguise.

Continuous integration enforces a discipline to integrate the tests more frequently and offers the benefits like reduced manual effort of running tests, easier trouble shooting, produces deployable working software at any point of time. It also helps to get the confidence in the software, provides better visibility of software health and transparency.

 

Continuous integration tools are evolved over the time. In the initial days, I remember using Cruise Control. Cruise Control had its own limitations like not able to detect the hanging builds, showing the test status in live. But the next generation tools are evolved with many features incorporated and fulfills all needs of CI users. I personally admired by TeamCity, as it offers great set of features and provides many build runners like command line runners, Junit, NUnit etc. In the recent past Jenkins is gaining momentum due to lot of features for free of use!

The general approach to hook in your tests into continuous integration tool is:

• Create a CI QA project in CI tool.
• Configure the source control in your QA project.
• Configure list of agent machines in the CI pool to run the test.
•  Choose the build runner. For example, if your tests are written in Junit use a Junit build runner. It can be NUnit or CLI runner depends on your need.
•  Create an ANT/MSBuild script and add this as a driver script in your CI QA project.
• Create a run decency or build chain dependency.
• Create publish rules.

Importance of automated tests in Continuous Integration

In today's business world, it has become a norm to work as a distributed team. If we take the example of OpenEdge development in Progress Software (The Company i work for), OpenEdge product contains multiple components like a language, database, servers, IDE, etc.. The development of these components is done using multiple technologies like C, CPP, Java, ABL, .net, etc.. Along with that, the development of this product is distributed among different teams across the world. So, application development environment might have multiple teams, multiple technologies involved. Development happens at one place and testing at other. So, it is always difficult to catch right person at right time.

In addition to distributed teams, now a days with AGILE development everywhere, applications are being released in a very short time to market. With this, development is being done iteratively and for each iteration whole Software Development Lifecyle needs to be completed within that short cycles.

QA face difficulties to keep up with these fast releases, since they need to complete whole regression testing of existing functionality along with the new feature testing. Along with the functional testing, they may need to do load testing, performance and system testing.

Continuous integration is a good way to minimize regression testing. In order for Continuous integration to succeed, QA relies heavily on individuals writing scripts for tools. The scripts need to be written not only to test the feature being implemented, but the feature we delivered a year ago still has to work.

Continuously integrated builds undergo automated testing whenever a developer pushes code to a repository, but running larger tests at specific times is still valuable. During a nightly build, run a full site or application load test for whatever we expect the user base to be at any given time. Then, toward the end of an iteration or sprint, stress the application to its breaking point to set a new bar for how many concurrent users it can handle.

So, including all our automated test suites to Continuous integration increases QA's confidence.