? QA Design Gurus

Mar 27, 2015

Memory Utilization - Am I seeing it right

While doing Performance testing, in addition to response times we need to understand how much OS-Resource our system consumes. OS-Resource might involve monitoring things like Memory,CPU and Network I/O. While working on a Multi-process or Multi-threaded applications and comparing memory consumption among them, it is very important to know how one calculates the memory.

               There are various ways to look at the memory of a process. Let’s look at a on-premise machine (Cent-OS) to look at the memory of a process


              
On using a “top” a linux based tool, I got the above memory snapshot for different “top” processes. So, which one should one consider for memory?

VIRT – virtual size of a process where it holds the memory within its process, memory shared with other processes and shared libraries.

RES – residential size of process which will accurately provide the physical size of the defined process (Code + Data)

SHR – Shared memory indicates the memory consumed by the shared libraries which are indicated in the virtual size of the process.

Now at a glance, If I need to understand the memory consumed by a process I would go  for the “Residential size”  of the process. Residential size actually gives us the accurate memory consumption of the process.

Now let’s say that you would like to monitor or look for “Core Memory” of a process. When running applications in Production, you might want to look at how much memory does the process itself takes (which does not include any memory of the shared libraries which might also be related to memory shared with other processes). So to calculate that you might need to look for the below formulae
                 
                    Core-Memory = RES – SHR.
  

The same kind of convention applies for windows too. There we call it as working set and private working set. You can configure your “task manager” to show which ever kind of memory(by default it shows private working set). If you wish to do it through script then you can use the windows API “WMIObject”.

Here’s a snippet

Get-WmiObject -class Win32_PerfFormattedData_PerfProc_Process | where{$_.idprocess -eq $processId} | select `
 @{Name="Process Id"; Expression = {$_.idprocess}},`
 @{Name="Counter Name"; Expression = {$_.name}},`
@{Name="CPU Usage"; Expression = {$_.PercentProcessorTime}},`
@{Name="Private Working Set"; Expression = {$_.workingSetPrivate / 1mb}}   


This should help you understand how much memory is your process used and which one adds significance

Node.js ! What is it for testing

Node.js has become extremely popular for its ability to run javascript in any platform with a Google chrome runtime engine for windows,unix and mac platforms, moreover as a server. Most important features that makes node.js adaptable are

1. Non-blocking event driven approach for handling requests(less CPU Intensive)
2. Two way HTTP communication
3. Amazing source-control mechanism

In addtion to that, it is all about javascript. So now, what is it for testing and where can we use it.

Node.js for testing javascript
In the Agile world, changes happen very frequently and rapidly. To keep upto the speed and validate the functionality instead of the UI changes one can just run the javascript over node.js on only platform. Instead of using heavy UI tools,one can use node.js which is sufficient and leight weight to execute and validate the javascript functionality.

Node.js for API  and BaaS testing
While working on a heterogenous applications to make everything consumable across different applications/languages they have to be exposed as services, that is where we have REST and SOAP API's. Executing REST API's are very easy using node.js and at the end it gives json which does not need to parsed and straight away displayed in the browser if we would like to look it in the browser elements. In addition to REST, node.js provides few libraries to even test SOAP API's like node-soap. The same applies for backend services which might be either REST or SOAP API's.

Node.js in Performance testing
While doing Performance testing, node.js can be developed as a driver to execute,monitor and display the results. Most of you have might seen the sample chat application that one can develop using node.js. We can take the advantage of node.js of two HTTP Communication in displaying the response times/execution times or OS resources data in graphs immediately.


Rather developing applications, hosting and maintaing them is very difficult. Progress helps you do that much easily with Modulus. Progress Modulus help's you deploy and run your node.js application on the cloud or on-premise.
In addition to that Progress Modulus provides detailed Performance metrics of the node.js application and a high speed load balancer(faster than nginx).

With all these tools, I hope it will be fun to develop apps and test using node.js

The Regression Pressure

“Product release is one sprint ahead, we are in a hurry to complete the regression to ensure we are not delivering a broken product which could lose a customer’s believe on us and in turn effect business.”
Now the case is very similar in all the product releases in almost all the IT product companies who are committed to an on time Quality delivery which helps customer meet his own timelines.

Let’s bring the Pressure down with a story ( 0 point)


A group of managers were given the assignment of measuring the height of a flagpole. So they go out to the flagpole with ladders and tape measures and they’re struggling to get the correct measurement; dropping the tape measures and falling off the ladders.
A tester comes along and sees what they’re trying to do, walks over, pulls down the flagpole, lays it flat, measures it from end to end, gives the measurement to one of the managers and walks away.
After the tester is gone, one manager turns to another and laughs, “Isn’t that just like a tester? We’re looking for the height and he gives us the length.”




I leave the moral of the story to the reader. :-)


Mar 25, 2015

Tuning tips of ATM Performance Testing Suite for OpenEdge DataServer

Introduction

Performance testing is used to determine the speed or effectiveness of a computer, network, product or device. This process can involve quantitative tests done in a lab under ideal conditions, such as measuring the response time or measuring the Transactions per second (TPS) in database at which a system functions.

The ATM benchmark has served as a database performance evaluation tool for many years. ATM application is modified according to DataServer. We are running ATM for Oracle and MSS DataServer  

Database Parameters

Server side Parameters

This should be a test of the database engine, so bottlenecks involving disk I/O, shared memory, and the BI need to be eliminated.  For each machine we assessed the amount of free memory available, and the amount of disk space available.  The following guidelines were used:

Load the entire database into shared memory.  If this is not initially possible, change the SCALE of the database to a smaller number so that the database can totally reside in shared memory.


Mar 24, 2015

Writing and running JMeter tests using Blazemeter

When I was asked to look at scalability testing, I came across an interesting tool, Blazemeter.
Usually, setting up the Load Testing infrastructure, writing automation around it, managing, maintaining and monitoring the load test infrastructure is a headache. We need to spend lot of time in setting up the infrastructure instead of focusing only on load testing.

Blazemeter is a Saas based high scalable load testing tool that can handle up to 3,00,000+ concurrent users. Their load test infrastructure is spread across major AWS regions. BlazeMeter enables us to write load test-scripts not only using JMeter but also using Selenium.

With BlazeMeter, all we need is to write the test-scripts, choose the amount of load-engines and run the test. The system takes care of the everything else. Unlimited number of load-engines are pre-configured and available at your disposal. Detailed graphical reports are generated during the load.

Recording:
The main feature I like in Blazemeter is that, it provides a Chrome Extension which can record browser actions and convert it to JMeter .jmx file. This script will be automatically uploaded to Blazemeter where we can run it using a single click.


Running tests:
For running the tests, we can configure the test time, duration and number of users. We can even schedule the test to run weekly in BlazeMeter. It is an added advantage. The load uses a pre-configured distributed load testing environment. The environment is ready to use and available at all times.
Blazemeter also provides REST APIS. Instead of manually starting the tests, we can make use of these APIs through which we can even run the tests by writing an automated script.

Monitoring:
Blazemeter provides live monitoring when the test is running which enables us to decide on the number and instance type for the test.
While performing load testing, it is important not only to monitor your Web Servers & Databases but also the agents from where the load is generated.

Results:
As soon as the test is completed, results will be sent to configured mail address. The reports will provide us with all the KPIs we need. Reports can be exported to csv or zip formats. We can easily share all the results with our team. Also, reports can be compared

Interesting plugins:
Integration with:

  • NewRelic: Used to monitor the application performance while load tests are being run.
  • Google Analytics: If we select the Google Analytics Option and provide account details of Google Analytics, BlazeMeter obtains the last 12 months of data and creates a test with 5 most visited pages and sets up the number of concurrent users based on that record.
  • JMeter, Teamcity: To include the load tests part of continuous integration.
  • WordPress
  • Drupal

Mar 23, 2015

Security Testing - Opportunities created by Cloud Computing

Security and Security Testing are considered as some of the great challenges with Cloud Computing. In addition to regular security issues, Cloud computing imposes a set of unique security constraints such as  identity federation management testing, multi-tenant penetration testing etc. A summary of various security attacks that need special focus by software testing team is illustrated. 


MSTest Vs NUnit

My experiences while choosing test frame work for automation lead to have below comparison chart.

 
MSTest NUnit
Has fairly good APIs, but not as many provided by Nunit (Refer here ) NUnit has a richer assert API compared to MS Test
MSTest has a provision to execute your test cases parallel way if test is not dependent on other test (Please refer here for details) Parallel execution is not supported
Slow in exeuction, because it creates folder for test run every time Faster compared to MS Test
MSTest took the file based approach with DataSource to provide test data, which is nice but you will always have to need to add csv, excel or xml data file. It doesn't support parametrized testing NUnit does not have a build in support for filebased datasources. It provides ability to write parametrized test cases
Visual studio will give you Code Coverage in MSTest For code coverage, we have to use external tools like DotCover / Ncover
MSTest has no x64 support. Tests always run as x86 NUnit can run tests in x86 and x64
It has ability to determine the order of tests (ordered tests) No ability to determine the order of tests. The order does not follow the lexical order of the attributes and will often vary between different compilers or different versions of the CLR
"AssemblyInitialize, AssemblyCleanup are two special attributes that can be used to bootstrap or teardown your test assembly. Execution flow is TestClass with ClassInitialize –> TestInitialize –> TestMethod –> TestCleanup –> ClassCleanup" "Execution flow is TestFixture with TestFixtureSetup –> Setup –> Test –> TearDown –> TestFixtureTearDown"
MS Test is shipped with Visual studio This is an opensource, Can be integrated with Visual Studio (as plugin), and any other 3rd party tools
Updates only with newer version of Visual studio lots of updates, as it is an opensource
It provides test agent to run the tests without need to have VS dev studio It has an independent test runner

If we need test code to be portable between NUnit and the Visual Studio test systems, giving us the best of both worlds, here is an interesting reference - https://msdn.microsoft.com/en-us/magazine/cc163643.aspx#S4.

 Conclusion cannot be done by giving best framework title to either, as the choice of implementation depends on your requirement.