Want real product installed in test environment instead of simulated product installation?

We test different kind of applications like Console Application,Windows Application & Web application.

To test any type of application we need system run time environment. To run unit tests or system tests we setup a run time environment which is mostly a simulated one not the real customer product run time environment.

There are some advantages with this simulated environment. Setup can be done easily,Debugging may easy, Continuous integration may be done easily.. etc

We can modify this simulated environment to real product environment with Install Automation.

Generally we use InstallShield or InstallAnywhere software to provide installer to customers to install our product. Both supports silent installation.

Silent installation is a installation type. We can install software without UI.

Using following command we can install software without UI

C:\xxxx_vxx.exe -i silent -f installer.properties

How to get install.properties?

First install the product using UI. Normally it asks for installation directory and other required options. After installation done successfully. Move to installed directory and search for install logs.

In one you can find the all user interactions recorded.

Sample installer.properties looks like below


#Destination and Working Directories
#-----------------------------------
USER_INSTALL_DIR=C:\\xxx\\xxx

#Database Type
#-------------
USE_EXISTING_DB=1
INSTALL_NEW_DB=0

#Database
#--------
DB_HOST_NAME=localhost
DB_PORT_NUMBER=9999
DB_USER_NAME=xxxx
DB_PASSWORD=xxxx
DB_NAME=xxx
DB_LOCATION=C:\\xxx\\xxx\\xxx


#Application Server
#------------------
USE_NEW_TOMCAT=1
USE_EXISTNG_TOMCAT=0
TOMCAT_DIR=C:\\xxx\\xxx\\xxx


You can use same installer.properties file to install on all OS platforms like Windows,Linux,Sun Solaris,IBM AIX, HPUX.. etc

Installer UI may not support in platforms like HPUX and IBM AIX. Using this silent install you can install the product easily.

You can use this to install on cloud machines by connecting using Putty.

Need to do large scale email analytics? Don't worry Rollbase can help you

Now a days people are busy. We don't want to use multiple applications.We want every thing on emails. 

For example: Forum Questions.

Employees can log into Forum application and they can reply but employees are happy if they get forum questions via email and they are happy to give reply via email.

Higher management  want every thing in email like weekly status, monthly status, daily discussions...etc

Of course we use rules/filters to move emails to separate folders 

After some years will have lot of emails. If we want generate some analytics based on a component or a word. Rollbase can help you.

Rollbase is a platform as a service (PaaS) software solution. You can get rapid application development in the cloud, with minimal coding. Build, deploy and manage cloud-based applications with a single tool designed to get you up and productive fast.

Rollbase provides API to read emails. You can write your own logic in JavaScript

http://documentation.progress.com/output/Rollbase/index.html#page/rb/email-api.html

Example

rbv_api.openPOP3("mail.mydomain.com", 995, "address@mydomain.com", password, null);
var arr = rbv_api.getMailMessages(100, 110);
for (var k=0; k<arr.length; k++) {
var m = arr[k];
rbv_api.println(m.get('subject'));
}
rbv_api.closeMailSession();

After writing logic to filter out emails using Rollbase other APIs you can create records in Rollbase system. Once you get data in Rollbase then you can utilize Rollbase features like Charts, Reports.

You can use batch jobs to run your logic daily and get live email analytics

Finally you can get analyzed report as a email 

Risk based Security Testing in Multi Tenant Cloud Application

Risk based Security Testing in Multi Tenant Cloud Application:

Despite of extensive Functional testing and Security, we see many instances of software when attacked or during normal operations, performing adversely in ways that were not anticipated. In large part, this is due to not testing fully for negative functionality, ensuring that applications do not do what they are not supposed to do. Most of the times QA might not give importance to negative testing due to spending much time on functional testing and pressure to complete testing on time.

In Multi tenant application, It is verify important that QA needs to ensure that data in application is secure and safer either during functional testing or negative testing. For example, User Management is a typical feature in any multi tenant

application. In Multi Tenant Application, Users from different tenants will use same application and database without knowing each others. QA should ensure that customer data should be safe and secure during functional testing. In this blog, we can explain few scenarios how can QA identify the security holes during functional testing in Multi Tenant Applications.

To get the security loop holes in the Product, QA can get the required information about the product/Customers using below steps 

1.  Logged into the system using Chrome/Firefox browser.
2. Do  start functional testing in UI and Preserve the network calls in browser using developer tools in browser.(Press F12 key to enable this)
3.  Observe Each and every network call for every UI operation to get the details of Rest API which will be invoked at back-end 
4.  Go through the Request headers and Responses for each Request to get the information about what kind of authorization is required to invoke the back end service directly.

QA will get enough information for each back end API during functional testing by following above steps. Now QA can try the following scenarios as part of Security Testing for typical UserManagment feature using Post Man by invoking directly back-end calls. There are chances of handling the following use cases in UI level but not at back end service level. 

1.  Logged into the system as a Tenant A’s user with admin role and Try to add a user in another Tenant B using add user API using post man.
2. Logged into the system as a Tenant A’s user with admin role and Try to get the user details of the Tenant B using get users API using post man.
3.  Logged into the System as a Tenant A’s user without admin role and try to manipulate the user details of same tenant and other tenant using edit user API using post man
4. Logged into the system as a Tenant A’s user and try to delete all the users in other tenant using delete user API using Post Man

My experiences as a QA in adopting AGILE

Here are my experiences as a test engineer in adopting the AGILE approach:

Initially when i was working in Waterfall model, our QA work for a feature completely depends on the functional specification written by developer. We used to consider that document as the main reference. Based on that, we used to write a detailed test plan and start testing the feature once the whole feature is available to us. Then write automation tests before the regression cycle begins for a release.

When our company, switched to AGILE model, the first question that raised in my mind is "How can QA engage effectively during a sprint before anything has been built?". Scrum is an Agile approach to software development, which focuses on delivering valuable business features in short development iterations of 2 to 4 weeks (In our case, it 3 weeks).  After working a few years as a QA in a Scrum team, I have learned that the role of QA in Scrum is much more than just writing test cases and reporting bugs to the team.

In initial days of scrum, I used to think daily stand-ups are nothing but wasting time and seriously, i used to see AGILE approach as A-JAIL. The concentration in scrum is on what to tell instead of the work being done. Also, the information being shared in scrum is well known by everyone. We know who is working on which feature, but why do we need a separate 15 min meet to just know a statement saying “working on so and so”. Yes, its more than the status. Team shares the detailed status where QA can know the details of the feature and the impediments being raised and what is stopping developers from continuing the work.

Here are few things i learned.

• In Waterfall model, QA involves at the end of development. But, in AGILE, QA will be involved right from the very beginning of a project. 
• Working together - Scrum team is a cross-functional team where development, QA, platforms and documentation teams work together as a team. 
• QA, the first and ever customer - I always used to think that QA is the first customer of the product and QA knows well about the product more-than development team who typically focuses on the “happy path” of the feature. Since, QA excels at identifying and capturing complex and negative test case scenarios. Including testers during Release and Iteration Planning, when user stories are estimated, QA can help the team think beyond the happy path. 
• Helping Product Owner - QA role provides feedback from their testing experience to the Product Owner and work closely with the Product Owner to help them develop detailed acceptance criteria for their user stories. QA can also help the Product Owner modify or enhance existing user stories to better reflect the true requirements. 
• Fast feedback – Developers can consult the QA about acceptance criteria or the expected behavior of any functionality from the user's perspective while working on the feature, which results in saved testing and bug fixing cycles.  
• Automation, QA’s best friend - In scrum (3 weeks), QA generally have less time to test the application. Also they are responsible for testing the feature implemented in current sprint as well as regression testing the feature implemented in last sprint. In such short cycles, automation is the best friend for QA (as alwaysJ). It helps to reduce the pressure QA feels. 
• Participation in Release readiness - With short sprint cycles, developers have less time to explore the complete functionality of their user stories on their own. As a result, developers typically consult with QA to better understand the user stories since they are the ones aware of the complete functionality and know each and every requirement and acceptance criteria. As a result, it is a good practice for QA to perform the demo at the Sprint Review meeting. 
• Preparing test strategies - Scrum believes in preparing only enough documentation to support the immediate needs of the team. As such, QA will prepare just enough high-level documentation for test strategies and plans to guide the team. 
• Sometimes it may not be possible to perform testing of non-functional aspects, such as system performance, within a sprint. This can taken once after the development sprints complete.

 

While QA still write tests and report bugs, they also support many other roles and responsibilities on the team. They are an important part of the team. 

Having worked as QA in AGILE team, i have learned many things, it has added many wonderful skills to my toolbox and has helped me learn how to play many different roles.

Overview of the JMS API

This overview defines the concept of messaging, describes the JMS API and when it can be used

What Is Messaging?

Messaging is a method of communication between software components or applications. A messaging system is a peer-to-peer facility: A messaging client can send messages to, and receive messages from, any other client. Each client connects to a messaging agent that provides facilities for creating, sending, receiving, and reading messages.

Messaging enables distributed communication that is loosely coupled. A component sends a message to a destination, and the recipient can retrieve the message from the destination. However, the sender and the receiver do not have to be available at the same time in order to communicate. In fact, the sender does not need to know anything about the receiver; nor does the receiver need to know anything about the sender. The sender and the receiver need to know only which message format and which destination to use. In this respect, messaging differs from tightly coupled technologies, such as Remote Method Invocation (RMI), which require an application to know a remote application’s methods.

Messaging also differs from electronic mail (email), which is a method of communication between people or between software applications and people. Messaging is used for communication between software applications or software components.

What Is the JMS API?

The Java Message Service is a Java API that allows applications to create, send, receive, and read messages. Designed by Sun and several partner companies, the JMS API defines a common set of interfaces and associated semantics that allow programs written in the Java programming language to communicate with other messaging implementations.

The JMS API minimizes the set of concepts a programmer must learn in order to use messaging products but provides enough features to support sophisticated messaging applications. It also strives to maximize the portability of JMS applications across JMS providers in the same messaging domain.

The JMS API enables communication that is not only loosely coupled but also:

    Asynchronous: A JMS provider can deliver messages to a client as they arrive; a client does not have to request messages in order to receive them.

    Reliable: The JMS API can ensure that a message is delivered once and only once. Lower levels of reliability are available for applications that can afford to miss messages or to receive duplicate messages.

When Can You Use the JMS API?

An enterprise application provider is likely to choose a messaging API over a tightly coupled API, such as a remote procedure call (RPC), under the following circumstances.

    The provider wants the components not to depend on information about other components’ interfaces, so components can be easily replaced.

    The provider wants the application to run whether or not all components are up and running simultaneously.

    The application business model allows a component to send information to another and to continue to operate without receiving an immediate response.

For example, components of an enterprise application for an automobile manufacturer can use the JMS API in situations like these:

    The inventory component can send a message to the factory component when the inventory level for a product goes below a certain level so the factory can make more cars.

    The factory component can send a message to the parts components so the factory can assemble the parts it needs.

    The parts components in turn can send messages to their own inventory and order components to update their inventories and to order new parts from suppliers.

    Both the factory and the parts components can send messages to the accounting component to update budget numbers.

    The business can publish updated catalog items to its sales force.

Using messaging for these tasks allows the various components to interact with one another efficiently, without tying up network or other resources. Following figure illustrates how this simple example might work.

Manufacturing is only one example of how an enterprise can use the JMS API. Retail applications, financial services applications, health services applications, and many others can make use of messaging.

PAS for OpenEdge / OpenEdge Next Generation AppServer

What is PAS?

1. PAS is an abbreviation for Pacific Application Server
2. Unified Application Server for Progress based applications
• OpenEdge
• Rollbase
• Corticon
3. PAS utilizes Apache Tomcat Server as its web server

What is PAS for OpenEdge?

Is a PAS For deploying, hosting and managing OpenEdge applications Integrated into one layer
Is available in two separate products:
    The Pacific Production Application Server for OpenEdge
    The Pacific Development Application Server for OpenEdge

Why PAS for OE?

1. Simple
• Administration
• Scalability
• Deployment
2. Extensible
• REST APIs for customer developed metrics, monitoring, and administration 
3. Analysis Tools
• Built-in metrics gathering, current state queries
4. Faster and optimized
• Runs same ABL application and client load with less memory and CPU consumption

PAS for OpenEdge Administration

1. tcman.sh
2. OpenEdge Explorer / OpenEdge Management
3. Management REST API
4. JMX remote access

Root Cause Analysis: Why and How

Do testers always have to provide root-cause analysis? No, they don’t need to and, in fact, all depends on a particular case. Every single testing activity is different from another and there are priorities you have to deal with as project resources are never limitless. There may be small features that don’t require that effort and dedication and there are large features that is quite vital to the product, root-cause analysis may be required.

However root-cause analysis is, indeed, a great thing, you should at least give it a try and here is why: it grants you with more information and data is pure power in a tester’s arms. After root-cause analysis testers do understand the feature under test in a significantly better manner. You can locate so many new marvelous things like critical areas you have never tested and you will gain additional scope during the analysis. And, surely, there will be more bugs for you to track.

Why isn't everybody doing it then?

This process is time consuming and not all projects are capable or willing to spend more additional resources on such tasks, especially if they are low-priority and the release cycles are very short. There are even times where some team members already know what hides in the root-cause making this activity useless.

How can I tell whether any root-cause analysis should be performed?

There are several factors, by measuring which you will be certain on whether the root-cause analysis activity is something you and your project can afford. 

First of all take a deep look at the defect. Is it worth is? 

Secondly go through your current test coverage as well as your test plan to make sure if you need this additional work in the first place. 

Then review defect priority to make sure if it requires additional attention and take a good look at your team/team members: are they up for more work helping you? Do they have time? Is it really necessary? 

And if all the mentioned above factors are satisfying you should go for root-cause analysis.

Finally, if you want to do it, convince with confidence!.

Source: http://blog.bughuntress.com/

Don’t Expect Magic From Test Automation

The primary reason for test automation is to free up QA time for interesting exploratory testing and to give confidence to the team that the application is still in good order as new changes are delivered.

Don’t expect automation to find lots of bugs. In fact, the number of bugs found by automation is always much less than manual and exploratory testing.

Few reasons Why Automated Tests Fail to Find Regression Bugs
It is widely believed that the purpose of automated tests is not to find new defects but rather find regression bugs as new features are developed. But even so, there are many occasions where regression bugs slip through that end up in production and really should have been caught by the automated regression tests.

Let’s examine the reasons why automated tests failed to find the regression bugs:

The Scenario Was Not Thought Of

Automated Tests are as good as the person who designed the tests. Automated Tests are a set of instructions that are executed against the target application. First, test analysts design test cases and come up with a list of scenarios. The scenarios are then scripted in a programming language to execute the scenarios automatically. Therefore, if a particular scenario was not thought of, it wouldn’t have been scripted to run as automated test.

The Scenario Was Thought Of But Was Not Scripted

It takes time to automate tests. Depending on the complexity of the tests, test engineers coding skills, flexibility of test automation tools and frameworks, some tests take a long time to automate and hence miss the chance to find regression bugs as new features are developed.

There Is A Bug In The Test Code Itself

There are situations where the automated tests do not really run against the tester’s intentions or assumptions. In other words the automation engineers made a mistake in coding the tests or did not insert verification points in correct places.

The Automated Tests Couldn't Execute Due To Environment Issues

This is particularly true when running System Tests via application UI, e.g. launching the application in browser. In such cases there are many dependencies on other applications, 3rd party or downstream systems, and if any of these are down or not responding or respond intermittently, the automated tests could not execute successfully and hence could not verify correctness of a particular test.

Poor Analysis Of Test Reports

After automated tests have executed, when there are failed tests, analysis is required to see the reason for the failed cases. This can take quite some time to analyze all the failed cases (many of which can be false positive). The analysis part is normally done manually and if the analysis is not done correctly, there could be genuine failures that are overlooked or masked by other issues.

Can you think of other issues why automated tests miss defects?

Source: http://www.testingexcellence.com/test-automation-tips-best-practices