24 by 7 UI Test Automation Challenges

We have automated most of the Business scenarios in our Product. Most of the times we have not faced any issues. However we have faced few challenges while running the same scenarios in 24 by 7.

Following are the challenges and Workarounds:

• Challenge: Observed Browser hang and not responding for the UI interactions. This might be due to continuous execution and load.
• Workaround: Setting Maximum Timeout for each test so that timeout exception will thrown to stop the browser automatically. Because of timeout, Next test will start running.

• Challenge: Java Script Timeout Error Popup because of load and continuous execution.
• Workaround: Setting Browser Configuration "dom.max_script_run_time" with maximum value.

• Challenge:Browser Profile Locking in case browsers have not closed properly due to different reasons
• Workaround: Cleaning up Browser Profiles using some batch script to delete the profiles periodically.

• Challenge:Capturing Screenshots will not work in remote machine in case of closing the remote machine directly. But Screenshot will more helpful while debugging tests. 
• Workaround:closing RDS directly will create this problem.close RDS with the below command to capture screenshot.
  tscon %sessionname% /dest:console

Performance testing of a modern web application - Key points

Modern web applications are built using heavy javascript files. Many applications are providing application development online. Many applications are integrated with other systems. These are all forcing us to do a Performance test.

We need to define application performance first. You may think that using the following parameters we can define. 

How fast servers are responding?

What is the response time of a request?

How the system is handling large set(1000 or 2000) of users?

Sometimes we need to consider all above parameters or any one. It depends on application

Remember the following key points before planning a Performance test.

Know your users:

You should know who is using your application, what type of users using your application, and finally how they use your application. You can get this information from production system logs, monitoring tools, sales and support team.

Test Data:

Test data should be close to production data. We should provide realistic data. We should not use non representative data, as it may not be easy to validate. We should not start with very large data, it should be equivalent to the production system. Test data should be increased depending on the understanding of the growth of data in the application.

Compare Test Environment & Production Environment:

We should replicate the test environment with the production environment. We should not execute performance test on a small machine and calculate the requirements. This never works.
We should use same machines, same database, same load balancers.. etc

What Is a Realm?

Realm

For a web application, a realm is a complete database of users and groups that identify valid users of a web application (or a set of web applications) and are controlled by the same authentication policy.

The Java EE server authentication service can govern users in multiple realms. In this release of the Application Server, the file, admin-realm, and certificate realms come preconfigured for the Application Server.

In the file realm, the server stores user credentials locally in a file named keyfile. You can use the Admin Console to manage users in the file realm.

When using the file realm, the server authentication service verifies user identity by checking the file realm. This realm is used for the authentication of all clients except for web browser clients that use the HTTPS protocol and certificates.

In the certificate realm, the server stores user credentials in a certificate database. When using the certificate realm, the server uses certificates with the HTTPS protocol to authenticate web clients. To verify the identity of a user in the certificate realm, the authentication service verifies an X.509 certificate. For step-by-step instructions for creating this type of certificate, see Working with Digital Certificates. The common name field of the X.509 certificate is used as the principal name.

The admin-realm is also a FileRealm and stores administrator user credentials locally in a file named admin-keyfile. You can use the Admin Console to manage users in this realm in the same way you manage users in the file realm. For more information, see Managing Users and Groups on the Application Server.

OpenEdge Supports?

Realm	Description
JDBC Realm	The JDBC realm is a user management system built on a database. It reads user, password, group, and Bussiness Process Server specific information from the database. It also provides password encryption
LDAP	The LDAP realm employs the LDAP directory service to retrieve user, password, and general group information. Other Business Process Server-specific information is retrieved from the database. LDAP runs over TCP/IP and features a hierarchical structure. Business Process Server supports Sun Java System Directory Server 5.2 and MS Active Directory for Windows 2000.
LDAP	The LDAP hybrid realm uses a combination of LDAP realm and Business Process Server database. Typically, the LDAP realm is used for authentication and Business Process Server database is used to store groups related information.
OERealm	The OEHybrid realm is a combination of the JDBC realm and the OpenEdge AppServer based service. Business Process Server (BP Server) supports the single point of authentication (SPA) service using the OEHybrid realm.

KeepAlive

A keepAlive (KA) is a message sent by one device to another to check that the link between the two is operating, or to prevent this link from being broken.

A keepalive signal is often sent at predefined intervals, and plays an important role on the Internet. After a signal is sent, if no reply is received the link is assumed to be down and future data will be routed via another path until the link is up again. A keepalive signal can also be used to indicate to Internet infrastructure that the connection should be preserved. Without a keepalive signal, intermediate NAT-enabled routers can drop the connection after timeout.

Since the only purpose is to find links that don't work or to indicate links that should be preserved, keepalive messages tend to be short and not take much bandwidth. However, their precise format and usage terms depend on the communication protocol.

TCP keepalive

Transmission Control Protocol (TCP) keepalives are an optional feature, and if included must default to off.[1] The keepalive packet contains null data. In an Ethernet network, a keepalive frame length is 60 bytes, while the server response to this, also a null data frame, is 54 bytes.[citation needed] There are three parameters related to keepalive:

    Keepalive time is the duration between two keepalive transmissions in idle condition. TCP keepalive period is required to be configurable and by default is set to no less than 2 hours.
    Keepalive interval is the duration between two successive keepalive retransmissions, if acknowledgement to the previous keepalive transmission is not received.
    Keepalive retry is the number of retransmissions to be carried out before declaring that remote end is not available.

Does OpenEdge AppServer support KeepAlive?

An OpenEdge application that employs the AppServer is vulnerable to failures in the persistent TCP/IP connections that are established between the client and the AppServer.   These connections can fail for the following reasons:

Category of Error	Description
closed connection	The client or server application has purposely closed the connection.
process abort	The client or server application process has aborted, but the TCP/IP stack on the host machine is still functional.
host machine crash	The machine where the client or server application is running has crashed.  The TCP/IP stack on the crashed machine is not functional.
network failure	The network between the client and server application host machines has failed.  The TCP/IP stacks on the host machines are still functional, but no route through the network exists between the two host machines.

In the first two cases (closed connections or process abort), TCP/IP notifies the remaining application[1], allowing it to react appropriately.  In the latter two cases (machine crash or network failure), however, the failures are not immediately detected by TCP/IP, leaving the application unaware that a problem has occurred[2].  When this happens, the application is often stalled, consuming system resources and unable to perform additional work.

The features described in this document attempt to improve the ability of the stalled application to detect these “undetected” network failures and react to them.  The specific use cases addressed by the Distributed Component Awareness (DCA) AppServer Keepalive protocol are described below, followed by a description of the protocol.

---

[1] The notification usually comes in the form of a close event when the application attempts to read data from the connection.

[2] Actually, a subcase of the ‘host machine crash’ scenario exists.  If the host machine crashes and restarts before the TCP/IP keepalive interval expires, then an attempt to send data to that machine will result in an immediate error as the old connection is unknown to restarted TCP/IP stack.  For purposes of simplicity, we will ignore this case.

To address the problems described above, OpenEdge has introduced the AppServer Keepalive (ASK) Protocol.  This protocol actually consists of two features, referred to as the serverASK and clientASK protocols.  The serverASK protocol allows the AppServer to detect failures of the client, while the clientASK protocol allows the client to detect failures of the AppServer.

It should be noted that these two protocols operate independently of one another, and therefore may be deployed as such .

Also Refer:
http://tldp.org/HOWTO/TCP-Keepalive-HOWTO/overview.html

Emailable Test Reports with XSLT

• Write your own XSLT to define view
• Write Test Results into XML
• Generate HTML Report by Translating XML to HTML using C Sharp.  
• You can get HTML Report like below. 

1.