Mobile App Testing with Telerik Test Studio

Over the last few years, Mobile Applications development has grown exponentially. QA needs to ensure that every mobile application meets high quality. Testing mobile applications is complex and difficult compared to desktop and web applications. In mobile world, with frequent OS upgrades, new devices support, it is really complex to test mobile apps and to automate testing.
We will be able to solve these kind of issues with Telerik Test Studio Mobile.

It allows developers to write automated functional tests in JavaScript using a set of APIs we devised to make the process as easy as possible. With Test Studio, we can run automated tests on real devices as well as emulators. All we need to do is to install Mobile Testing agent on the devices we need to test. Agent will take care of connecting mobile device to our test studio script.

AWS Device Farm for Mobile App Testing

We have different types of mobile devices. They vary in the form factor, OS and various types of apps (native, hybrid, mobile-web) used in them.

Ref: http://www.flurry.com/sites/default/files/blog-images/SizeMatters_Chart2-resized-600_0.png 

This creates unique challenges to QA apps developed for mobiles. It is difficult to have all these combinations of devices with us. further running automated test cases across multiple real devices is another challenge.

AWS Device Farm is one solution for that. It allows you to test iOS or Android apps of native, hybrid or mobile-web categories on physical devices of your choice in the Cloud. They offer hardware and software isolation which helps us to ensure security and comes with several attractive features such as flexible pricing, schedule your runs, execute tests in parallel, consolidate results etc. 

Ref:https://d0.awsstatic.com/product-marketing/Mobile/DeviceFarm-productlandingpage-banner.png

Localization testing by changing your system locale

Localization: It is the process of adapting internationalized software for a specific region or language by adding locale-specific components and translating text.

Things to take care during installation while testing openedge with non-english environment

OpenEdge by default allows user to install OpenEdge in 14 different languages. Here is the list below-

Czech(VZE)	Polish(POL)
Dutch(Dut)	Portuguese(POR)
English-American(AME)	Portuguese-Brazilian(BRZ)
English-International(Eng)	Spanish(SPA)
French(FRE)	Spanish-Latin American(SPL)
German(GER)	Swedish(SWE)
Italian(ITA)	Chinese-Simplified(SCH)

Apart from these 14 languages it also supports other supplement languages as well. Here is the list of 22 supplement languages below-

Arabic	Croatian
Chinese (traditional)	Danish
Hebrew	Finnish
Japanese	Greek
Korean	Hungarian
Persian	Icelandic
Serbian	Lithuanian
Thai	Norwegian
Turkish	Slovak
Slovenian	Spanish - Mexican
Romanian	Russian

How to test your internationalization configuration using ABL?

OpenEdge database support internationalization . It means you can build application for non-english language/region and perform required operations. While working with non-english languages, you need to  use appropriate language setting to perform the operation and do the character processing correctly.

You can use ABL query to test the setting from your client. Here is the ABL query below- 

MESSAGE   

"Database    = " DBCODEPAGE(1)      SKIP   

"Collation   = " DBCOLLATION(1)     SKIP   

"-cpinternal = " SESSION:CPINTERNAL SKIP   

"-cpstream   = " SESSION:CPSTREAM   SKIP   

"-cpcoll     = " SESSION:CPCOLL     SKIP   

VIEW-AS ALERT-BOX.

Be a Technical Tester

Testing is a “technical” discipline that is different to programming and testers add a lot of value to projects and it depends on the context they work in. But in general they need to have technical skills to get success in their job.  As a tester I believe, the only way I can do a good job (testing) is by understanding technical and functional aspects of product in detail.

A technical tester adds lot of value to the product by

• Understanding the architecture of the product
• Reviewing the code he needs to test
• Working with tools (testing tools) or scripts (to automate) to help his work
•  Debugging issues by troubleshooting them from logs or system feeds
• Being updated with technical aspects of infrastructure 

A test engineer should be at least be technical enough to do their job effectively and to talk the same language with the rest of your programming and testing peers.

Ref: https://martijndevrieze.files.wordpress.com/2013/06/waterfall-traditional-testing.gif

What is Kerberos Authentication? How to test it using fiddler

In every organization, employees use many applications in their day to day life. E.g. Project Management, Leave management, IT support, Timesheet...etc. All applications may not be provided by one vendor. Each application has its own authentication mechanism. Each employee needs log into these applications daily. Redundancy of entering the same username password on a daily basis for all the internal applications by Employees can be eliminated by implementing the Kerberos Authentication. Using this Kerberos Authentication, we can authenticate the user automatically if that employee is in the same private network. Most of the organizations are using Windows operating system and Active Directory authentication for their Desktop/Laptops. Many application vendors are also supporting this Kerberos authentication.

Kerberos is a network authentication protocol. It is designed to provide strong authentication for client/server applications by using secret-key cryptography. It has the following characteristics:

• It is secure: it never sends a password unless it is encrypted.
• Only a single login is required per session. Credentials defined at login are then passed between resources without the need for additional logins.
• The concept depends on a trusted third party – a Key Distribution Center (KDC). The KDC is aware of all systems in the network and is trusted by all of them.
• It performs mutual authentication, where a client proves its identity to a server and a server proves its identity to the client.

Ref:https://msdn.microsoft.com/en-us/library/Bb742516.kerb01_big(l=en-us).gif

Manual testing

Once the application is implemented with this Kerberos authentication, then the user should be able to access the application without asking login credentials. The application should have an account with the same login name.

 Open any browser and access the application. It should automatically provide the access to the application.

 Sometimes it may not work this authentication, if the DNS server has any problems. Client machine should be able to ping the application hosted server machine and the same IP should be shown at server side if we execute the ipconfig command.

Enabling Kerberos Authentication in Firefox


Firefox does not automatically perform Kerberos authentication against any sites. You must manually add sites to a trusted sites list.

To enable Kerberos authentication in Firefox:

• Open Firefox and enter about:config in the address bar. Dismiss any warnings that appear.
• In the Filter field, enter negotiate.
• Double-click the network.negotiate-auth.trusted-uris preference.
• This preference lists the trusted sites for Kerberos authentication.
• In the dialog box, enter the domain, such as abc.com.
• Click the OK button.

The domain that you just entered in the network.negotiate-auth.trusted-uris should now appear in Value column. The setting takes effect immediately; you do not have to restart Firefox.

Testing using Fiddler

• Download and install the fiddler software at client machine, not at the server machine http://www.telerik.com/fiddler
• Start the fiddler. It automatically captures the traffic
• Access Kerberos Authentication enabled application in any browser E.g., Chrome
• Fiddler captures all requests and displays in left-hand pane
• Click on first successful(200) request
• Click on "Inspectors" tab, then "Request Headers" section -> Headers
• Look for the "Cookies/Login" section or "Security" Section
• If you see the Authorization token begin with “YII” then Kerberos is functioning, if you see “TlR” then Kerberos did not function

Kerberos Working

or

Kerberos not Working

Graphical Test Planning - A picture is worth a thousand words

In software testing world, we write test plans with some list of test cases. A tester writes a test plan for a feature and sends it for review. Higher management and developers will review the test plan and provide their comments. Tester incorporates those comments. Higher management may not have time to read the entire document, thus ending up with only developer's review comments. Similar to architect diagrams in feature's functional specs, test plan can accompany a picture to explain the testing flow/scenarios.

Graphical Test Planning(GPT) is a way of creating a test plan without the need to write entire documents. This methodology was created by Hardeep Sharma of Citrix Systems. GPT helps us to find design-level issues at the earliest.

What a Graphical Test Plan is:

• A structured relationship diagram
• A list of behavioural areas that need to be tested
• A method of getting greeting feedback about what needs to be tested
• A method of tracking progress of design and execution of testing

What a Graphical Test Plan isn't:

• A flow diagram
• A QA manager's thoughts or mind map
• A feature list
• A hand-holding exercise for a test engineer on how to use the application

Ref: http://event.dnd.no/ttc/wp-content/uploads/sites/39/2015/01/David_Bradley.pdf

Advantages of this methodology

• Enables people to understand and work with complex systems

• Describes the exact behaviour we expect to observe from the system

• Find bugs from the beginning of the project

• No waiting for documents, code, etc.

• Be an integral part of the whole project lifecycle

• We understand the system better

• Communicates knowledge and understanding 

• Verifies the design

• Enables good Project Management & Control 

• Provides flexibility and agility

• We improve the system design

• Discover inaccuracies and ambiguities

• Identify potential design issues

• Ensures the RIGHT product is built

• Many people can contribute. Not only testers any person who is involved in this project can contribute to this GTP. Architects, PMs, UX, Dev, QA..etc 

• Test against the confirmed behaviour, not implementation

How to improve QA team's product knowledge

Generally in Product QA team, manager/lead assign some components/features to each person and they become an expert on that feature. Each team member should have the knowledge on the entire product not only on a particular feature. This will help to improve the quality of product and reduces the risk for the company if some team changes are required. Each team member wishes to learn about other components/features, but they hardly get time to learn. We can create opportunity in the following ways.

Conducting training or technical sessions

Training will be benefited if we are forming a new QA team. Instead of each person learning about the product one lead/manager should provide some training to the team. This will save team's time and everybody will start from the same page.  Each team member should conduct technical sessions on their expertise feature. This will help other team members to learn about all the features of the product. This not only improves product knowledge in the team, but also improves the communication skills of the team members.

Sprint Reviews

Ref: http://agiletrick.com/wp-content/uploads/2014/05/Review-Meeting-300x155.png

QA team should attend each and every sprint review and listen carefully about stories presented by other team members. This will improve knowledge on new features. Somebody should maintain sprint review demo recordings. This will help us if any team member misses the sprint review. QA team should practice to give the demo for completed features in sprint reviews.

Answer a forum/support issue every day

Ref: https://goo.gl/RnVU7K

Answering a forum issue is a good practice to learn about the product. Even if we don't know the answer if you spend some time on that issue you will get the answer. If you cannot figure it yourself then you should contact respective dev owner. I am sure they will help you. Don't  worry if anybody answers that question before you. Still you learned something about the product. Management also should allocate some time for answering forum questions. This will improve customer support as well as team's product knowledge.

Changing QA owners for each release

This will provide an opportunity to each team member to get hands on exercise on each feature. When we provide ownership, team member will feel responsibility for that feature and put 100% effort to learn and test. It helps to reply forum threads quickly and also feels like missed the scenario or learn about a corner case if any customer reports any defects on that feature.

Conducting Product Quiz

Each team member can prepare some list of questions on their expertise feature. Conduct a quiz within the team. Each question can open a discussion and at the end of the discussion somebody (Lead) should note down some notes. This should be a fun activity, not a serious one.   

What is Touch Time? How it impacts Delivery

 In agile model, Dev engineers and QA engineers work on a story and deliver it at the end of the sprint. Touch time can be defined as the "The actual time spent developing a product or service with the ultimate goal being the addition of value to the end consumer". Let me take the following imaginary scenario to explain this concept.

A dev engineer completed the development of the story in 1 hour on Monday. QA engineer needed to start verifying it on Tuesday morning, but as QA engineer is already busy with another story, he couldn't pick it up. QA engineer verified that story on Wednesday in an hour and logged a critical defect. Dev engineer couldn't pick that defect immediately and he fixed it on Thursday in 1 hour. QA engineer picks it on Friday and closed the story. In this example, actual time spent on this story is 4 hours and story marked as completed in 40 hrs. Here the Touch time is 10%.

Why do we do multitasking

As part of resource utilization, we want to do something instead of waiting for a task to become unblocked. That exactly happened in above example. If the team multi-tasked to fill the waiting time, how many features get done? Few features or likely much fewer.

Single Tasking and Partner Pairing

In this Single Tasking approach, developer and tester becomes a small team, and work on a single task/story. Developer thinks about implementation and tester thinks about test cases. They should interact with each other. Tester can understand the implementation and developer can the understand about the advanced test cases. E.g. what happens if we enter internationalization characters. Tester can also prepare the automation test. If we follow this approach both will benefit and they can deliver the next story with more quality as they know how developer and tester thinks. We need not follow this approach if our current approach Touch time is ok. Need to consider other tasks like planning and deployment while calculating touch time.

single tasking is not realistic as we need to catch up on email, meetings, support calls ... etc. We should calculate Touch time frequently and we need to work on changes to reach reasonable Touch time.

Ref: https://hakanforss.files.wordpress.com/2014/08/flowefficiencyformula.png

Let QTP monitor your application's performance

While working with our applications, we see our application's performance going down on performing a particular action. Reasons for this might be different. We mainly see these kind of issues when resources reach its capacity. Resources can be CPU resources, memory, application, physical disk and network resources. So, finding these kind of bottlenecks is difficult in manual testing. If you are using QTP to automate your testing, then you have a solution for this.

Ref: http://goo.gl/FXePGx

QTP provides a functionality called Local System Monitoring. Enabling this functionality for a particular script lets QTP monitor and record resource usage for each and every step performed. Finally, it shows a graph with resource usage statistics. When a particular step is selected in result, corresponding resource stats will be shown in graph. So that, it would be easy to know on which action in application, performance is going down and the reason for this (memory leak/thread leak/gdi object count, etc..). In addition, you can export data from the System Monitoring tab for a variety of file types.

Following are few simple steps which need to be performed to monitor system counters:

• Enable Local System Monitoring from File->Settings->Local System Monitor.
• Provide the name of the application to be monitored.
• Add system counters to be monitored. We can even set limit to system counters. So that whenever a particular counter reaches the limit, your test run will fail.    
•  Once test run completes, result page will shows system monitor graph.

These statistics can be exported to any type of file for further investigation.