Telerik Test Studio Runtime setup

Test Studio has many features and recently we have used its Scheduling service for running our automated test lists. Although there is detailed documentation here on how to setup the runtime, many issues have come up while doing it manually. Here are few things which will help us in understanding the feature better.


This is how the scheduling service works:

•  When a test list is created and scheduled, the all tests which are present in the list only will be uploaded to Storage DB – TestStudioStorage in the SQL Server 2008 which will be installed with a customized selection while installing Test Studio. (The DB details can be seen in the file at “C:\Program Files (x86)\Telerik\Test Studio\StorageService\CloudStorageSelfHost.exe.config”
•           The DB then copies all the files to a Temp folder (%Temp%\Projects\) and from this location the file locations are picked up while scheduling.
•        Once a list is uploaded to DB and any changes are made later into the same list, new changes will not be updated in DB – An existing issue in TTS. This requires a manual clean up of the files in the DB using a tool called DBBrowser.exe. This will allow us to delete the desired files. Scheduling the list again will update the DB with new changes.
• We can try out the dynamic list option for creating a test list and see if that updates the project files automatically to the DB. 
• Also, there is an option where you can right click on the schedule which you've created and upload the latest files to the DB. Please see the steps here.

Below are the things which are to be kept in mind while using Test Studio Scheduling Server:

• Ensure that the Telerik Scheduling and Storage Services are started and running.
•  The port for connections should be proper while installing the Scheduling Server. More info here.
•  Connection to the Execution Server should be up and available in the list of running services.
• All the references which are used in the solution level should be included inside the project folder for the tests to recognize and map accordingly.
• If any relative test paths are used in coded steps, i.e. Test as step – coded step (As mentioned in the step 2 above, relative path tests will not be uploaded to the temp folder), those tests should be uploaded to the storage DB. This can be achieved by creating a dummy test list and simply running it – This would upload the test files to DB (The result of this dummy list can be ignored, it is just to upload files to DB). Once these tests are uploaded to DB, they can be called using relative paths in the tests of test list. Else, we will encounter the error “FileNotFoundExeption” for the tests in relative path.

However, there is a minor glitch in the scheduler wherein the first test in the test list is not executed, from the second test, the list runs fine. As a workaround, we can add a dummy test as the first test in the list and scheduled it so as to not disturb the desired batch of tests.

An alternative to stop using the scheduler and run the test lists locally with email triggers after each run of the test list can also be achieved by following below steps:

• Create a class library with a class which extends IExtensionExecution library.
• Ovevrride the methods such as OnAfterTestCompleted, OnAfterTestListCompleted and write custom code to trigger emails/write logs etc.
• Build the project and add the project.dll to the references of the project which contains tests/test lists (CSTestUI in our case)
• When the individual tests are run, OnAfterTestCompleted is invoked and when test lists are executed, OnAfterTestListCompleted is supposed to be invoked.

But I have checked this behavior and observed that OnAfterTestCompleted method is getting invoked at the end of each test and generating email too, but OnAfterTestListCompleted method is not getting invoked.

There is a post in telerik test studio forums (link) which provides a workaround for the same – available in the documentation link. But even the workaround does not seem to work and the support team did not provide any solution to this. Even if we prefer to run locally, we can navigate to the test lists in the local machine through UI. So, We can continue to use the scheduler to get the result reports. 

There could be other workarounds / solutions for these issues which can be updated in this post at a later point of time when found! :)