Storing Test execution log to an external file using TTS

Whenever we run/debug Telerik Tests normally (without using scheduler or test list), our execution log will not be stored on disk.

Execution log is cached in-memory and will persist until you rerun the test case or close studio. So if we use our own frameworks to run telerik cases then we will not have logs stored on our disk.

Similarly if you have 3 tests running in a test list, result will not be shown until the execution of list completes.

So to look at the result of test1 we have to wait until the complete test result is executed.

To overcome these challenge, TTS suggest to override OnAfterTestCompleted method for individual tests in order to store result in logfile.

Below is the sample working code to achieve it.

public IList<AutomationStepResult> stepResults { get; set; }

        public override void OnAfterTestCompleted(TestResult result)

        {

            stepResults = result.StepResults;

            string passed = Convert.ToString(result.TotalPassedSteps);

            string notRunSteps = Convert.ToString(result.TotalNumberOfNotRunSteps);

            string overall = Convert.ToString(result.Result);

            string resultMessage = Convert.ToString(result.Message);

            string startTime = Convert.ToString(result.StartTime);

            string endTime = Convert.ToString(result.EndTime);

            string totalTime = Convert.ToString(result.Duration);

            string s = Convert.ToString(DateTime.Now.Date.ToShortDateString()) +    Convert.ToString(DateTime.Now.Hour);

            string dateOfExecution = s.Replace("/", "");

            var filePath = this.ExecutionContext.DeploymentDirectory + " \\TestResults\\" + result.TestName + dateOfExecution + ".txt";

            Log.WriteLine(filePath + "*********");

            if (!File.Exists(filePath))

            {

                File.Create(filePath).Dispose();

            }

            var file = new StreamWriter(filePath, true);

            file.WriteLine(resultMessage);

            file.WriteLine("Number of passed steps: " + passed);

            file.WriteLine("Number of not run steps: " + notRunSteps);

            file.WriteLine("Total test result: " + overall);

            file.Close();

        }

But still drawback of this method is we need to write this piece of code in each and every test if you want for each test.

Sumologic and its Key features

Building applications became easier over the years with the adoption of Saas, PaaS and IaaS which help in building complex applications and services using different servers, technologies, hardware etc.

If any issue is raised, it is very difficult in figuring how customer encountered it and what is the exact problem.

Whenever we want to debug real time scenarios like above, we follow below procedure.

1.      We require lots of information and logs from different places which contains different timestamps.

2.      Spend huge time in analyzing all those logs to understand what the exact problem is

3.      Then finally coming to conclusion and fixing it.

So, imagine how easy it will be if the log and data from different places are collected and stored at one place. How good it will be, if we get complete results just by running simple queries? How nice it will be if we have brilliant visualizations and dashboards which gives us the proper error.

Yes, one way to solve above steps is to use Sumologic.

What is Sumologic?

Sumo Logic is the industry’s leading cloud-based data analytics service and also a next-generation log management service that reduces the complexity of managing log data in enterprise IT organizations and extracting strategic operational insights from that data.

 

Key features of Sumologic

Below are the four key features of Sumologic.              

                              

1.      Collect and centralize

Sumologic can collect terabytes of data from any app, cloud, device, custom hardware, sensor and network sources.

Recently in June 2015, Sumologic added support for collecting Data from Docker infrastructure as well.

              

Most efficient way to get your logs into sumo is to send them directly from our application.

How to send data from our applications – There are different ways using which we can send data to Sumologic

1.      It supports different open source libraries for Java, .net using which we can send data to Sumologic.

2.      Data can be directly sent through HTTP web API

3.      It also provides support for different Cloud technologies like Google Apps, AWS etc. using which we can send data.

4.      Data can be sent by installing collector agent in a machine

2.      Search and Analyze

Once the data is pushed to Sumologic we can start searching within minutes. Sumo Logic search syntax uses logical and familiar operators, allowing us to create ad hoc queries quickly and efficiently

We can save searches to re-use later or to run as regularly scheduled searches that can be delivered to our email address

The basis of Sumo Logic Search Syntax is a funnel or "pipeline" concept. Beginning from all of our current Sumo Logic data, we enter keywords and operators separated by pipes ("|"). Each operator acts on the results from the previous operator so that we can progressively filter and pinpoint our search until we find exactly what we are looking for.

Simple example of search is - keyword search | parse | where | group-by | sort | limit

Keyword search  – A full text search expression is called as Keyword Search. i.e. anything we type in the query field before the first pipe is always a full-text search expression (ex: _sourceCategory, _sourceName, _sourceHost etc.)

Sumo Logic Operators – Remaining all in the above syntax like parse, where, group-by, sort, limit are sumologic operators.     We can get detailed information about all operators here.

3.      Monitor and Visualize

We can build custom dashboards with different visuals provided by Sumologic. This helps in monitoring data in real time.

It supports different types of charts such as bar, pie, map, etc. which we can use for specific use cases.

4.      Alert and Notify

In Sumologic we can design real time alerts on some search query and configure email notifications to respective groups/members whenever an error condition matches.

These alerts will run based on particular time range that we define while configuring the alert.

This alert and notification feature helps dev and qa teams to directly triage the issue instead of waiting till someone intimates about the issue.

There is some operator limitations for Real Time alerts. More Details can be found here