Best Practices - Working with Schedules

A schedule enables Leapwork users to define when flows should run, and on which agents - whether at pre-selected times or after specified time intervals. Schedules can be triggered either on-demand (such as through REST API calls to Leapwork) or at a single or recurring time interval.

Why should I use schedules?

Users can leverage the power of schedules to achieve many scenarios that otherwise look strenuous. One of the major challenges most organizations face is running automation in a continuous manner and ensuring the optimum usage of agents by doing parallel testing. When we follow best practices, Leapwork schedules can help us to overcome these challenges.

When should I use Schedules?

Execute Unattended Automation
If we wish to execute our test cases automatically without manual interventions, we will need to schedule our tests. With Leapwork, we can configure the time recurrence of the schedule, which will enable us to test even when we are away from the office.

Achieve Continuous Testing
Continuous testing is the process of executing automated tests as part of the software delivery pipeline in order to get early feedback for each release candidate. It advances and stretches out test automation to address the complexity and pace of application development and delivery. We can use the Leapwork schedule functionality to achieve continuous testing with minimal effort.

Achieve Parallel Testing
Parallel Testing implies testing various applications or sub-components of one application simultaneously to decrease testing time. Parallel tests consist of two or more parts (projects or project suites) that check the different flows or functional attributes of an application. These flows are executed on individual computers or devices simultaneously. The ability to test more than one application part "in parallel" enables us to reduce test time and become more efficient. The use of schedules in Leapwork is strongly recommended for parallel testing.

Implement CI/CD Pipeline
A CI/CD pipeline is the foundation of a cutting-edge DevOps setup. It overcomes any issues among development and operations teams by automating the building, testing, and deployment of applications. Automated pipelines remove manual errors, provide standardized development feedback loops, and enable fast product iterations. Leapwork's Public REST API enables us to use schedules to support the implementation of a CI/CD pipeline.

How should I use schedules?

Continuous Testing using schedules

Scheduling is one of the major features of Leapwork. If used correctly, it can ensure less time to market and high product quality with a reduction in development costs.

There are two ways to run schedules and perform continuous testing:

• Triggering the schedule using Leapwork's REST API
  The schedule can be hooked to our ALM (Application Lifecycle Management) or build and deployment tools. This means our deployment tool can trigger Leapwork's REST API endpoint to start the automated tests on the given environment. For example, one scenario is to make an automated test pass into our manual test entry criteria. This can be achieved by triggering a smoke test schedule in Leapwork at the post-deployment stage of the build. If the smoke test suite passes then the manual testing team can enter into the testing phase for new features.
• Use the scheduler to test continuously
  Leapwork's scheduler has a feature that enables users to execute tests at regular intervals. For example, if we know our code is deployed to the test environment every 2 hours, then it makes sense to schedule the tests to run soon as the build is deployed and to test it twice instead of once. If the first test fails due to environmental or other external factors, this can be verified in a second test run.

Run parallel tests to reduce time to production

Running tests in parallel can give benefits like:

• Time saving: Running test in parallel reduces the time of execution drastically, which means we can test faster and get quicker feedback.
• Better test coverage: Testing on multiple configurations can be achieved with multiple agents set up with different browsers and platforms.
• Fewer flows: One flow can be used to test multiple environments, which means one test can cover many configurations thereby reducing the number of tests.
• CI/CD: Due to faster feedback we can make UI tests part of CI/CD pipelines.

Leapwork enables users to run multiple flows at the same time in different machines by executing them in parallel. Users can also run tests pairwise, where each flow will run on all selected environments to test all possible discrete combinations. The following article explains how to achieve parallel and pairwise tests in Leapwork.

Set an automatic notification after test execution is completed

Automatic notifications help the team to be on the same page and take prompt actions if a test fails. Leapwork provides an 'Actions' option, where users can add actions that are triggered on specified conditions. Each schedule can contain multiple actions. Using these options ensures timely notifications about failures so the team can fix bugs quickly to save time and costs.

For more detail, please visit the scheduler's action page.

Stop testing if test failure reaches a certain threshold

If our test environment is not working or there is a bug in the application that does not allow us to test properly, then it makes sense to stop/skip the execution of our automated test to save resources and ensure the team can take corrective actions quickly.

Let's try to understand this scenario with an example: Assume our login feature is not working and we are using that feature in 80% of our test cases. If we are executing our test, we will usually come to know this after a full test automation run is completed. This might take hours if we have a large number of tests, which means we have wasted resources and time.

Instead, Leapwork enables us to stop the execution run if 20% of tests fail or if the user wants to disable the scheduler on the failure of 'n' number of test flows. With this feature, we can get quick notifications of failure and fix the underlying issue instead of wasting time and resources.

Here is how we can configure error handling in Leapwork:

The Leapwork scheduler provides control over automation flows and helps to optimize time and resources. Among many things, it includes the following:

• Granular control to run flows once or more at specific times during the day, in specific time-zones.
• The estimated run time is automatically calculated based on past performance and parallelization options.
• Recurring schedules can be controlled with working hours windows and queue options.
• Steps in a run-list can be easily parallelized; either divided out across selected agents or run on all.
• Scheduled runs can be skipped or robustly queued based on the agent's busy status.
• It can be used with CI/CD systems to ensure faster time to market.

For any further clarification, please contact our Priority Support.