How to best integrate with CI-CD pipelines for test execution

Why Testing in CI/CD Matters

Shift-Left Defect Detection

Defects discovered late in the release cycle are exponentially more expensive to fix. By embedding Leapwork tests into CI/CD pipelines, organizations can identify issues at the integration stage rather than waiting until production. This 'shift-left' approach reduces rework, lowers defect leakage, and improves collaboration between developers and testers.

Quality Gates

CI/CD pipelines are not just about automation—they are about governance. Quality gates ensure that code cannot progress to the next stage unless it meets defined quality criteria. Leapwork schedules can serve as these gates: if a smoke test or regression flow fails, the pipeline halts automatically. For example, if the checkout process fails during a Leapwork schedule in Azure DevOps, the pipeline will immediately stop deployment and flag the build as failed. Conversely, if a non-blocking UI alignment test fails, the pipeline logs the issue but continues. This creates accountability and enforces standards without manual intervention.

Release Confidence

The ultimate goal of integrating tests in CI/CD is confidence. When developers push code, they need immediate feedback on whether critical business processes are intact. Leapwork provides not only execution but also detailed logs, screenshots, and JUnit reports that are accessible directly from the pipeline. This visibility ensures that stakeholders across development, QA, and operations trust the release process.

Architectural Blueprint

At a high level, integrating Leapwork into a CI/CD pipeline follows this flow:

1. Code Commit

Developers commit code to version control systems such as Git or Azure Repos. Every commit triggers the CI/CD pipeline.

2. Build and Package

The CI/CD engine (Jenkins, Azure DevOps) compiles and packages the application, creating deployable artifacts.

3. Automated Validation

Leapwork schedules are triggered to validate smoke flows, regression packs, or end-to-end business processes.

4. Result Capture

Results are captured in real time and exported in formats like JUnit or XML, making them compatible with dashboards.

5. Decision Gate

Pipelines evaluate test outcomes. Critical test failures (e.g., checkout process failure) block deployments, while non-critical issues are logged but do not stop delivery. This balance keeps velocity without compromising quality.

Integration Patterns

Jenkins Plugin

Leapwork provides a native Jenkins plugin, allowing test schedules to be added as build steps. The plugin communicates with Leapwork’s REST API but abstracts configuration through Jenkins’ UI. This reduces scripting overhead and ensures results are captured as part of Jenkins build artifacts.

For a step-by-step guide, see our KB article on integrating Leapwork with Jenkins (see KB: How to integrate Leapwork with Jenkins).

Azure DevOps: YAML Pipelines

Leapwork integrates with Azure DevOps YAML pipelines by providing a task that can be defined within pipeline files. Teams specify controller endpoints, API keys, and schedule IDs. This ensures test executions are version-controlled alongside application code, promoting consistency and traceability across environments.

For a specific example, see our KB article on running Leapwork schedules via YAML in Azure DevOps (see KB: How to run Leapwork schedule via YAML build pipeline in Azure DevOps).

Azure DevOps: Release Pipelines

For release orchestration, Leapwork provides an extension in the Visual Studio Marketplace. Once installed, Leapwork tasks can be added to agent jobs in release pipelines. This pattern enforces validation at each promotion stage (QA, staging, production).

For detailed setup, see our KB article on Leapwork Azure DevOps plugin-based integration (see KB: Leapwork Azure DevOps Plugin based Integration).

Best Practices and Pitfalls

Use Schedule IDs Instead of Names

Schedule names can change as features evolve, but IDs remain constant. Using IDs ensures pipeline stability and traceability, preventing failures caused by naming updates.

Secure Sensitive Data

Never store API keys or credentials directly in pipeline YAML or Jenkins configs. Instead, use secret stores such as Jenkins Credentials or Azure Key Vault. This safeguards automation against unauthorized access and supports compliance.

Define and Calibrate Quality Gates

Not all tests carry the same weight. Critical flows like login or payment should halt pipelines immediately on failure. Non-blocking tests, such as cosmetic UI checks, can log issues while allowing progress. Calibration of gates avoids both false confidence and unnecessary pipeline stoppages.

Parallelize Test Runs

Running tests sequentially can bottleneck CI/CD adoption. Leapwork supports distributed execution across environments. Designing automation suites for concurrency reduces feedback loops from hours to minutes.

Maintain Plugin and Agent Configurations

CI/CD ecosystems evolve quickly. Outdated plugins can break compatibility (e.g., Jenkins and Java 11 JAXB issues). Regularly updating Leapwork plugins ensures pipelines remain secure and functional. Similarly, align Azure DevOps agent pools with Leapwork execution requirements to avoid stalled jobs.

Conclusion

Integrating Leapwork with CI/CD pipelines transforms automated testing into a driver of agility and confidence. With native plugins, YAML definitions, and release pipeline integrations, Leapwork becomes a seamless extension of DevOps workflows. By following best practices—using IDs, securing data, calibrating quality gates, and maintaining tooling—organizations can achieve faster releases without sacrificing quality. The result is a balanced pipeline: fast, reliable, and business-ready.