Best Practices - Test Data Management & Planning

Test data plays a crucial role in automation and the availability of correct and reliable test data is critical for high-quality delivery. However, it is not that easy to get the required test data easily.

Let's understand with an example: Assume that we are testing a new application that has not yet been released to production. At this point, the biggest question in front of us is how can we get data for the below-mentioned scenarios when there is no data in production.

Example scenarios:

Data for testing a new feature
Finding boundary and equivalence cases
Data for negative testing

The best data is usually found in production. However, it is not always easy to extract the right data for all of our test cases, hence the importance of implementing a test data management and generation strategy. With one in place, we can test all critical scenarios in our application.

In this article, we will explain the best practice for generating and using test data for the following cases:

Manually generate test data and use it in Leapwork

Users can add test data at various levels in Leapwork - as parameters, variables, schedules, API, type text, and set text building blocks. If we are manually entering data at the block level and not passing it from a data source, then it is called 'data hard coding'.

For short runs, this is a good strategy to get quick wins. However, in the long term, we might struggle. Data hard coding in this way makes it very difficult to scale and maintain test flows, especially when we have primary key constraints in our data. For example, a form can be filled with only one phone number n this case, a static data test will work only once. Hence we suggest replacing static data with dynamic data or linking it to a data source as soon as possible.

Automated test data generation in Leapwork

Leapwork provides an out-of-the-box automated way to generating dynamic data using its generator blocks. Generator blocks can be used to generate dynamic data such as random or sequenced numbers, text, e-mail addresses, dates, time, and more.

Generate date/time: The 'Generate date/time' block will generate a date/time value that contains the current system date and time
Generate number: The 'Generate number' block will generate whole or decimal numbers within a minimum and maximum range, optionally only using odd or even values
Generate password: The 'Generate password' block will generate a random combination of numbers and symbols, which can be used as a password

Using the above building blocks, users can generate unique data such as email addresses and phone numbers, etc. With this approach, users can create unique data every time a test flow runs without worrying about having to hard code unique data. Due to the uniqueness of data generated with Leapwork, users don't have to worry about data cleanup afterward as the data will always be unique.

In the ideal scenario, tests should be designed in a way that they create the data first then execute the test. At the end of the test, the data created should be removed by the same test. If one uses automated data generation methodology then it should be easy to create atomic automation with less stress on test environments.

The benefit of this data generator pattern is numerous. It means:

There is no external data dependency
We can create sub-flows for various data like a unique email address, then reuse it multiple times
We can generate any type of string number or unique data at runtime
There is no need to clear data for unique constraints

Using external test data sources with Leapwork

Testing with a large set of data requires data to be organized through external data sources such as Excel, CSV, text files, and databases. Leapwork enables users to interact with these external data types through its out-of-the-box integration with these tools. As a best practice, if we have a large set of data, we should use one of the below data providers as per our requirement:

Read-Write Excel

The 'Read Excel' and 'Write Excel' blocks in Leapwork make it easy to parameterize and drive automation cases with structured and tabular data. Whether we want to perform smoke tests by picking random test data from an Excel file, or need to loop through thousands of rows in functional test cases, it only takes a minute to start using the "Excel" block.

The Read Excel block is used to read an Excel file, select a range of data, and then use that data to drive an automation case
- The Writes Excel block is used to write data into an Excel file. A user can select a range of header columns and then insert/append data into the Excel file under the selected columns

The Read and Write Excel blocks should be used when data requirements are limited and less effort is required to add, maintain, and delete data in those Excel files. Go here to learn more about Leapwork's Read Excel and Write Excel blocks.

Test Data from a database

When the test data requirement is huge and there is a need for frequent insert, read, update and delete operations then the best practice is to go for ODBC compliant database connectivity for test data management.

Leapwork's 'Database block' is used to execute a query on an ODBC-compliant database such as Oracle, SQL Server, DB2, MySQL, MongoDB, or any other. This can be highly useful for driving automation cases with a large set of data and using these databases to perform operations like Read/Insert/Update data in the database through Leapwork database block.
Users can also use Excel as a database if they require to perform read/insert/update operations
Read/ write text files

Read and write text file operations might be required to create and read data in some situations. Leapwork's 'Command-Line' and 'PowerShell' blocks should come in handy in those situations as they have the capability to read write text files:

Command Line blocks can be used to read and write into text files:

Write File Read File
- The Command-Line block can also be used to execute batch and PowerShell scripts for data generation and other activities, the user should use them when required.

Non UI test data generation

Leapwork offers the capability to interact with applications without using the user interface. This enables Leapwork users to generate test data through non-UI protocols such as REST API, SOAP API, and command-line interfaces like Powershell and Command Prompts.

As a best practice, we suggest creating data with non-UI methods, and if the application has an API interface, this should be used to create test data before the test is executed.

For example, assume that we are testing a content management system where we need multiple user types to be created in order to check the access level of those users (such as admin, user, contributor). If our REST API interface has the capability to create these users then we should call those API interfaces to create different users as test data then log in as those users in the test. By calling the user REST API endpoint, we should be able to create those users and get the information to log in. In this way, we will be able to create test users faster and data will be generated, which makes the test independent of external data sources.

Leapwork's 'HTTP block' is used to call REST API interfaces, whilst the Command Line block is used to run command line and PowerShell operations. Here are examples of usage of the HTTP block in the context of JSON and SOAP calls.

Environment Database Baseline

An environmental database baseline is a process of getting fresh production data to our test environments on a regular basis so that we can create a production-like test environment with real test data so we can test and debug real-time scenarios.

As a best practice, users should try to get daily/weekly/monthly baselined data from production to work with good quality test data in our test environments, and test on an environment similar to production.

Usually, these database baseline scripts are written in SQL, and then they are triggered by Leapwork through a schedule.