Data-driven testing in Java sample


There are times when you run across functionality you want to test with various inputs - to see how the system behaves, test the expected outcomes, or to see if the system breaks in an odd way.

Wiring up all these permutations into a set of automated tests can be quite a task, and those tests tend to be hard to maintain over time. Luckily, there is a way to make things easier in this area ( as you’re already used to when it comes to Ranorex Webtestit :wink: )

Data-driven testing, a neat test design strategy, allows you to create test scripts that read and import test data from data sources such as Excel sheets, CSV files, databases, etc. rather than using hard-coded values.
In this article, we will describe our Data-driven sample project, that uses the fancy @DataProvider to read the test data from a CSV file.
Basically, we will test this login form with 3 sets of credentials stored in the CSV file. The goal is to read data from the CSV file and use it in a real test.

  • Let’s start by creating a folder in the project tree named data, where we will store the actual CSV file and create the data provider class file.

Often you will find samples on the internet telling you to place the DataProvider into your actual test file. This is ok for a quick and dirty demo but should be avoided in favor of a clean separation of concerns and reusability in multiple tests files.

Depending on the test requirements, the CSV file format can be slightly different for each test. The file might or might not contain the header record, values can be placed under single or double quotes, so some additional operations may be required to get the actual data. In our case, we are reading the data line by line and saving it into an ArrayList using indexes.

  • Create a Page Object file - we created a Page Object file and captured the username, password, and login success / fail message elements using the Ranorex Selocity extension

    You will notice that the actual login message is trimmed, because of the presence of a special character × in the message text, that we excluded.

  • Create the class - Let’s move on and create the` class file in our data folder. Using the CSVReader we will read the data from the CSV file. Each line of the file represents one credentials combination. As the DataProvider needs a two-dimensional Object[][], we have to convert the ArrayList<Object[]> that we got from the CSVReader.

    Now we are done with setting up the CSVReader and the actual @DataProvider, which we’ll use our CSV data in the Test to populate the username and password fields and assert the login success/fail message.

    Data provider returns a two-dimensional object to the test method and the test method will invoke M times in an M * N type of object array. For example, if the DataProvider returns an array of 3*3 objects, the corresponding test case will be invoked three times with three parameters.

  • Create a Test file - coming to the actual test, we created a new Test file, and in the @Test annotation, we are specifying our data provider that was named “Credentials”, and the type of the dataProviderClass (CredentialsDataProvider.class).

  • Showing the test case names in the report
    Using the implemented ITest interface, and formatting the test name under the @BeforeMethod annotation with the following method:

      public void BeforeMethod(Method method, Object[] testData) {
      testName.set(String.format("%s [Username: '%s'; Pass: '%s']", method.getName(), testData[0], testData[1]));

    That can of course be customized, we will provide the test case names in the report.
    Don’t forget to add the @Override annotation for this to complete your customized test case name

      public String getTestName() {
          return testName.get();

    As our CSV file contains three sets of credentials, so this test will execute three times. The first credentials are the correct ones, and the other two are faulty. We are going to test the login form by trying out all the credentials and asserting that the proper message is shown.

  • Create a custom Assert message - we will use the org.testng.Assert.assertEquals(String actual, String expected, String message) method and create a custom message (iteration indicator) in case of test failure. The message will be visible in the report, providing you with the information on what set of credentials, while logging in, resulted with an inappropriate login confirmation message for that test run.

In this article, we described the steps needed to create a data-driven automated test using the DataProvider feature of TestNG. There are three test scenarios – one where the correct credentials are being submitted to the form and the success message is asserted, and two where incorrect credentials are submitted and the login fail message was asserted.
We also created a custom assert message displayed in the test report, that helps us to recognize which set of credentials caused the test to fail.

Download this sample project directly from within the application, under the Samples category of the Welcome screen. This sample project is also available for download from our Git repository.

Release notes v0.22.0