Set TestNG to execute test files down the line rather than alphabetical


#1

I realized while running through the quick start guide that Webtestit was running each @test function in the test file in alphabetical order. Although in my test file I have the test case “SearchForItemTestCase” BEFORE “AddItemToCartTestCase” it would always execute the “AddItemToCartTestCase” BEFORE “SearchForItemTestCase”.

I built my own PO’s and test file for our web based applications and the behavior was the same. It appears to execute alphabetically through the test file instead of in the order which they are scripted.

I tried adding priority to each @test function (i.e. @Test(Priority = 1) and it DOES execute in the priority affinity defined. However, if set to Run all test files and if I have priority set in 2 test files, it will run priority 1 tests from 2 separate test files if they share the same priority affinity. (i.e. TestCasew/priority 1 from test file one, then TestCasew/priority1 from test file 2)

Is there a way to define in the pom or script specifically to ignore affinity and just execute as ordered in my script?

Thanks for your help!


#2

Hi @FCarpio
Nice to hear from you!
If you want to execute your @Test’s in a specific order you can indeed add the (Priority = x) parameters as well as (dependsOnMethods={"Test1"} ) if you have some dependent tests, like in your case.

Now this will execute by hitting the Run all test files button, and as you said it will execute all files present in the project, and to avoid this, you can simply hit Run all tests from file codeLens feature and as the name itself says, all the test from your selected test file.

Hence the Run all test files button checks the tesng.xml, another way is to set the “preserve order” parameter in your testng.xml file:

<test name="YourTestName" preserve-order="true">
    <classes>
        <class name="YourTestNGTestClass">
            <methods>
                <include name="AddItemToCartTestCase" />
                <include name="SearchForItemTestCase" />
            </methods>
        </class>
    </classes>
</test>

You can find out more about running tests in specific order here


#3

Is the testng.xml file located @ “C:\Users<UserName>\Documents\Ranorex\WebTestIt<ProjectName>\src\test\resources”?

Any edits I make to that file, whenever I close and reload the project are wiped out and set to default. I’ve tried with local projects and GIT based projects and same result.

Also, I have 2 tests in my project as shown below:
webtestit_tests

When I go into my testng.xml file it ONLY has references for Test1, why is that? (see below)
testng


#4

Hi @FCarpio!
Yes, that is the file location. This file can be edited directly from the application, and it is located in the resources folder in the project tree.
Try to edit the file from the app, but editing outside it should save the changes and it should remain in the saved state, I will try to reproduce this behavior you encountered.
For the other part, the “Test1” is a default name of the test name value and the TestNG template, so not related to your Test1 :slight_smile: You will have to edit the properties of the TestNG.xml file to fit for your project and tests.
Regards