CLI Reporting-- Is PDF available?


#1

Hello:
So I understand the following:

  1. In order to run in a CI environment, we are going to have to install Webtestit on the server. Thats done.
  2. We know that Jenkins can invoke Webtestit via the command line and get a 0 back for success.
  3. What we want is to also have the PDF available so we can attach it to the email that Jenkins sends out after a run. Is this possible?
    Thanks!

#2

hi @micheleaz!
By this you mean to have a report in the PDF format that you need to send out with Jenkins?
In the CI command –report-file-destination="%WORKSPACE%\testresults" specifies the location of the test report folder, but unfortunately, thre is no option to specify the format of the report file or convert it to PDF other than using the IDE (After opening the report, there is an option to save the report as a PDF).
Now we do have something planned regarding the report file formats and conversion. We will keep you updated on this topic.


#3

Hi @smatijas!
Is there a JUnit report created or XML is the only thing that is generated? I am trying to figure out what exactly we can report back after a run currently. When a 0 is generated by the command line, where is this result sent?
Thanks!
Michele


#4

Currently, only an XML type report is being generated. As you can see in this topic, Ranorex Webtestit is returning the status code of your test run, and code 0 meaning success. Regarding your request of generating PDF type reports from CLI run, we added this task to our backlog, so you can expect this feature to be available in some of the upcoming releases :slight_smile:


#5

Hi @micheleaz,

the reason for 0 as return is within a CI environment, those tools most of the time expect an external tool triggered to return an error code, where 0 is the universal indicator for ERROR_SUCCESS (no errors). So your CI server would with that know that the step succeeded and continue with the next step or end the pipeline run successfully.

By default Webtestit just creates JUnit files as a result. You can find these in the /reports folder. The reason for that is that most CI servers (Jenkins, TFS, …) have means to interpret JUnit files and create their dashboards from it. Alternatively tools like Allure Report also make use of JUnit as the format to generate HTML reports. So as you can see we picked that as it is kinda the most used generic data format for test results.

Now as @smatijas explained we’re already investigating a solution to create PDFs along the XML as an output. That has impacts on the CLI though as it requires UI access (even if hidden) which is fine with the current approach of how our CLI works by being part of the whole IDE. Splitting things up though would create troubles for that feature.
So long story short, we’re on it but it’ll might take a bit longer to find the best way forward.