Receiving error 104 via Jenkins execution and "project is newer than version of Webtesit"


#1

Hi all:
So we updated from version 1.0.10 to 1.0.11. We have a world of trouble. Here are my steps:

  1. Opened Webtestit on all automation assets and let it update and install.
  2. Opened each one of our projects and updated the code and repushed to origin.
  3. Ran a test project and got compile errors.
  4. Removed all Java off machine (both oracle and openjdk)
  5. Reinstalled OpenJDK 14 on machine and let it set Java_home variable.
  6. Tested and all was great inside gui.
  7. Went to Jenkins and ran one of our Jenkins webtestit projects. Got error 104.
  8. Went to machine where Jenkins is pointed. Verified that it indeed had the updated version. Ran the bat file that points to the same project that Jenkins was pointing to. Received error that project is a newer version than Webtestit. Verified again that the new version was installed.
  9. Deleted and Reinstalled Webtestit. Verified again that the new version got installed. Ran bat file again. Got the same error.

I am at a loss. I don’t know how to fix this. Any ideas? Please note that this has brought our Selenium testing to a standstill. This is a work stoppage. Any thoughts as to not requiring us to update and let us opt in?

Thank you.


#2

Hello @micheleaz,
the error “project is a newer version than Webtestit” should only appear in case the version of the project and the version of the webtestit installed are different.
You can verify if the migration was done by cheking the webtestit.json file and see if the version is the same as the webtestit version installed. Also check if after you updated every project, the webtestit.json file was pushed into the repository correctly.

Let me know if this helps.


#3

To make it easier to compare, you can check the webtestit version on the machine where you are running the tests and getting the error with this command:

<path-to-webtestit>webtestit.exe --version

the result should be 1.11.0

After that, check if inside the project you are running the webtestit.json file contains the line:

version: "1.11.0"


#4

Hi! I got both file versions-- both the webestit.exe and the json file are 1.11.0. I will include screen shots. Please remember that the issue is not through the GUI but on CLI.
Thanks!
Michele


#5

Hi @micheleaz thank you for your answer.
I have to ask you to make one more test, so we can understand better the issue.
Can you try to run the test locally using Webtestit GUI and using CLI but on a different machine, like one that you use for development or similar? This way we can figure out if it is a problem with the jenkins machine or a generic issue related to the project.


#6

Hi! So I dug into the issue and got us fixed. Here is what I found:

  1. Tried the above steps on Automation1 and GUI worked and CLI worked. I looked at what was different on Automation1 versus Autoserv4 where the tests are run. Basically the solutions on Automation1 were updated manually from within the GUI. The second difference is Automation 1 is Windows 10 enterprise and Autoserv4 is Windows server 2016. The code on Autoserv4 where the CLI is pointed is a part of the Jenkins workspace and is updated through the Jenkins job. So here is what I did to fix the issue

A. I wiped out All Jenkins workspaces and had Jenkins clone the workspace again from the GIT repository.
B. I noticed there was a good bit of weirdness as far as the upgrade from 1.0.10 to 1.0.11 on the Windows Server 2016 box. It looked like it would partially update. Inside the GUI it would say the version number was 1.0.11 but when I went to add/remove programs and looked there, the version number was 1.0.10. The only way I could get it to fully update was hit File-Restart after the update is downloaded and then the update triggers and installs and everything is updated.

So here is where I think some bugs are buried:

  1. I had to delete and reinstall all JDK’s on the machine for Webtestit to find JavaC. Mind you, everything was working under 1.0.10 and then broke immediately on all machines under 1.0.11.
  2. There seemed to be some file version weirdness in GIT and the Webtestit files when they were checked in and pulled again from GIT. Wiping out the repo on the test machine and letting Jenkins clone a clean copy seemed to fix this.
  3. Install on Windows Server 2016 seems to have bugs.

Management really wants to implore you to let us turn off automatic updates. We realize that we have a pretty complicated CI system and to save us downtime, we need to be able to stand up new versions on test machines so we can minimize downtime. We just can’t have this downtime and churn with every major update.


#7

Hey there @micheleaz,

thanks for digging into this and letting us know about your findings. This sounds like a super unfortunate series of events that came together. Especially the partial update sounds super strange. Tbh we also haven’t verified WinServer 2016 as a target machine which by itself could be the host of additional issues.

As for automatic updates. We realize that these may create issues, as every update of every software can, and will certainly have to think about a way to tackle this. Blocking the download though is just one part of the story. The motivation behind this factor was that in an environment with multiple concurrent users on different machines, if one would update and the rest stays on the old version you run into sever trouble where the other members on older versions add modifications which afterwards can be only uncovered with serious troubles. So in order to reduce both support overhead on our customers and our owns side we decided to bite the bullet and force updates. We will give this more proper thinking to come up with something better though.

With regards to a complicated CI environment I think we can sing a story as well, having a central Azure DevOps ALM and a bunch of staged nodes across every platform. And to conclude the story, with every release we’re planning to do, we lose at least a day to pull in major dependency updates and fix those on the fly. So we clearly hear your and feel your pain and will try the best to reduce it for you as good as we can.

Thanks again for your efforts and continuous great feedback @micheleaz. You’re helping us shaping a better overall product this way.