ArtOfTestRunner.exe ignores parts of JSON settings file

Progress® Telerik® Test Studio® Feedback Portal

Create an account Log In

Back to Feed

Request a Feature Report a Bug

Completed

Last Updated: 03 May 2023 16:32 by ADMIN

Liam

Created on: 17 Jan 2023 01:11

Type: Bug Report

ArtOfTestRunner.exe ignores parts of JSON settings file

Hi there,

I'm looking for some clarification on what settings can be changed when passing a settings file to the ArtOfTestRunner.exe.

When I try to run a test list the settings for "Web" appear to follow the values I set as well as being able to change the Execution Delay, however when I try to change the 'RerunFailedTests' value it will still use the value set in the original list.

Example settings.json I used

{	"Settings": {
  "__type": "ArtOfTest.WebAii.Core.Settings", "__value": {
    "Web": {  "__type": "ArtOfTest.WebAii.Core.Settings+WebSettings", "__value": {
        "RecycleBrowser": false,
        "DefaultBrowser": 12,
        "BaseUrl": "https://BaseUrl.com/qa",
        "KillBrowserProcessOnClose": false,
        "KillBrowsersBeforeStart": false
    }},
    "ExecutionDelay": 0,
    "RerunFailedTests": true
}}}

How can I pass in the RerunFailedTests value from the settings json file?

Thanks for your help,

Liam

4 comments

ADMIN

Shtilianov

Posted on: 03 May 2023 16:32

Hi,

The problem is fixed and the fix will be available with the next release. All settings in the json file will be applied correctly in the CLI Runner.

Regards,Miroslav Shtilianov Progress Telerik

Virtual Classroom is the free self-paced technical training portal that gets you up to speed with Telerik and Kendo UI products including Telerik Test Studio! Check it out at https://learn.telerik.com/.

ADMIN

Shtilianov

Posted on: 20 Jan 2023 08:05

Hi Liam,

Thank you for the feedback and the correction. Spot on.

I'm sharing the PS script version with the quotation marks included for future references.

Regards, Miroslav Progress Telerik

Virtual Classroom, the free self-paced technical training that gets you up to speed with Telerik and Kendo UI products quickly just got a fresh new look + new and improved content including a brand new Test Studio course! Check it out at https://learn.telerik.com/.

Attached Files:

RunTests.zip

Liam

Posted on: 18 Jan 2023 15:16

Hi Miroslav,

Thank you for your help. I was able to use the supplied script to modify the rerunfailedtests value directly as you said.

There was a slight issue in the script that I would like to point out for future use.

Line 65 ((Get-Content -path $fullTestListPath -Raw) -replace "RerunFailedTests: ^(true|false)$" , ("RerunFailedTests: "+ $rerunSet) | Set-Content -Path $fullTestListPath)

Was not able to find the setting in the file as there were missing quotation marks around RerunFailedTests. To fix this I used the line found below:

((Get-Content -path $fullTestListPath -Raw) -replace '"RerunFailedTests": (true|false)' , ('"RerunFailedTests": '+ $rerunSet) | Set-Content -Path $fullTestListPath)

ADMIN

Shtilianov

Posted on: 17 Jan 2023 15:04

Hello Liam,

Thank you for sharing detailed description of the encountered issue. I was able to test it on my end and it turns out there is a problem with the custom settings json and the rerun property. It will not override the property in the test list as it's supposed to happen.

Therefore I converted this discussion to a public bug report - this is the item on the public portal. You can use the Follow button to get notifications on any changes in its status.

Meanwhile you can try using the attached powershell script to achieve what you need. The script is directly replacing properties in the target test list. There are different parameters you can set and change. Like, for example, in the current case the below line will directly set the rerun on failure and set it to true:

\RunTests.ps1 -testListName <yourTestListName> -rerunSet true

The script is designed to be put in the root folder of the project. You can also use it for other settings, originally it was designed to replace and switch the baseURL of a single test list when using the CLI tools and CMD in CI.

Regards, Miroslav Shtilianov Progress Telerik

Attached Files:

RunTests.zip