TestList.Tests in test extension is null when running a dynamic test list

Progress® Telerik® Test Studio® Feedback Portal

Create an account Log In

Back to Feed

Request a Feature Report a Bug

Declined

Last Updated: 15 Dec 2023 09:50 by ADMIN

ADMIN

Cody

Created on: 30 Aug 2013 18:35

Type: Feature Request

TestList.Tests in test extension is null when running a dynamic test list

If you write a test extension implementing IExecutionExtension, hook into the OnBeforeTestListStarted method you normally can get the list of tests to be executed using TestList.Tests. But if the test list being executed is dynamic (instead of static) AND you are using the command line runner (ArtOfTest.Runner.exe) this field is left at a null, preventing you from obtaining the list of tests. I expected to obtain a list of tests that matched the filtering of the dynamic test list. If you use the IDE and execute the dynamic list locally, this field is populated as expected.

This is 2013 R1 specific. I don't know if 2012 R2 has this problem.

5 comments

ADMIN

Konstantin Petkov

Posted on: 16 Sep 2013 14:02

Hi David, 

Yes, the project is scanned for tests matching the given criteria which doesn't help in the context of test execution extension, thus the issue you raised. As far as I see it's more of updating the correct workflow during runtime so that we update it instead of adding duplicate calls just for the sake of addressing this scenario. 

I definitely need the responsible developers to research this change more though before I can give you some estimate. I hope the team can look into this one in the coming weeks.

Thank you for your patience on the matter!

David

Posted on: 11 Sep 2013 00:16

Hi Konstantin -

I appreciate the concern for performance, but wondering why this would be extra work - seems like if you are running a dynamic test list you would have to scan regardless?

  -- David.

ADMIN

Konstantin Petkov

Posted on: 02 Sep 2013 13:29

Hi David,

I'm sorry, I don't see any public API that can help retrieve the tests besides this property. 

The tricky part here is to ensure implementing that will not hurt the performance of test list execution. Loading the tests from a project explicitly may turn to be quite expensive in terms of time and memory, especially for large projects. We need to have access to all tests to filter according to the dynamic test list rules.

Regards!

David

Posted on: 31 Aug 2013 12:27

Hi Konstantin -

Is there an alternate way to get access to the list of tests in the extention "on before test list started"?

    -- David.

ADMIN

Konstantin Petkov

Posted on: 31 Aug 2013 07:18

Hello,

TestList.Tests has never been designed to give something useful in the context of dynamic test lists. Dynamic test lists rely on the Rules you specify when you create the test list in order to calculate the tests to be executed runtime.

Thanks!