As more and more results are added to the storage service, it seems to significantly impact how long it takes for tests to start executing on a remote execution server. Once customer reports that in the beginning it took 5-7 minutes for execution to begin. After their DB file grew to 5GB, it now takes 20-25 minutes for execution to start. The size of the project did not change. It took them only one day of doing nothing but running multiple test lists to see this behavior.
Load traffic is not captured when using an existing web test with BaseURL set in the settings of the Project. Here is a short video demonstration of the problem: http://screencast.com/t/3tj1TsX03JLY
Steps to reproduce: 1. Navigate to an app containing input text element 2. Type in the input text element 3. Enable 'SimulateRealTyping' step property 4. Run the test in MS Edge Log exception: [02-23 16:02:33,ArtOfTest.Runner.exe(6096:4),Error] ExecutionEngine.CatchExecuteStepException() : EXCEPTION! (see below) Outer Exception Type: ArtOfTest.WebAii.Exceptions.ExecuteCommandException Message: ExecuteCommand failed! InError set by the client. Client Error: Unknown error BrowserCommand (Type:'Action',Info:'NotSet',Action:'InvokeJsFunction',Target:'ElementId (tagName: '',occurrenceIndex: '-1')',Data:'document.getElementsByTagName('input').focus()',ClientId:'8c513c4e-1f29-4088-a6d9-23dbbc6c3dbc',HasFrames:'False',FramesInfo:'',TargetFrameIndex:'-1',InError:'True',Response:'Unknown error') InnerException: none. HRESULT: 0x80131500 (Official ID (if app.) = COR_E_EXCEPTION, Error Bit = FAILED, Facility = FACILITY_URT, Code = 5376) Call Stack: at ArtOfTest.WebAii.Core.Browser.ExecuteCommandInternal(BrowserCommand request) at ArtOfTest.WebAii.Core.Browser.ExecuteCommand(BrowserCommand request, Boolean performDomRefresh, Boolean waitUntilReady) at ArtOfTest.WebAii.Core.Actions.InvokeScript(String script, Boolean refreshDom) at ArtOfTest.WebAii.ObjectModel.Element.GetValue[T](String propertyName, T defaultValue) at ArtOfTest.WebAii.ObjectModel.Element.GetValue[T](String propertyName) at ArtOfTest.WebAii.ObjectModel.Element.Focus() at ArtOfTest.WebAii.Controls.HtmlControls.HtmlControl.Focus() at ArtOfTest.WebAii.Design.IntrinsicTranslators.Descriptors.SetTextActionDescriptor.Execute(Browser browser) at ArtOfTest.WebAii.Design.Extensibility.HtmlActionDescriptor.Execute(IAutomationHost autoHost) at ArtOfTest.WebAii.Design.Execution.ExecutionEngine.ExecuteStep(Int32 order)
For various reasons we often update the identification logic to many elements that are used in our tests. Also very often, it happens that when recording a step against an element that is defined and working (i.e. the criteria is correct), Test Studio would none the less record the action against a new element. I tend to suspect it has to do with the fact that we tamper with the identification logic allot as mentioned above.
Steps to reproduce: 1. Open a project from TFS. 2. Toggle on/off a step feature "Continue on Failure' only once. 3. Check in to TFS, get latest version of project. Expected: The check in to prompt for saving the test files locally and then check in Actual: Only check in dialog is displayed. If you get latest version after the check in the change would not be present.
When performing the following steps to fill and update a popup form window: 1. Press Save_Button. 2. Wait for element 'Field_Country' 'is' visible. Test Studio produces the following error: Unexpected error while waiting on condition. Error: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> ArtOfTest.WebAii.Exceptions.ExecuteCommandException: ExecuteCommand failed! InError set by the client. Client Error: System.NullReferenceException: Object reference not set to an instance of an object.
When executing test lists in a larger project there is huge delay before the test list start executing. There is a long wait between each test in the list as well. Steps to reproduce provided in the internal description.
When using the VS plugin, if you change the code behind file to use a custom base class (which in turn inherits from BaseWebAiiTest), when you double click on a coded step to open it, the code behind file does open, but the all the coded steps in the test "magically" disappear. Doing the same sequence in Test Studio does not have this problem. A project that shows this problem is attached to the internal bug report.
When you change a Page property in the Visual Studio's Elements Explorer, the changes are not applied if you don't have at least one test opened which contains an element from that Page. Visually the Page property is changed, but if you close and reopen the VS solution you'll notice that the changes you made are reverted back to their previous state. However if you have a currently opened test which contains an element from that Page, the test will be marked as 'dirty' and the changes will be saved succesfully.
The test environment: * Master test suite is at about 240 tests and growing. * This is currently divided into 9 test lists of varying sizes. * The execution lab currently is 4 physical desktop machines. I schedule the test lists to run across these 4 machines. When I want to see the results of the test lists, I may be able to look at one or two test list results. But, it appears the application holds all these results in memory. Therefore, as I move through and look the results of each list I encounter out of memory exception and Test Studio crashes. Sometimes gracefully (with the prompt to save work) and sometimes it just disappears. The workaround is to leave the test results tab after viewing one or two lists. However this further slows down and disrupts the process of reviewing the results.
Steps to reproduce: 1. Copy and Paste an existing test case (bound to an Excel datasource) 2. Rename the new copy and save it. 3. Change the data binding or any other step in the new saved copy and save again. 4. Close and open the project. Expected: All changes to be saved in the newly copied and renamed test. Actual: Only the new name is saved but all further changes are not present. Note: On the other hand if the save action in step 2. is missed and all changes are saved with a single save action, all of them are present after reopening the project.
Steps to reproduce: 1. Open Project 2. Copy / Paste a Test 3. Rename the Test 4. Open the copied test and change a test parameter 5. Save the project 6. Relaunch Test Studio and reopen the project 7. Open the same test again 8. The previously edited steps parameter is not changed
In a specific customer application, when we use "Wait for visible" test steps, Test Studio randomly fails the test step with the following exception: ArtOfTest.Common.Design.Exceptions.VerificationWaitException: Unexpected error while waiting on condition. Error: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> ArtOfTest.WebAii.Exceptions.ExecuteCommandException: ExecuteCommand failed! InError set by the client. Client Error: System.ArgumentException: Tag collection is either empty or has less elements than the element occurrence requested. RequestedIndex: '2', ElementLength: '1' Steps to reproduce and a sample test project are attached in the internal description.
Using the framework(not recording), I am trying to get some elements in a WPF RadWindow that I am opening in my application. I use the following code: var window= webaii.ActiveApplication.WaitForWindow("", 30000); var radWin = window.Find.AllByType("myWindow"); What I am doing is to get the Microsoft Window and then find the RadWindow, as it is hosted in it. The Window is found as expected, but then radWin returns 2 RadWindows that are the same. Even if I am not trying to find RadWindow but some element in it, for example Button the find expression returns 2 duplicate Buttons.
When using Frame Tagging, it is not possible to locate elements on a tagged frame when referencing the elements from coded steps. For repro see the attached project in the internal description with more details.
WPF Memory leak when trying to test modal dialogs. When you try to open/close dialog window with testing framework it cause memory leak and child dialog not garbage collected. We do it in loop and for some iteration Telerik testing framework can't find dialog and break by timeout. We found it with windbg + sos dll. Results: Scan Thread 14 OSTHread 25bc DOMAIN(006A72D0):HANDLE(Pinned):1513e4:Root: 039593f8(System.Object)-> 029ccf78(System.Collections.Generic.Dictionary`2[[System.IntPtr, mscorlib],[Telerik.TestingFramework.WpfExtension.WpfCommunication, Telerik.TestingFramework.WpfExtension]])-> 02a5650c(System.Collections.Generic.Dictionary`2+Entry[[System.IntPtr, mscorlib],[Telerik.TestingFramework.WpfExtension.WpfCommunication, Telerik.TestingFramework.WpfExtension]])-> 02a4d990(Telerik.TestingFramework.WpfExtension.WpfCommunication)-> 02a4da38(Telerik.TestingFramework.XamlExtension.ClientProcessor)-> 02a4d9c4(Telerik.TestingFramework.XamlExtension.TechSpecific)-> 029cd424(MainApplication.ChildWindow) It seems that some static dictionary is referenced to ChildWindow which cause memory leak. It seems that problem in WpfCommunicationEntryPoint.Comms. I found only one static dictionary with this signature (problem can be in another place). How to use windbg + sos.dll: 0. Run test from VS MemoryLeakSmokeTestFixture.MemoryLeak_TelericFramework 1. Run windbg 2. File->Attach to process-> choose MainApplication 3. Load sos.dll : .loadby sos clr 4. Fild objects: !DumpHeap -type ChildWindow 5. Find who are referenced to it: !gcroot OneOfTheIntancesAdresses. You are going to see the same results. Sample project in attach Thank you. Looking for advice
After upgrade the project references are all referred to the older version even though the installed version is the latest and the tests are failing with an error loading referenced assembly . When the project references are manually updated , the problem is solved, however this creates huge inconvenience for customers with multiple projects. Exception from TS follows: Situation: Error loading referenced assembly Outer Exception Type: System.IO.FileLoadException Message: Could not load file or assembly 'ArtOfTest.WebAii.Design, Version=2013.1.1002.0, Culture=neutral, PublicKeyToken=4fc62bbc3827ab1d' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040) HRESULT: 0x80131040 (Official ID (if app.) = FUSION_E_REF_DEF_MISMATCH, Error Bit = FAILED, Facility = FACILITY_URT, Code = 4160)
When using scheduling to run tests remotely, when doubleclick on the result in the callendar it shows all test as passed, however the result is shown as failed and the message Can't reach the remote executor exception: Error while copying content to a stream. Refer to the screenshot attached. Screenshots and project files for repro provided in the internal description.