if you use the C# using declaration and have JustMock advanced (elevated) mode enabled, the runtime will throw an InvalidProgramException.

Find below the sample code that demonstrates the issue:

public class TestClass : IDisposable
{
    public void Dispose()
    {
    }
}

[TestClass]
public class Fixture
{
    public interface ITest { }

    [TestMethod]
    public async Task Test()
    {
        ITest mock = Mock.Create<ITest>();
        using TestClass test = new();
    }
}

My team and I have spotted some odd behaviour with the latest version of JustMock (2015.3.929.5) when targeting a Windows Store app.

If we create a mock for an object in a helper method, the mock fails when making assertions for calls to the mock.

The following code illustrates the issue:

        [TestMethod]
        public void ThisWillFail()
        {
            var subject = CreateSubject();

            subject.DoSomething();

            subject.Assert(s => s.DoSomething(), Occurs.Once());
        }

        [TestMethod]
        public void ThisWillPass()
        {
            var subject = Mock.Create<ISubject>();

            subject.DoSomething();

            subject.Assert(s => s.DoSomething(), Occurs.Once());
        }

        public interface ISubject
        {
            void DoSomething();
        }

        private static ISubject CreateSubject()
        {
            return Mock.Create<ISubject>();
        }

In this code, the first test will fail but the second test will pass. The only difference is that, in the first test, we're setting up the mock in a helper method.

We have a "Unit Test Library (.NET for Windows Store apps)" referencing the Telerik.JustMock assembly. I have attached a simple project containing this implementation.

It's worth noting that the same code passes in a regular .NET class library; it only fails in a "Unit Test Library (.NET for Windows Store apps)". It's also worth noting that this worked under an older version of the assembly (2014.3.1021.2).

Any help would be appreciated, as we currently have around 3,000 tests and a good proportion of them set up their mocks using a helper method in this way.

Regards
William Cowell

Considering the following simple test scenario:

public abstract class TestBase
{
    public static TestContext TestContext { get; set; }
}

[TestClass]
public class UnitTest1 : TestBase
{
    [ClassInitialize]
    public static void ClassInitlialize(TestContext ctx)
    {
        TestContext = ctx;
    }

    [TestMethod]
    public void TestMethod1()
    {

    }
}

[TestClass]
public class UnitTest2 : TestBase
{
    [ClassInitialize]
    public static void ClassInitlialize(TestContext ctx)
    {
        TestContext = ctx;
    }

    [TestMethod]
    public void TestMethod1()
    {

    }
}

Attempt to run the tests above with JustMock profiler enabled fails with System.InvalidProgramException. The issue is not reproducible with MSTest.TestFramework and MSTest.TestAdapter packages prior to 3.0.x.

We are trying to run unit tests via xUnit and mocking method occurence using JustMockLite. Method under arrange is creating a underlying Task however, once in a while test fails with following error

build	11-May-2021 14:36:09	               Telerik.JustMock.Xunit.AssertFailedException : Multiple assertion failures:
build	11-May-2021 14:36:09	               1. Occurrence expectation failed. Expected exactly 1 call. Calls so far: 0
build	11-May-2021 14:36:09	         Arrange expression: x => x.CallAsync(IsAny(), IsAny())
build	11-May-2021 14:36:09	               2. Occurrence expectation failed. Expected exactly 1 call. Calls so far: 0
build	11-May-2021 14:36:09	         Arrange expression: x => x.NotifyAsync(IsAny())
build	11-May-2021 14:36:09	               
build	11-May-2021 14:36:09	               ---- Telerik.JustMock.Diagnostics.DebugViewDetailsException : State:
build	11-May-2021 14:36:09	         Elevated mocking: disabled
build	11-May-2021 14:36:09	         
build	11-May-2021 14:36:09	         Arrangements and expectations:
build	11-May-2021 14:36:09	                   Arrangement (id=0) x => x.Load(IsAny()):
build	11-May-2021 14:36:09	                 Met: Occurences must be in [1, 1]; calls so far: 1. 
build	11-May-2021 14:36:09	             Arrangement (id=1) x => x.InitializePolicy(IsAny(), IsAny()):
build	11-May-2021 14:36:09	                 Met: Occurences must be in [1, 1]; calls so far: 1. 
build	11-May-2021 14:36:09	             Arrangement (id=2) x => x.RequestBulkSync(IsAny(), IsAny(), IsAny()):
build	11-May-2021 14:36:09	                 Met: Occurences must be in [any, 1]; calls so far: 1. 
build	11-May-2021 14:36:09	             Arrangement (id=3) x => x.CallAsync(IsAny(), IsAny()):
build	11-May-2021 14:36:09	                 Unmet: Occurences must be in [1, 1]; calls so far: 0. 
build	11-May-2021 14:36:09	             Arrangement (id=4) x => x.NotifyAsync(IsAny()):
build	11-May-2021 14:36:09	                 Unmet: Occurences must be in [1, 1]; calls so far: 0. 
build	11-May-2021 14:36:09	         
build	11-May-2021 14:36:09	         Invocations:

The issue is about interoperability between JustMock and AutoFixture. The unit test run can be 90% faster if all the fixtures are created before the JustMock arrangements.

Here is a sample code that figures out the issue:

[Fact]
public void Slow
{ 
    var fixture = new Fixture():

    var items1 = fixture.Create<List<Item>>();
    Nock.Arrange(() => ItemsRepository.GetItems1()).Returns(items1);

    var items2 = fixture.Create<List<Item>>();
    Nock.Arrange(() => ItemsRepository.GetItems2()).Returns(items2);

    var items3 = fixture.Create<List<Item>>();
    Nock.Arrange(() => ItemsRepository.GetItems3()).Returns(items3);

    var items4 = fixture.Create<List<Item>>();
    Nock.Arrange(() => ItemsRepository.GetItems4()).Returns(items4);
}

[Fact]
public void Fast()
{
    var fixture = new Fixture():

    var items1 = fixture.Create<List<Item>>();
    var items2 = fixture.Create<List<Item>>();
    var items3 = fixture.Create<List<Item>>();
    var items4 = fixture.Create<List<Item>>();

    Nock.Arrange(() => ItemsRepository.GetItems1()).Returns(items1);
    Nock.Arrange(() => ItemsRepository.GetItems2()).Returns(items2);
    Nock.Arrange(() => ItemsRepository.GetItems3()).Returns(items3);
    Nock.Arrange(() => ItemsRepository.GetItems4()).Returns(items4);
}

The test duration should not be dependent on the exact implementation.

Dear Telerik team,
It is nice to have a way to generate mocks.

But it is annoying to have lots of those messages for references in XML comments. I had to turn the feature off. Which might be the case for other customers too.

Maybe you want to have a look into it.

The following code snippet causes a hang in the test execution while being debugged:

Mock.SetupStatic(typeof(TimeSpan), Behavior.CallOriginal, StaticConstructor.NonMocked);
Mock.Arrange(() => TimeSpan.FromSeconds(15)).Returns(TimeSpan.MinValue);

The issue can be temporary solved by disabling the DebugWindow via JustMock extension menu. 

Hello

I generate a syntax tree which I will format with Formatter.Format() from the package Microsoft.CodeAnalysis.CSharp.Workspaces 4.4.0 and .NET 6. A test exists where the formatter is used but when the JustMock profiler is enabled an InvalidProgramException is thrown. When the profiler is disabled everything works fine. It fails on Windows and on Linux.

Exception

  Message: 
System.InvalidProgramException : Common Language Runtime detected an invalid program.

  Stack Trace: 
ContextIntervalTree`2.ctor(TIntrospector& introspector)
FormattingContext.ctor(AbstractFormatEngine engine, TokenStream tokenStream)
AbstractFormatEngine.CreateFormattingContext(TokenStream tokenStream, CancellationToken cancellationToken)
AbstractFormatEngine.Format(CancellationToken cancellationToken)
CSharpSyntaxFormatting.Format(SyntaxNode node, SyntaxFormattingOptions options, IEnumerable`1 formattingRules, SyntaxToken startToken, SyntaxToken endToken, CancellationToken cancellationToken)
AbstractSyntaxFormatting.GetFormattingResult(SyntaxNode node, IEnumerable`1 spans, SyntaxFormattingOptions options, IEnumerable`1 rules, CancellationToken cancellationToken)
Formatter.GetFormattingResult(SyntaxNode node, IEnumerable`1 spans, Workspace workspace, OptionSet options, IEnumerable`1 rules, CancellationToken cancellationToken)
Formatter.Format(SyntaxNode node, IEnumerable`1 spans, Workspace workspace, OptionSet options, IEnumerable`1 rules, CancellationToken cancellationToken)
Formatter.Format(SyntaxNode node, Workspace workspace, OptionSet options, CancellationToken cancellationToken)
UnitTest1.Test1() line 23

Reproduction

You can reproduce this by writing an unit test for that (I used xUnit):

        [Fact]
        public void Test1()
        {
            var classText = @"using System; namespace TestNameSpace.Orders { public class Order
                            {
                                public Guid Id { get; set; }
                            }
                        }";

            var syntaxTree = CSharpSyntaxTree.ParseText(classText);
            var workspace = new AdhocWorkspace();

            var formattedClassText = Formatter.Format(syntaxTree.GetRoot(), workspace).ToFullString();

            var expected = @"using System;
namespace TestNameSpace.Orders
{
    public class Order
    {
        public Guid Id { get; set; }
    }
}";
            Assert.Equal(expected, formattedClassText);
        }
    }

System Info

JustMock

See attachments. We do not use the free edition.

.NET

dotnet --info
.NET SDK:
 Version:   7.0.200
 Commit:    534117727b

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19045
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\7.0.200\

Host:
  Version:      7.0.3
  Architecture: x64
  Commit:       0a2bda10e8

.NET SDKs installed:
  6.0.406 [C:\Program Files\dotnet\sdk]
  7.0.200 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.14 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.14 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 6.0.14 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 7.0.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
  x86   [C:\Program Files (x86)\dotnet]
    registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

 

The issue is demonstrated with the following sample:

[Theory]
[MemberData(nameof(GetMemberDataContext))]
public void ValidParameters_Success(int param1, int param2)
{
	// Arrange
	Mock.SetupStatic(typeof(MyClass), Behavior.Strict, StaticConstructor.Mocked);
        Mock.Arrange(() => MyClass.method1()).Returns(true);

        // Act
	IService service = new Service();
	bool result = service.method2();

	// Assert
	Assert.True(result);
	Mock.Assert(() => MyClass.method1(), Occurs.Once()); // <-- the test fails here because it reports that method invocation occurs twice
}

The issue in not observed if the code is modified in the following way, which indicates behavioral incinsistency:

[Theory]
[MemberData(nameof(GetMemberDataContext))]
public void ValidParameters_Success(int param1, int param2)
{
	// Arrange
	Mock.SetupStatic(typeof(MyClass), Behavior.Strict, StaticConstructor.Mocked);
        Mock.Arrange(() => MyClass.method1()).Returns(true).OccursOnce();

        // Act
	IService service = new Service();
	bool result = service.method2();

	// Assert
	Assert.True(result);
	Mock.Assert<MyClass>();
}

The documentation on Fluent Mocking ends with this statement:

Important

Note that when you use Fluent Asserts only arrangements marked with either MustBeCalled or Occurs will be verified. For other tests you have to use the explicit assert.

What this fails to note is that, while this is true of the function "Assert", there is another function, "AssertAll", which will flag an error if any Arranged function call was not utilized.

On a related note, I left other suggestions for this same page a day or two ago.  I would have liked to leave the above statement using the same feedback utility, but I can no longer find the control that I used to leave those initial suggestions.

Update reference functionality wrongly suggests updating JustMock assembly reference and even more - detects currently referenced one as a lite version, see the screenshot below:

The test run is aborted when a .testsettings is used for executing JustMock tests. The .testsettings contains only a description.

<?xml version="1.0" encoding="UTF-8"?>
<TestSettings name="Local" id="15694c75-be1c-4113-9d42-2cbe1013c41c" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
  <Description>These are default test settings for a local test run.</Description>
</TestSettings>

It seems that the usage of the MSTest element is breaking the execution.

Workaround: As .testsettings are deprecated use .runsettings instead without MSTest element.