Unit test woes

Feb 20, 2012 at 4:14 PM

I've been struggling for two days to get something working with unit tests on a project.  I've tried NUnit (w/Silverlight extension), xunit.contrib (w/Silverlight), MS Test, ReSharper, Silverlight Tools Test support, etc.

All of them fail for me miserably in different ways.  I'm using VS 2010, and just trying to get tests to run successfully in the IDE.  The ultimate goal is to get them running in a CI server, but that is secondary to just being able to run them in anything that works.

I'm using Simple MVVM Toolkit - my build of that is not the most recent, but I'm not sure if it will make a difference or not.  As mentioned, I've tried many things, and none of them worked.  It seems people have the most success using MS Test with the Silverlight support built into the tools project, so here's where I'm at:

Let's call my Silverlight Project Foo

1) Right-click solution, add new project, select Silverlight Unit Test Project, name it Foo.Test

2) Add reference to Foo in Foo.Test

3) Add a mock service class for one of my view models:

using Foo.Services;
using SimpleMvvmToolkit;

namespace Foo.Services
{
    [ServiceAgentExport(typeof(IFooService), AgentType = AgentType.Mock)]
    public class MockFooService : IFooService
    {
        public void GetLoggedInUserId(Action<string, Exception> completed)
        {
            throw new NotImplementedException();
        }

        public void GetVersion(Action<string, Exception> completed)
        {
            throw new NotImplementedException();
        }
    }
}

4. Create a Test class as follows:

using Microsoft.VisualStudio.TestTools.UnitTesting;
using Foo.ViewModels;

using SimpleMvvmToolkit;

namespace Foo.Test
{
    [TestClass]
    public class FooViewModelTest
    {
        readonly InjectedViewModelLocator _locator = new InjectedViewModelLocator();

        private FooViewModel _vm;
        
        [TestMethod]
        public void TestConstructor()
        {
            _locator.AgentType = AgentType.Mock;
            _vm = _locator.FooViewModel;

            Assert.IsNotNull(_vm);
        }
    }
}

5. Now I try to run the test.  I'm using ReSharper which adds buttons in the editor side-bar to launch the tests with MS Test, so that is what I do, and here is what the result is:

TestConstructor : Failed

Unable to create instance of class Foo.Test.FooViewModelTest. Error: System.TypeInitializationException: The type initializer for 'Foo.InjectedViewModelLocator' threw an exception. ---> System.Exception: Catastrophic failure (Exception from HRESULT: 0x8000FFFF (E_UNEXPECTED)).
at MS.Internal.XcpImports.CheckHResult(UInt32 hr)
at MS.Internal.XcpImports.Deployment_GetCurrent(ref IntPtr pDeployment)
at System.Windows.Deployment.get_Current()
at System.ComponentModel.Composition.Hosting.Package.get_CurrentAssemblies()
at System.ComponentModel.Composition.Hosting.DeploymentCatalog..ctor()
at FooInspection.InjectedViewModelLocator..cctor() in InjectedViewModelLocator.cs: line 64
--- End of inner exception stack trace ---
at Foo.InjectedViewModelLocator..ctor()
at Foo.Test.FooViewModelTest..ctor() in FooViewModelTest.cs: line 11

Any ideas on what is going wrong here.  Like I said, I've tried nearly every alternative to get unit testing to work, and they each seem to fail spectacularly in their own unique way.  Has anyone else gotten this to work, and what did you do differently?

Thanks in advance

Feb 20, 2012 at 5:22 PM

Second try: I created a new SimpleMVVMRiaServices project using the template, and ran the tests in that project by setting the test project to the default and F5, and it runs successfully in the browser.

So, I dumped my own Test project, and generated a new one.  I created a new mock following the same pattern in the demo project, and created a test that was identical to the ItemViewModel test in the demo project - except it was testing my own view model which had a similar service method to fetch a collection of entities.

Given all that, if I set the test project to default and launch it, the browser comes up, asks if I want to run all the tests, and then an exception is caught: 

An unhandled exception ('Unhandeld Error in Silverlight Application 
Code: 4004
Category: ManagedRuntimeError
Message: System.InvalidOperationException: The current instance of WebContext is not available.  You must instantiate a WebContext and add it

Normally, the WebContext is bootstrapped in App.xaml file (this is how it is done in the regular Silverlight project), but I see that this is not done in the App.xaml that is generated for Silverlight Unit test projects.  I'm wondering if the reason this is thrown is b/c I do some things with WebContext in some of the other view model classes that are created by the locator -- but then again, I thought those were lazily created (only when the property is referenced)?

I'm going to take a wild guess that WebContext can't be used in Silverlight Unit Testing Framework?  Even if I track down where this is happening, I guess it still doesn't solve my problem.

Coordinator
Feb 20, 2012 at 6:22 PM

With a RIA Services app, you can't start the client app.  You can only start the web host project.  The client is linked to the web app and won't run on its own.

Feb 20, 2012 at 6:30 PM

Hi Tony, maybe there was some misunderstanding...I'm simply following your instruction in the SimpleMVVMRiaServices sample project that is generated with the project template..in the test project, this README =>

Third ReadMe for Simple MVVM with WCF RIA Services
To perform Silverlight unit tests, simply set the Test project as the startupproject and press Ctrl+F5 to start it.  Click the button to run all tests.The tests should pass.

This works in the demo project.  It launches the app, and the browser, and runs the tests in the browser.  In my project it fails somewhere b/c WebContext is being referenced.  So now I am trying to find out how to carve that code out or mock it.

There are so many different options for testing...so many frameworks, and sadly none of them work out of the box.  The only one I haven't tried is SilverUnit which requires a $700 dependency from TypeMock.

It doesn't help that people are abandoning ship on Silverlight in droves...meaning there isn't much further dev in tools / frameworks, especially for testing.  Makes me really regret the choice to use this tech in the first place.

Feb 20, 2012 at 7:13 PM

Finally got this working.  In one of my ViewModel constructors I was hooking into the WebContext.Current.User.LoggedOut += etc..

Commenting this out made the exception go away.  I also found that if I added a new WebContext() to the App.xaml.cs constructor 

ApplicationLifetimeObjects.Add(new WebContext()) 

..the problem also goes away, but I'd rather not have that.  I'll find another way to hook into the LoggedOut event.

That said, I kinda hate the browser-based tests.  I've done stuff in FlexUnit and GWTUnit before, and they also had similar things.  Browser based tests are horrible simply becase they take so long.

The good news is you can use StatLight: http://statlight.codeplex.com/documentation

Download it, point the binary at your .xap file and it runs them on the command line, and you can configure to re-run on a rebuild, and it plugs into CI servers like TeamCity.  Finally, something positive in the world of Silverlight app testing :)

Finally, I think the Service Locator pattern is great and all, but it isn't really how I like to do my testing...I find it too limiting.  You can't easily change the behavior of your mock.  For example, you write your mock to return a collection of length 3 entities, but what if you also want to test what your code does when it returns empty or null?  The behavior is hard-coded into the mock.

I prefer mock frameworks which allow you to inject dynamic behavior right in the test case.  I come more from the Java world, and I used to use PowerMock and EasyMock heavily.  I know there are mock frameworks out there like this for .NET, and now I have to go find out which one is the best for me.

If anyone comes here trying to figure out what testing solution is the best for Silverlight -- after my two days of exhausting all the different options:

StatLight + MS Test + Silverlight Toolkit (Silverlight Unit Test Project template) seems to be the way to go.

I'd prefer NUnit over MS Test, but I did not have much luck with it.