Definitive Guide To Parallelization in C# [MsTest,NUnit,SpecFlow]

^{What is parallelization?}

^{Parallelization is the process of running tests simultaneously on different environments. Such as different browsers or operating systems. It is very useful because it can dramatically decrease the test execution time.}

^{I’ll show you three of the most popular test parallelization frameworks you can use in your C# test automation framework:}

1. NUnit
2. MSTest
3. SpecFlow

^{Parallelization with NUnit}

^{NUnit is one of the most popular testing frameworks for .NET languages. It is open-source, easy to understand, and has user-friendly attributes.}

^{How to set up parallelization in NUnit}

^{The first thing you need to do is define the project’s parallel scope. Secondly, the level of parallelism, i.e., the number of tests that can be run in parallel, in your AssemblyInfo.cs class. It should look like this:}

^{Consequently, add the [TestFixture] attribute to the test class. And finally, use the attribute [Parallelizable] in the test fixture.}

^{With NUnit, the parallelization will be done at the class level. The class attributes will look like this:}

^{This attribute means that the test and all its descendants may be run in parallel with others at the same level.}

^{You can read more about the scopes of test parallelization in NUnit in their official documentation.} 

^{Advantages of using NUnit for parallelization}

^{The greatest advantage in using NUnit for test parallelization is that is allows cross-platform and cross-browser testing. For instance, here’s a quick example of defining various configurations:}

^{And then how to use them use them in the test class:}

^{This will result in tests being run in parallel on the different platform and browser configurations.}

^{I also have a sample project here you can check out.} 

^{NUnit limitations}

One of the main limitations of NUnit is that it does not run test methods in parallel, only test classes. Methods inside a class will be run one after another. To parallelize with NUnit at the test method level, you can set up your tests like this. However, I warn you, it’s a bit of extra work. You might not find it worth the time:

---

^{Parallelization with MSTest}

^{MSTest is Visual Studio’s built-in test library. Its greatest advantage is that it allows us to run tests in parallel at the method level.}

^{How to set up parallelization in MSTest}

Setting up MSTest for running tests in parallel is very simple and straightforward. Just like we did for NUnit, we need to add the Parallelize property in the AssemplyInfo.cs:

[assembly: Parallelize(Workers = 100, Scope = ExecutionScope.MethodLevel)]

The above property instructs our code to run up to 100 test methods in parallel. In addition, there’s no need to do anything special in the test classes. Why? Because the parallelization is already set up and the test methods will run in parallel:

Advantages of using MSTest for parallelization

The biggest advantage of MSTest is that it allows parallelization at the method level. As opposed to the other frameworks which only allow parallelization at the class level. So, for example, If you have 100 test methods in 5 classes, MSTest will let you run 100 tests in parallel. However, with NUnit, you can only run the 5 tests in parallel, one from each test class at a time.

It’s also very easy to configure – just one line of code, and you’re ready.

^{MSTest limitations}

^{As opposed to NUnit, MsTest can run at the method level. However, not at the class level. This makes cross-browser or cross-platform testing more difficult to set up.}

^{If you’re not sure what to choose between NUnit and MSTest, you can check out this article}

---

^{Parallelization with SpecFlow}

^{For the BDD fans out there, SpecFlow is a great C# framework. Therefore, it allows writing the test scenarios in plain English. It uses the Gherkin syntax (Given – When – Then), and then defining the step definitions in C#.} 

^{How to set up parallelization in SpecFlow}

SpecFlow is more complex than MSTest and NUnit when it comes to parallelization. Most importantly, you need to do is to set up a Hooks class that should look like this:

Next, you need to configure a DriverSetup class. Here, you have the [BeforeScenario] method, where you initialize the driver and register it as an instance.

Lastly, in the step definition class, you need to set up the parallelization attribute:

[assembly: Parallelize(Workers = 100, Scope = ExecutionScope.ClassLevel)] 

The parallelization is done through MSTest, so this part is done exactly the same way. This means you can add this configuration to the AssembleInfo class instead of the step definition.

With the above setting, 100 tests can be run in parallel. However, with SpecFlow, parallelization is done at the class level. In other words, only feature files can run in parallel.

You can check out this Git repository for a project sample with SpecFlow parallel tests.

Advantages of using SpecFlow for parallelization

If your project is already a BDD project that uses SpecFlow, the advantage is that running feature files in parallel saves a lot of execution time.

^{SpecFlow limitations}

In conclusion, the biggest disadvantage, I would say, is that it is so difficult to configure.

Apart from the complex configuration, SpecFlow parallelization only allows feature files to run in parallel and not individual tests.

What do you think?

Which testing framework do you prefer and why?