Hi Anton, it’s a pleasure to be talking to you today. Thanks for taking the time to talk with us. Can you tell us a bit about yourself?
I saw that you’ve been working on a new project recently, can you please tell us more about this project?
Yes, you are right. The name of the project is Bellatrix. It is a fifth-generation full-stack test automation framework. It is a set of .NET libraries and tools that help you to automate web, desktop, mobile and API applications.
What problems does this solve?
Recently I wrote a comprehensive article about the problems that the modern test automation frameworks should be able to solve. There you can read for more than 40 issues that we should think about. I started the project over one year ago. Part of the reason how I came out with the idea was that as a QA consultant many companies wanted from me to develop for them test automation framework. However, I thought that writing the same thing over and over again and supporting it is not scalable enough. So, I came up with the idea of creating a test automation framework as a service. Which means that the test automation framework is written once in the best possible way. Then it is covered entirely with tests (we have over 6000 automated tests covering every single feature). The next part is to have comprehensive documentation and starter kits/demos. It is designed in a way that it can be incorporated, customised for every context. Moreover, you can quickly add new features on top of it. Most people didn’t realise that the most significant part of the tests maintenance is fixing the existing tests, upgrading tooling and so on. This is the thing that we handle for our clients. Moreover, other services that we offer are analysation of existing automation practices, rewrite of existing tests, training and development of new custom features. I like to make an analogy with custom-tailored suits that are made to measure perfectly for you.
How does one get started using Bellatrix and what are the prerequisites?
The tool has a free version (you can use all features for unlimited time up to 100 tests). So, you need to download it from our website. You will receive an email containing the installation executables. The installation is quite simple, just run the executable.
The tool is cross-platform which means you can write tests on Windows, Mac and Linux. You can use Visual Studio Community or Visual Studio Code.
We provide a little bit more goodies for Visual Studio- such as project, item templates and code snippets. For both of them, we provide the so-called Starter Kits for Web, Desktop and API. These are projects with demos on how to use each feature of the framework with fully working examples. There you can find detailed information on how each functionality works, and as a bonus, there are multiple exercises for you.
Can you provide us a code example of how to get started with Bellatrix?
This is how to create starter kit from the command line:
bellatrix create --template=Bellatrix.Web.GettingStarted --name=YourProjectsName --location="PathWhereToCreate"
You can create the same using Visual Studio project templates (File -> New -> Project).
Web
API
Desktop
Is it true that Bellatrix allows us to more easily create Page Objects? What problems does it solve for an automation engineer?
Yes. We have created over ten different item templates for Visual Studio.
This means that instead of copy-pasting existing page objects, the whole code will be generated for you. Sometimes you can miss deleting some old code or forget to rename something correctly which can lead to further problems. Anyhow this features saves time and you can create the necessary code errorless much faster.
Moreover, we use a modified, improved version of the pattern where we use 3 partial classes for separating action methods, element locators and assertions.
This way the maintainability of the tests using the page object is much more comfortable especially for large pages. If you have to change the locator strategy for a specific element, you can locate it much faster. The separation makes the navigation through methods much easier and improves the readability of the code significantly.
Want to do it yourself? I wrote an article about how to create page object item templates for Visual Studio.
Can you provide a code example of a test using the page object with Bellatrix?
The first method doesn’t use page objects. Then we rewrote the tests using Bellatrix page objects. As you can see the second test is much shorter and more readable.
Bellatrix-AssertedNavigatablePage – contains 3 files- one for actions, one for element declarations and one for assertions (all of them make one-page object)
Bellatrix-NavigatablePage– one for actions and one for elements (all of them make a one-page object)
Bellatrix-Page– one for actions and one for elements (all of them make a one-page object), don’t have methods for navigation
For most pages, you will use AssertedNavigatablePage. You want to be able to navigate to these pages with URL, perform some operations on them (click or type something somewhere). The test to be a test it should assert something, so this is why you need the assertion files.
However, there are some pages which are just part of a larger workflow such as the middle pages of a shopping cart. You don’t have the need or the way to navigate to them through URL. You don’t need to assert anything since the important stuff are displayed on next pages. In this case, you can use the simple Page.
I also see that you provide screenshots on fail, which is always very useful. Can you please provide a code example of how to make this happen? How does the screenshot look after it is taken? Where is it stored?
Documentation on
The only thing you need to add to your test is the class attribute ScreenshotOnFail.
[ScreenshotOnFail(true)]
All Bellatrix test projects come with a testFramework.json settings file where you can
"screenshotsSettings": { "isEnabled": "true", "filePath":
"C:\\\\\\\\\\\\\\\\Troubleshooting\\\\\\\\\\\\\\\\Screenshots" }
Read more about how the magic happens, we use JavaScript.
This is how a standard image made with WebDriver looks:
As you can see you see only the visible part of the screen, which is not very useful isn’t it?
This is how the full-page screenshot with Bellatrix looks like:
Also you have static analysis. I always hoped for an easy way to do static analysis with my test automation. How does one get started doing static analysis with Bellatrix? Exactly what steps do we need to take to make this possible in our automation? After we implement this, what will be the result?
To get started using the static analysis with Bellatrix you need to create a test project from one of the ready-to-go Bellatrix templates. There we have pre-installed the required libraries and set some default rules.
I wrote several articles how you could integrate static analysis in your test automation.
The first tool we use is StyleCop. In the article, you can find what you need to do to install it and what are the various benefits. In the next two posts, you can see the must-have default quality rules you should enable. They are the ones we included in Bellatrix by default.
The second tool that we utilise is EditorConfig. Read how to install it and again in next publications find the most important rules you should enable. We use the same ones for Bellatrix.
The two tools are complementary to each other since they catch slightly different “smells”/problems in your code.
They produce warnings and “warn” you when there is a problem with your code so that you can fix it fast.
You can view all warnings in the Error List window. Once you double-click on a warning, the file where the issue is situated is opened and selected.
Also you have static analysis. I always hoped for an easy way to do static analysis with my test automation. How does one get started doing static analysis with Bellatrix? Exactly what steps do we need to take to make this possible in our automation? After we implement this, what will be the result?
To get started using the static analysis with Bellatrix you need to create a test project from one of the ready-to-go Bellatrix templates. There we have pre-installed the required libraries and set some default rules.
I wrote several articles how you could integrate static analysis in your test automation.
The first tool we use is StyleCop. In the article, you can find what you need to do to install it and what are the various benefits. In the next two posts, you can see the must-have default quality rules you should enable. They are the ones we included in Bellatrix by default.
The second tool that we utilise is EditorConfig. Read how to install it and again in next publications find the most important rules you should enable. We use the same ones for Bellatrix.
The two tools are complementary to each other since they catch slightly different “smells”/problems in your code.
They produce warnings and “warn” you when there is a problem with your code so that you can fix it fast.
You can view all warnings in the Error List window. Once you double-click on a warning, the file where the issue is situated is opened and selected.
For someone starting with Bellatrix, do you have any special tips and tricks, since you are the creator?
The most valuable tip I can give is to play with the starter-kits and read all of the comments. We put much thought building them, and they should be more than enough for anyone to start writing even more advanced logic such as customisation of the framework.
Also, for the next one month, we provide free email support so they can use that too.
Does Bellatrix have another amazing feature that I didn’t talk about? If so, please tell us about it, how to get started and what problem it solves.
There are at least a couple, but I think the one that worth mentioning most is the layout testing feature.
It helps you to test the responsiveness of your website or even a desktop app.
Here is an example of how we use it:
Bellatrix resizes your browser and app. Then finds the elements and based on their coordinates make the various calculation and assertions whether some element is above, below, left, right, inside, etc. to another one. Bellatrix comes with 120+ layout assertion methods.
Anton, again, it was a pleasure talking to you today on Ultimate QA. It’s always great to meet a passionate individual that is driving the automation community forward. I’m looking forward to trying Bellatrix and using it to improve my test automation. It seems like a dream solution for .NET. Talk to you in the future. Please come back and update us with new information about Bellatrix.