Setting up FitNesse for Team Foundation Server

Setting up FitNesse for Team Foundation Server

Introduction

In this article we will describe how we set up FitNesse for .NET for different projects using different versions of the .NET Framework and how to integrate this into a Microsoft Team Foundation Build Server.
The initial aim was that every developer can create & execute FitNesse tests locally on his machine, as part of the Visual Studio project together with the production code. The tests should be checked in into source control together with the code and then also be executed within the build process.
To reduce installation & maintainance effort and reduce overall complexity on the server site, we wanted to use only one FitNesse instance on the build server, that should execute the FitNesse tests of all the projects build with that machine, indepently from the .NET version used by the projects.

Installation

The first step was to download & install FitNesse. Here we followed the excellent instructions of Gojko Adzic, given in chapter 2 of his book Test Driven .NET Development with FitNesse, which is in general a top reference for using FitNesse for .Net.
Hint: Be aware to download the version of FitSharp that fits to the version of .NET that is used by your project!
We also followed Gojko´s example of the “Hello World” test to be sure we had set up everything correctly.
Within our solution we already had a project for Tests. Here we added a subfolder named “FitNesse” where we put in all the FitNesse related items (fixture code, seperate config files for the DB connection to be used, etc.).

On filesystem level we also created a directory called “FitNesse” within that subfolder which we used as the root for our FitNesse-installation.
On the buildserver we installed FitNesse to C:\FitNesse
We then wrote our fixture code fitting for our application, followed by creating the FitNesse tests.

Structure of Tests

The structure of our FitNesse wiki pages & within the solution can be seen in the following screenshots:

 

Reason for the additional “The Tests”-layer is the (relative) paths to the fixture DLLs differ between a local developer´s machine and on the build server.
The configuration is done in the highest layer “FitNesseTfsTest” and the “The Tests” wiki and its subpages are later used within the build. This enables us to use different configurations locally and on the build server. (Another possibility to achieve this would be to pass the the config values as parameters when starting FitNesse. Advantage of our solution is that the values of the configurations are visible within the wiki sites instead of the parameter names) (Maybe topic for a seperate blog post?)

Classpath on the workstation

Classpath on the buildserver

More details on this later on when we explain what is happening on server site.

Be aware of the versions!

Because we had to serve different projects using 3.5 and 4.0 versions of .NET, we also had to use different versions of FitSharp, fitting to the .NET versions. The standard folder name for the FitSharp components within the FitNesse folder is dotnet2, to keep things clear we renamed that folder to dotnet35, respectively dotnet40. This not only avoids confusing, but will makes things also easier on the build server when using both versions within one instance of FitNesse.

Execution

We´ve created the following powershell script to be executed within the build process to execute the FitNesse tests:

    • First the FitNesse tests for the specific project are being copied from the build directory to the folder of FitNesse installation of the build server
    • set-location sets the working directory, so that FitNesse is being started in the right directory; then the FitNesse server is being started in a seperate process; finally we wait three seconds to be sure the server is up and running
    • the FitSharp Runner.exe is being called to execute the tests. (more info on the Runner.exe: www.syterra.com)

; detailed results of the tests can be found in the result.txt

  • The FitNesse server is being stopped
  • The exitcode of the powershell script is being set to the same value as the execution of the Runner.exe
  • All of it is being embraced by a global try-catch-block to ensure the FitNesse java process is being killed and the script ends with an ExitCode different then 0

Intention of this is, that the Runner.exe delivers 0 as exitcode if all the FitNesse tests have passed. The exitcode of Runner.exe will then be set as exitcode of the powershell script and can then be handled by the build process.

Adapt TFS´s build template

To run the FitNesse Tests as part of the TFS build process, we had to modify the build process template.The goal was to call the powershell script at the end of the build process and make the build fail if FitNesseTests failed. Here is the procedure for doing this, it is highly inspired by these excellent Ewald Hofman’s tutorials:

First create a new build process template file based on your active file. To do that, edit the build definition, in the Process tab, show template details, click New… and copy the file to create a new one. Name it FitNesseTestTemplate.xaml.
Open the file created, locate the block “Try Compile, Test, and Associate Changesets and Work Items” inside “Run On Agent”.
Just bellow this block, drag and drop a new sequence block and name it “Run FitNesse Tests”.
This block located at the end of the process is the good place to run our FitNesse tests.
If you started with the default template, your workflow looks like this.

Add the integer ExitCode to the variables of this block.

Now Add an InvokeProcess activity inside the Run FitNesseTests. Its role is to call the powershell executable on the build server with the script we have created.
Click on the InvokeProcess activity and change the properties:

FileName: The path to powershell.exe on your build server .
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe

Argument: The path to the script, here we can use the global MS Build variable SourceDirectory.
String.Format("""& '{0}\runFitNesseTest.ps1'""", SourcesDirectory)

Result: ExitCode
This allows us to get the result code from the fitsharp runner.exe.

To see results from the powershell command inside the MS Build log, drop a WriteBuildMessage activity on the “Handle Standard Output”, pass the stdOutput variable to the Message property and change the Importance property to: Microsoft.TeamFoundation.Build.Client.BuildMessageImportance.High
Do the same for a WriteBuildError activity on the “Handle Error Output” . Pass the errOutput variable to the Message property.
Add an If activity bellow the InvokeProcess activity with the condition ExitCode <> 0
In the Then section add a Throw activity and set the Exception property to:
New Exception(“FitNesse tests failed”).
Thus your build process will fail if the FitNesse tests fail.
Now the sequence block looks like this:

Now we will insure that the MSBuild global variable SourceDirectory point to the root of the solution inside the source control. Edit the build definition and set these values in the workspace tab:
Source control Folder: $/YourTFSProject/FitNesseTfsTest
Build Agent Folder: $(SourceDir)
Now save and check-in the template file you’ve just created.

Results

Now in the TFS build explorer, we can see the trace of the script invocation in the log view.

If the FitNesse tests failed, we can see this error:

To see a more detailed test result we can inspect the result.txt file inside the dotnetXX folder on the build server since we have specified it on the script when calling runner.exe.
We could see the result directly in the build log view by parsing and writing the content of result.txt in Powershell output.

Authors

Maxence Bonhomme & Christian Baumann

Christian Baumann

I´m an agile practitioner with a strong focus on software testing. I started my professional career as a consultant in the area of test automation (GUI- as well as performance-testing), working in many different business domains; then became test automation engineer/ project manager verification/ test team lead in the healthcare sector. After getting in touch with "Agile" in the beginning of 2008 for the first time, I started reading (and learning) more & more about, being "infected" by this kind of working. In February 2011 I started for Agile Partner and and I´m courius what the next chapters of this story will be!

No Comments

Post a Comment

Comment
Name
Email
Website