I’ve been experimenting lately with TeamCity 5 EAP and so far it’s been a pretty awesome experience. I was up and running within minutes and I was swarmed with beautiful graphs and statistics with specifics even per-test. Getting something like that up with CC.NET is not a trivial task.
Anywho, with TC5 code coverage is one of the cool new features added for .NET, but unfortunately only NUnit is supported. Not that that’s a bad thing, but some people prefer to use other testing tools. Two notable contenders are xUnit.net and MbUnit.
I like the fact (pun intended) that xUnit.net makes it a point to prevent you from doing bad practices (like
[ExpectedException]), and I like how MbUnit is so bleeding edge with useful features (like
[Parallelizable] and a vast availability of assertions).
And with that I set up to figure out how to get TC working with Gallio, but the following should work with any test runner.
It certainly was a pain to set up because it took a lot of trial and error but eventually I figured it out. I analyzed the build logs provided in each build report and noticed something interesting…specifically:
[13:51:23]: ##teamcity[importData type=’dotNetCoverage’ tool=’ncover’ file=’C:\TeamCity5\buildAgent\temp\buildTmp\tmp1C93.tmp’]
[13:51:30]: ##teamcity[buildStatisticValue key=’CodeCoverageL’ value=’94.85067’] [13:51:30]: ##teamcity[buildStatisticValue key=’CodeCoverageM’ value=’97.32143’] [13:51:30]: ##teamcity[buildStatisticValue key=’CodeCoverageC’ value=’98.68421’] [13:51:30]: ##teamcity[buildStatisticValue key=’CodeCoverageAbsLCovered’ value=’921.0’] [13:51:30]: ##teamcity[buildStatisticValue key=’CodeCoverageAbsMCovered’ value=’218.0’] [13:51:30]: ##teamcity[buildStatisticValue key=’CodeCoverageAbsCCovered’ value=’75.0’] [13:51:30]: ##teamcity[buildStatisticValue key=’CodeCoverageAbsLTotal’ value=’971.0’] [13:51:30]: ##teamcity[buildStatisticValue key=’CodeCoverageAbsMTotal’ value=’224.0’] [13:51:30]: ##teamcity[buildStatisticValue key=’CodeCoverageAbsCTotal’ value=’76.0’]
The first message happens after NCover.Console is done its thing. After NCoverExplorer is done its thing, the statistics are published. I set out to mimic this functionality with Gallio, but what’s described here should work with any test runner.
- Disable code coverage in TC. We’re doing it manually instead.
- In your build script, run your unit tests with NCover and generate a coverage.xml report.
- Run NCoverExplorer on coverage.xml and generate reports ncoverexplorer.xml and index.html.
- Create a zip file of index.html and name it coverage.zip.
- Configure coverage.zip to be an artifact in your TC configuration (this is to enable the tab).
- Parse out ncoverexplorer.xml with XPath and output the statistics.
Certainly a lot of things to do just for the sake of pretty statistics reporting….but it was the weekend and I was bored. With the help of MSBuildCommunityTasks, the zip file and XML parsing was made a lot easier.
After that, viola! Code coverage + Gallio on TeamCity 5!!!
Unfortunately, NCoverExplorer’s report only reports the # total of classes and nothing about unvisited or covered, so for those values I set to 0/0/0 (BTW, you need all values present for the statistics to show). A task for next weekend!!!
(Edit: I suppose I could should also mention that you could technically avoid all the trouble above and hack it with this:
#if NUNIT using TestFixtureAttribute = NUnit.Framework.TestFixtureAttribute; using TestAttribute = NUnit.Framework.TestAttribute; #endif
And it’ll work just fine as well).