Share via

Merging coverage reports with Bullseye

  • Article

I've previously recommended Bullseye. And there is another nifty feature with Bullseye you should know; the ability to merge reports. This is pretty useful when you have one report from your unit tests and one from some other type of test run. Use this command to merge reports with Bullseye:

 covmerge.exe -c -fMergedData.cov file1.cov file2.cov

Comments

  • Anonymous
    July 22, 2009
    I have merged 2 .cov files, how to identify that the generated .cov file is correctly merged. Thanks

  • Anonymous
    July 22, 2009
    Why would you like to verify this? Why not trust the tool? Why would it not be correct? But the way to verify is to look at the data and the merged data will be max coverage in the two original files. Let's assume that your first coverage file tells you that function A has 25% coverage and function B has 50% coverage. The second coverage file then has function B at 75% and function C at 100% coverage. Then the merged file should show: A-25%, B-75%, C-100%.

  • Anonymous
    November 30, 2009
    re: Merging coverage reports with Bullseye >Then the merged file should show: A-25%, B-75%, C-100%. This doesnt always happen. If function B has modified, bullseye retains the cov for the one which is more recent. So if the second coverage file has an modified function B , and has coverage B- 25% and Func C at 100%, then the merged file would show A-25%, B-25% , C-100% Is there anyway to get around this?

  • Anonymous
    November 30, 2009
    @ram: I didn't know that because I run my coverage with the same build. Does not make sense to me to run coverage runs with different versions of the application. The only thing that makes sense is that if a function is changed you get the latest coverage rather than the merged because whatever value you had before is no longer valid since you changed the function. Hence I don't think there is anything that needs to be fixed...

  • Anonymous
    December 02, 2009
    >>Hence I don't think there is anything that needs to be fixed... True. But consider a case where only function B is modified, (with A and C remain unchanged) and the file has a newer version. In that case merged coverage report would only be the one which corresponds to the recent version of the source. In which case the merged coverage would have A-0%, B-25% , C-100% and earlier obtained coverage for A is discarded. There are pros and cons for this approach, but one would expect Bullseye to leave it to the user to a pass on an parameter to covmerge to do either.

  • Anonymous
    December 02, 2009
    @ram: I disagree that it is up to the user to decide what to do if two merge files being merged have been using different versions of the code because if there are different versions basically all coverage from the old merg file is no longer up to date. I think it is much more productive to focus on getting your different runs using the same version and doing it automatically. It just makes more sense to me.

  • Anonymous
    March 02, 2010
    Guys, how about merging 2 coverage files build with different path. For ex: covfile1.cov has project directory c:x and covfile2.cov has project directory c:xyz. In this case whether Bullseye merger will merge the files(if the same function is not modifed; instead only new files are added in the recent covfile2.cov)?

  • Anonymous
    March 02, 2010
    @Rajan: Never tried that. Don't see why it should not work since the two sets of code are different. But also, since the two sets are completely different there is no benefit from joining the files other than the ability to look in one file instead of two...

  • Anonymous
    March 03, 2010
    Let's take one example. If more than one developer started working on one feature then as per the BullsEye's behavior, they both have to MAINTAIN the same directory structure of their project(in their dev machines), in order to merge their .cov files from Unit Testing. Am I right? My question here is, why BullsEye do this much check and why don't it identify the project/file/func names and do the merge of .cov files?

  • Anonymous
    March 03, 2010
    Two developers working on the same project/feature should have the same folde structure because they both chech their code into some version controlsysyem, right? The root may be different, but from the rot down it should be the same. Don't even understand why you would run coverage seperatly since there is a merged version of the code; in the version control system...

  • Anonymous
    March 04, 2010
    Thing is, our directory structure is same but root is different. When root is different, BullsEye merge is not merging the .cov files at all. Fine. What i want to convey is, let's take, Dev1 writes Func1 in File.cpp and Dev2 writes Func2 in File.cpp, both took code from same SCM. Here, Dev1 could run unit test cases to check the coverage of his func and same with Dev2. Once they are done, their manager wants to know the code coverage % of the new module. Obviously they got to merge to show the result? this is what my use case is. Whether they take code from SCM or not, as per the BullsEye, they have to maintain the same directory structure(from root) to build the code for code coverage and to create .cov file. when the result is asked they got to merge the .cov files.

  • Anonymous
    March 04, 2010
    @Rajan: They don't obviously have to merge their seperate results. I would say you get the latest from SCM and run the tests and get a single coverage file. I think that is the only thing that makes sense. I know having different roots is not a problem. Other than that I donät understand if you've experienced a problem and something does not work for you, in which case you need to talk to Bullseye support. Or you're asking if this will work in which case I think there is something missing since the type of testing you describe; testing one module using two different versions does not make any sense to me.

  • Anonymous
    June 21, 2010
    how to get separate coverage for each solution files separately. Example : A.sln -> coveraheA.cov                  B.sln -> coveraheB.cov

  • Anonymous
    June 22, 2010
    @Mithun: Before running any tests, copy the coverage file generated when compiling to the new desired names. Then set the COVFILE environment variable to different values for each solution. Or just build and run each solution seperatly deleteing (or probably moving it away for later analysis) the coverage file in between

  • Anonymous
    June 22, 2010
    The comment has been removed

  • Anonymous
    June 22, 2010
    @Mithun; I would guess that you do not run from a VS prompt but I would suggest you contact Bullseye support.

  • Anonymous
    November 12, 2010
    We have number of .so files. some of them are compiled with bullseyes and some are not.  how can i know which .so is linked with bullseyes.

  • Anonymous
    November 13, 2010
    @Asheesh Goyal: Getting off topic here but assuming you're on some unix flavor ldd is the command to use: man ldd for how to use it.

  • Anonymous
    December 19, 2011
    I am facing while building driver after enabling bullseye. Build is completing but *.cov file is not generating. Any suggestion will he very helpful?

  • Anonymous
    December 20, 2011
    The comment has been removed

  • Anonymous
    May 01, 2013
    hi i wanted to find a way out to get different .cov files respective of the files that are run. If I run a batch file, then is there any method to get .cov files for every file that is run in that batch file? Please reply as soon as possible, Im stuck on this!

  • Anonymous
    May 02, 2013
    @DesireMore If I remember correctly the cov file name is defined when you instrument your binary (i.e. essentially at compile time). As such you can easily give different names for each of your binaries if you want to. That is the right thing to do. A dirty work around is to rename the cove files after each run in your batch file.