This Question is Answered

1 "correct" answer available (4 pts) 2 "helpful" answers available (2 pts)
12 Replies Last post: Mar 19, 2010 4:24 PM by Pavel Sher  
Markus Bischof Newbie 12 posts since
Mar 14, 2010
Currently Being Moderated

Mar 16, 2010 6:13 PM

EMMA - Unsteady coverage results

Hi!

 

We are using EMMA as code coverage tool in our build.

 

We noticed that the coverage results are really unsteady. Sometimes, classes are covered, but in the next build, they aren't covered (without changing anything in the code).

 

Is there anything we can do in order to get stable results from EMMA?

 

Regards,

Markus Bischof

Pavel Sher JetBrains 2,148 posts since
Mar 6, 2008
Currently Being Moderated
Mar 16, 2010 6:28 PM in response to: Markus Bischof
Re: EMMA - Unsteady coverage results

Coverage can be unstable because some code is not always working the same way. For example, there can be code which tries to allocate some resource and if allocation fails, tries again. In some build you may have successful resource allocation from the first attempt, while in other build - it is not. In these cases coverage can differ.

 

Also coverage can differ if there is some platform specific code.

Pavel Sher JetBrains 2,148 posts since
Mar 6, 2008
Currently Being Moderated
Mar 17, 2010 3:15 PM in response to: Markus Bischof
Re: EMMA - Unsteady coverage results

Probably this is a bug in EMMA. In fact we are moving to use IDEA coverage engine, because EMMA is not supported anymore by its authors. IDEA coverage engine is available starting with TeamCity 5.0. So if possible, I would recommend to upgrade to TC 5.0.

Pavel Sher JetBrains 2,148 posts since
Mar 6, 2008
Currently Being Moderated
Mar 17, 2010 7:19 PM in response to: Markus Bischof
Re: EMMA - Unsteady coverage results

No, you don't need to install anything. Could you please check is there a coverage-error.log file in the working directory on the agent? Also please provide your build log. If it contains sensitive information you can send it to teamcity-feedback@jetbrains.com

Pavel Sher JetBrains 2,148 posts since
Mar 6, 2008
Currently Being Moderated
Mar 18, 2010 10:00 AM in response to: Markus Bischof
Re: EMMA - Unsteady coverage results

What include / exclude patterns do you use?

Pavel Sher JetBrains 2,148 posts since
Mar 6, 2008
Currently Being Moderated
Mar 18, 2010 12:24 PM in response to: Markus Bischof
Re: EMMA - Unsteady coverage results

Do you run your tests in separate JVM (is fork enabled)? If not please try to enable fork.

Pavel Sher JetBrains 2,148 posts since
Mar 6, 2008
Currently Being Moderated
Mar 19, 2010 4:24 PM in response to: Markus Bischof
Re: EMMA - Unsteady coverage results

Congratulations! I think we need to add some note on the web UI about fork requirement.

More Like This

  • Retrieving data ...