GCOV

GCOV can be used to gather code coverage information.  GCOV is a component of
G++, and therefore can only be used on machines which use that compiler (this
should not be an issue, as code coverage is machine-independent).  In order to
use GCOV, the configure script needs a few extra options.  Under normal
conditions, this should just be:

$./configure CXXFLAGS="-fprofile-arcs -ftest-coverage" [other options]

If the configure script does not automatically detect G++, you may have to
specify it as an option:

$./configure CXX=g++ CXXFLAGS="-fprofile-arcs -ftest-coverage" [other options]

Also, note that it is a good idea to remove optimization options via "-O0"
(i.e., dash-oh-zero).  Optimization can change the accuracy of the coverage
information.

Then do:

$make

and optionally:

$make install

Now, run whatever functionality tests you wish to run.  In theory, this should
be every test you have.  Time lapses and test order do not matter.  If you are
not testing the entirety of Trilinos, it may also be a good idea to run tests
for packages that depend on the packages you are testing, as these external
tests may test your packages in ways that your tests do not.  Note that since
GCOV will eventually be part of the daily or weekly tests, it would be a good
idea for these functionality tests to reflect what is on the CVS repository
(whether that means updating your files or the repository's).

Unfortunately, GCOV by itself cannot work with multiple files at once.
However, there is a Perl script (Trilinos/test/coverage/gcovmerge) which
automates the process of running GCOV over a directory and summarizing the
resulting information.  In order for the latest version of gcovmerge to work,
your machine must have GCC 3.4.3 or later.

At minimum, gcovmerge takes one argument: the path to search for .gcda files
(which indicate files that have been compiled and tested as explained above).
The following arguments affect the coverage testing (these should come before
the path):
   --with-files=min-max [0-100]
       When specified, no coverage information is displayed for
       any individual files with a coverage percent outside of 
       the range given.

   --with-directories
       When specified, coverage percentages are displayed for
       each directory examined.

   --without-files
       No coverage information will be given for individual files.

   --exclude=word []
       No files with word in their name or path will be examined.
       This option may be specified more than once, in order to
       exclude a variety of files.

   --with-total   
       The total coverage for all files examined will be displayed.

   --source-dir=path
       The path to look for source files if the source and build
       directories are seperate.  This directory should have an identical
       hierarchy to that of build_dir, or else this script will not be
       able to reconcile the data in source_dir with that in build_dir.
       If not specified, the script will assume that the source files
       are in build-dir.

   --with-gcov=GCOV
       Execute GCOV instead of "gcov". This is useful for using a
       different version of gcov than is located in the user path,
       or one with a different name (e.g., gcov-4.0)

   --old-school
       Revert to previous (ver 1.5) behavior of gcovmerge. Coverage
       information will be generated from a single executable per 
       source file. This methodology results in lower and less accurate
       coverage numbers.

   --with-count
       Specifies that coverage should be reported as "n of t", instead of
       the percentage.

   --with-percent
       Specifies that coverage should be reported as a percentage.
       If neither --with-count nor --with-percent are specified,
       then --with-percent is the default.

   --with-report
   --with-report=REP
       Specifies that a report should be generated.
       This report will produce a listing of the
       all source files whose coverage was analyzed,
       wherein the lines of code are displayed as either
       covered, not-covered, or not compiled.
       If --with-report is given without the REP
       option, then the filename for the report
       will be coverage.html.
   --with-full-paths
       Outputs full pathnames instead of abbrieviated file names.

Running gcovmerge with the default options from the Trilinos level will most
likely generate too much information to really use.  Specifying the search path
and the options will tone this down to something more useful.  For example, to
check the coverage percentages on all source in the EPetra package (but not
tests, documentation, or examples), one would do this:

$./commonTools/test/coverage/gcovmerge --exclude=test/ --exclude=example/            \
  --exclude=doc/  --source-dir=./packages/epetra --with-gcov=/opt/gcc-3.4.4/bin/gcov \
  ./BUILD_DIR/packages/epetra

To get a summary of the coverage on each package, one could do this:

$./commonTools/test/coverage/gcovmerge --without-files              \
  --with-directories --exclude=test/ --exclude=example/             \
  --exclude=examples/ --exclude=doc/ --source-dir=./packages        \
  --with-gcov=/opt/gcc-3.4.4/bin/gcov                               \
  ./BUILD_DIR/packages

In theory, every package should have 100% test coverage.  This, however, is not
a practical goal.  A more realistic goal is for every source file to have 85%
test coverage.

gcov and gcovmerge are not perfect.  It is impossible to run gcov on particular
source files -- this could happen if a source file is a templated header that
was left completely untested, or if a source file in the requested source-dir
was never compiled.  In such cases, these source files will not be reported.

If you want to see specific coverage information for any particular file, you
should use GCOV directly:

$gcov dotofile.o
100.00% of 1 lines executed in file /usr/lib/.../ansilib
Creating ansilib.gcov.
80.00% of 10 lines executed in file ../.../dotofile.cpp
Creating dotofile.cpp.gcov

(Note that this command must be run from the same directory as g++ was invoked.
When following the configure/make/make install pattern, this would normally be
the directory that contains dotofile.o.)

Obviously, the information regarding library files is quite useless, so it will
be ignored here.

The .gcov files are recreations of the original source files, with extra
annotation to indicate how often certain lines have been executed:

$cat dotofile.cpp.gcov
     -:		0:Source:dotofile.cpp
     -:		0:Object:dotofile.bb
     -:		1:#include <iostream>
     -:		2:using namespace std;
     -:		3:
     1:		4:main(){
     1:		6:  int num;
     1:		7:  cout << "Please enter a number:\n";
     1:		8:  cin >> num;
     -:		9:  //comment
     4:	       10:  for(int i = 2;
     -:        11:	i < num;
     -:	       12:	i++){
     4:	       13:    if(num % i == 0){
     1:	       14:      cout << num << " is divisible by << i << ".\n";
     1:	       15:      exit(0);
     -:	       16:    }
     -:	       17:  }
 #####:	       18:  cout << num << " is prime.\n";
     -:        19:}

The first column indicates how many times each line was executed (- means that
the line is not executable and ##### means the line was never executed).  The
second is the line number in the code (lines 0 are for gcov's use). The
remainder of the file is the actual source.
