CMS MessageLogger: Obtaining Message Statistics

CMS MessageLogger Service
Obtaining Message Statistics

The MessageLogger service can provide statistics about the message issued during the course of a job. The information is placed in tables either at the end of a log file or in an individual file. The user code can also trigger statistics summaries of messages issued thus far --
see The edm::LogStatistics() Function

These statistics tables, written when the MessageLogger service encounters its postEndJob callback, include:

process TEST = 
  service = MessageLogger {
    vstring destinations = { "error_messages.txt" }
    vstring statistics   = { "statistics.txt"
			   , "another_stats" 
			   , "error_messages.txt" 
			   }
    PSet statistics.txt       = { string threshold  = "WARNING"   }
    PSet anotherStats         = { string output = "more_stats.txt" }
    PSet error_messages.txt   = { string threshold  = "ERROR" }
  }
  untracked PSet maxEvents = {untracked int32 input = 5}
  path p = { myAnalysisModule }
  module myAnalysisModule = ModuleThatIssuesMessages { }
  source = EmptySource { }
}
The vstring statistics = { "statistics.txt" , "another_stats" , "error_messages.txt" } directs the system to attach a three statistics destinations, to be controlled by parameter sets named statistics.txt, another_stats, and error_messages.txt. (Normally, only one statistics destination is desired; here we use three to illustrate various control options.)

PSet statistics.txt = { string threshold = "WARNING" } directs statistics.txt to respond to (and keep track of) only LogError and LogWarning messages. Statistics destinations do not impose limits for each category (they only count the messages anyway) but they are sensitive to thresholds of which severities to report about.

PSet anotherStats = { string output = "more_stats.txt" } directs the statistics destination controlled by the anotherStats PSet to write its output to the file more_stats.txt. If no output were specified, then the statistics destination would write its summary to a file with the name listed in the statistics vstring. For example, with the above .cfg file, the statistics.txt destination would write to statistics.txt.

PSet error_messages.txt = { string threshold = "ERROR" } controls both the error_messages.txt output destination, and the error_messages.txt statistics destination. Thus that statistics destination will respond with a threshold of LogError, and will write its end-of-job summary to the end of the file error_messages.txt.

In general, any statistics destination whose output file name (as determined either by an explicit output string or implicitly as the same name used to control the destination) matches the file of an ordinary destination, will write its summary output at the end of that ordinary destination's file.


Here is a sample statistics summary output:

Process
 type     category        sev    module        subroutine        count    total
 ---- -------------------- -- ---------------- ----------------  -----    -----
    1 cat_A                -w UnitTestClient_A                       3*       3
    2 cat_B                -w UnitTestClient_A                       3*       3
    3 cat_A                -e UnitTestClient_A                       3        3
    4 cat_B                -e UnitTestClient_A                       3        3
 
* Some occurrences of this message were suppressed in all logs, due to limits.
 
 type    category    Examples: run/evt        run/evt          run/evt
 ---- -------------------- ---------------- ---------------- ----------------
    1 cat_A                1/1              1/2              1/3
    2 cat_B                1/1              1/2              1/3
    3 cat_A                1/1              1/2              1/3
    4 cat_B                1/1              1/2              1/3
 
Severity    # Occurrences   Total Occurrences
--------    -------------   -----------------
Warning                 6                   6
Error                   6                   6


The edm::LogStatistics() Function

The user program can force each statistics destination to write its current message counts and summary to its file or stream, before the end of job automatically causes that writing. To do this, invoke the function
edm::LogStatistics();
(Just as for edm::LogError() and the other message commands, this function is defined by including MessageLogger.h).

While ordinary analysis code probably has no need to invoke this, one can use the framework's features to set up a sensible schedule of outputting statistics. For example, one can place a call to edm::LogStatistics() in a postRun callback.

The default behavior is to output the statistics thus far accumulated, and then to continue collecting statistics. If, for some statistics destination, it is preferable that the statistics be reset to zero after each summary output, this can be directed in the PSet for that statistics destination:

process TEST = 
  service = MessageLogger {
    vstring destinations = { "error_messages.txt" }
    vstring statistics   = { "statistics.txt", "runstats.txt" }
    PSet statistics.txt  = { string threshold  = "WARNING"   }
    PSet runstats.txt    = { bool reset = true }
  }
  untracked PSet maxEvents = {untracked int32 input = 5}
  path p = { myAnalysisModule }
  module myAnalysisModule = ModuleThatIssuesMessages { }
  source = EmptySource { }
}
In this example, the statistics kept by runstats.txt will be reset after each statistics output triggered by edm::LogStatistics().

Of course, the bool reset parameter is moot if the program never explicitly invokes edm::LogStatistics(), since it is immaterial whether the statistics are reset after the final automatic end-of-job summary is output.

USCMS Software and Computing Home Page - CMS MessageLogger Service Page - Configuration File


Mark Fischler
Last modified: November 29, 2005