- Generated by
1.9.1
|
HEBench
|
HEBench report compiler is a tool provided as part of the frontend. It is designed to lighten the work of Test Harness by encapsulating the functionality of performing statistical computations and producing human readable summaries for benchmark reports generated by Test Harness.
Report compiler is composed of two parts: a shared library that exposes the report compiler functionality and can be used directly by other applications; and an executable application, wrapping around the shared library interface, that allows users direct access to the functionality and configuration of the report compiler via command line arguments.
The following shows how to use the report compiler command line interface and its features.
For more information on using the shared library interface, see the reference hebench::ReportGen::Compiler
The report compiler command line interface executable is one of the applications installed with HEBench frontend.
The syntax for using the report compiler command line interface is shown below. It consists of calling the executable and a space separated list of command line arguments to configure the run. The available options are listed in the options table on this page as well as available through report_compiler help option.
By default, Test Harness calls upon the report compiler using default compiler options to generate summaries and statistics from the benchmark reports generated during a run. If users do not want or need the default compiled information, this functionality can be disabled on Test Harness via command line options. See Test Harness User Guide for more information. Users can run the report compiler manually on the output of Test Harness to produce the summaries and statistics.
To display the report compiler help which lists all the available command line options, use the following command:
Positional argument | Position | Description |
|---|---|---|
input_file | 0 | Input file to be used during report generation. This can be a single benchmark report generated by Test Harness or a file containing a list of report files to process. Each filename in this file can be absolute, or relative to this file location. The directory for this file, and sub-directories, must be available for writing. |
Option | Required | Description |
|---|---|---|
--time_unit <time_unit_name> -tu | N | Specifies the time unit to be used for the generated reports if no time unit is specified for a report type. The value of this parameter must be one of: "s" - seconds "ms" - milliseconds "us" - microseconds "ns" - nanoseconds If missing, an appropriate unit will be automatically selected per benchmark such that the the displayed time is in the range [1..1000] whenever possible. |
--time_unit_overview <time_unit_name> -tuo | N | Specifies the time unit to be used for the generated overview. If no time unit is specified, the fallback time unit specified for parameter --time_unit is used instead. |
--time_unit_stats <time_unit_name> -tut | N | Specifies the time unit to be used for the generated statistics. If no time unit is specified, the fallback time unit specified for parameter --time_unit is used instead. |
--time_unit_summary <time_unit_name> -tus | N | Specifies the time unit to be used for the generated summaries. If no time unit is specified, the fallback time unit specified for parameter --time_unit is used instead. |
Option | Required | Description |
|---|---|---|
--show_overview <bool: 0;false;1;true> --overview | N | Specifies whether or not to display report overview to standard output. Defaults to "TRUE". |
--silent_run --silent | N | When present, this flag indicates that the run must be silent. Only specifically requested outputs will be displayed to standard output. When running silent, important messages, warnings and errors will be sent to standard error stream stderr. |
-h, /h, \h, --help, /help, \help | N | Shows this help. Application exits after this. |
On a default run, Test Harness produces report files for each benchmark executed from a backend as well as a file containing a list of all the benchmark report filenames.
Input file for report compiler can be either one of these report files generated Test Harness or the benchmark list.
For each benchmark executed in a run, Test Harness outputs a CSV report file. On a successful completion, this report file contains detailed information for each event recorded during the benchmarking. The contents of this file are not binary, but not very human-readable friendly either.
Using a report file as input will have report compiler generate a friendly CSV summary and a file containing results of statistical computations performed on the report collection of event.
The benchmark list file is a text file where each line is the filename for a CSV benchmark report generated by Test Harness. Each filename can be an absolute path, or relative to this list file.
Using this file as input will have report compiler read and produce summary and statistics for each report as input file.
For each CSV report successfully parsed, report compiler will generate two files:
summary.csvstats.csvBoth files will be placed in the same directory containing the initial report. The name for the report filename will be pre-pended to the output filenames in an attempt to avoid overwriting other outputs. For example, if the report filename is he_test_report.csv, the output files will be named he_test_summary.csv and he_test_stats.csv respectively. If these files already exist, they will be overwritten without warning.
Furthermore, report compiler will generate an overview file, <input_filename_noext>_overview.csv, in the same directory as the input_file. This overview contains a table listing the statistics for the Main Event measured for each report per row. This is intended as an easy, at-a-glance overview of the compiled run and contains complete information of each report's main event.
The statistics file generated for a report contains the complete view of the output.
The summary file contains a friendly excerpt of the statistics file that can be used for a quick preview of the important data extracted from the report.
If the input report represents a failed benchmark, neither statistics nor summary are generated. However, an entry in the overview file will indicate a failed benchmark for the corresponding report.
See Statistics and Summary Output Format for the full details.
Each table row of the overview file is the statistics row for the Main Event recorded from the corresponding statistics file, along with some extra information to help identify the benchmark configuration and locate the report used to generate the statistics.
See Run Overview Output Format for the full details.
1.9.1