Merging Reports
Prev | Chapter 3. Running Lire | Next

## Merging Reports

Lire supports the merging of reports: one can combine two reports into one bigger report. This can be used to generate e.g. a weekly report from 7 daily reports, or generate a site-wide report from reports about the behaviour of each server on a site.

We describe how to manually merge reports using the command line tools lr_xml2report, but the simplest way to use merging is through the DLF store interface which is described at Chapter 4, Using DLF Stores.

We give an example.

Example 3.4. Merging Reports

To process two BIND v9 logfiles, and merge the reports, one would run:

    $ lr_log2report --ouput xml bind9_query /var/log/named.2.gz \
        $XMLDIR/20020622.xml
    $ lr_log2report --output xml bind9_query /var/log/named.1.gz $XMLDIR/20020623.xml
    $  lr_xml2report --tempate dns_default \
        --merge $XMLDIR/20020623.xml  $XMLDIR/20020622.xml $ASCIIDIR/20020622-20020623.txt

The `--template` parameter is required for merging and specifies the report configuration template that should be used to merge the reports. You should probably use the same than the one that was used when you generate the reports. If you didn't specify one, (like in the above example) you should know that the default template is named ``superservice`_default`. The list of available report configuration templates can be displayed by using the `lr_xml2report --help report-templates`.

The `--merge` option is used to specify the other XML reports that should be merged before formatting the report. The lr_xml2mail command uses the same options for merging.

### Gotchas

The merging functionality is very powerful, and allows you to shoot yourself in the foot. We document some pitfalls.

#### Holes in your reporting period

When merging XML report files xml.3 (2002-06-02 08:50:48 CEST - 2002-06-09 08:05:06 CEST) and xml.1 (2002-06-16 08:18:40 CEST - 2002-06-21 22:13:09 CEST) , the generated report will gladly display "Reporting on period: 2002-06-02 08:50:48 CEST - 2002-06-21 22:13:09 CEST": There is no saveguard against forgetting in-between report files.

#### Take care when changing report configuration template parameters

In some cases, changing the report configuration template just before merging might lead to bogus data in your report.

Consider this case: our firewall template contain a subreport top-pkt-by-src with the `ips_to_show` set to 10, We process some firewall logs, and archive the XML reports. If we change the `ips_to_show` to 100 and merge the XML reports. This could incorrectly omit some IPs! You've got no guarantee the exact top 100 IPs are shown. This is due to the fact the XML reports do not contain all information from the log: they're reports, after all.

Due to these issues, the merging is implemented with some heuristics: we keep more data than what's requested by the user in the XML report, to be able to handle most after-the-fact merging requests. We've tested the algorithm with a pretty broad range of real-life log files, and found out generally, the merged reports do give a good reflection of what actually has happened on the network: the heuristic is pretty well choosen. However, if you really need guaranteed 100% accurate data, generate your report directly from the raw logs. If you just want a quick overview, the merging is more suitable. Just make sure you're not cranking the limit parameters up too high in this case.

See also the Report Generating chapter in the Lire Architecture part of the Lire Developer's Manual.

Prev | Up | Next
Generating A Report From A Log File | Home | Sending Anonymized Log Files To A Responder