Advanced log analysis techniques

When protecting large networks, quite often a large amount of logging is produced. The task of correlating different sources of logs, and isolating the relevant entries becomes daunting, and the need for analysis tools is more pronounced. Although many log analysis products are available on the market which simplify the analysis task, it is the analysis technique which is most important. In this talk we present advanced techniques for log analysis using simple open source tools. We give real world examples as used in incident analysis for web logs and firewall logs. Finally this technique is extended to general log files, and in particular to network packet dumps. We show how packet dumps may be analysed to retrieve information about network anomalies.