Overview

Manual log review can miss critical signals when data volume grows. This script automates pattern matching, tracks event counts by category, and captures sample lines for quick triage context.

Real-world relevance: Automated log parsing supports SOC workflows by reducing repetitive manual review and improving visibility into recurring error and security events.

Objective

• Read and process a target log file line by line
• Search for configurable keyword patterns such as ERROR, FAILED, WARNING, and authentication failure events
• Count occurrences of each pattern and identify the lines where they appear
• Output a timestamped summary report to the terminal and a text file
• Keep the script flexible so patterns and the target log file can be easily updated

Script Walkthrough

1. Defined a named dictionary of event patterns using regex and keyword matches.
2. Opened the target log file and processed each line with buffered reading.
3. Checked each line against all configured patterns and recorded matches with line numbers.
4. Calculated per pattern totals and assembled structured report output.
5. Wrote a timestamped report file and printed a concise terminal summary.

Script Preview

#!/usr/bin/env python3
"""
Automation Lab 3 - Python Log Parser & Alert Script
Scans a log file for defined event patterns and generates a summary report.
"""

import re
from datetime import datetime
from pathlib import Path

LOG_FILE   = "/var/log/syslog"   # Change to your target log path
OUTPUT_DIR = Path("./reports")
OUTPUT_DIR.mkdir(exist_ok=True)

PATTERNS = {
    "ERROR":                r"\bERROR\b",
    "WARNING":              r"\bWARNING\b",
    "CRITICAL":             r"\bCRITICAL\b",
    "Authentication Failure": r"authentication failure|Failed password",
    "Segfault":             r"segfault|kernel BUG",
    "Disk Full":            r"No space left on device",
}

timestamp   = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
output_file = OUTPUT_DIR / f"log_report_{timestamp}.txt"

matches = {name: [] for name in PATTERNS}

print(f"Parsing: {LOG_FILE}")
with open(LOG_FILE, "r", errors="replace") as fh:
    for line_num, line in enumerate(fh, start=1):
        for name, pattern in PATTERNS.items():
            if re.search(pattern, line, re.IGNORECASE):
                matches[name].append((line_num, line.rstrip()))

with open(output_file, "w") as report:
    report.write(f"Log Parser Report - {timestamp}\n")
    report.write(f"Source: {LOG_FILE}\n")
    report.write("=" * 60 + "\n\n")
    for name, hits in matches.items():
        report.write(f"[{name}] - {len(hits)} occurrence(s)\n")
        for line_num, content in hits[:10]:   # cap preview at 10 lines
            report.write(f"  Line {line_num}: {content}\n")
        if len(hits) > 10:
            report.write(f"  ... and {len(hits) - 10} more.\n")
        report.write("\n")

print("\nSummary:")
for name, hits in matches.items():
    print(f"  {name}: {len(hits)} match(es)")
print(f"\nFull report saved to: {output_file}")