Automation Lab 3: Python Log Parser and Report

Overview

When a log gets big, it is easy to miss the lines that matter. This script counts each type of event and keeps a few example lines so I know where to start looking.

Real world relevance: Scanning logs automatically saves time in a SOC and makes repeat errors and security events easier to notice.

Objective

• Read and process a target log file line by line
• Search for configurable keyword patterns such as ERROR, FAILED, WARNING, and authentication failure events
• Count occurrences of each pattern and identify the lines where they appear
• Output a timestamped summary report to the terminal and a text file
• Keep the script flexible so patterns and the target log file can be easily updated

Script Walkthrough

1. I made a list of patterns to look for, using keywords and regular expressions.
2. I opened the log file and read it one line at a time.
3. I checked each line against every pattern and saved the line number of each match.
4. I counted the matches for each pattern.
5. I wrote a report with the date and time and printed a short summary in the terminal.

Script Preview

#!/usr/bin/env python3
"""
Automation Lab 3 - Python Log Parser & Alert Script
Scans a log file for defined event patterns and generates a summary report.
"""

import re
from datetime import datetime
from pathlib import Path

LOG_FILE   = "/var/log/syslog"   # Change to your target log path
OUTPUT_DIR = Path("./reports")
OUTPUT_DIR.mkdir(exist_ok=True)

PATTERNS = {
    "ERROR":                r"\bERROR\b",
    "WARNING":              r"\bWARNING\b",
    "CRITICAL":             r"\bCRITICAL\b",
    "Authentication Failure": r"authentication failure|Failed password",
    "Segfault":             r"segfault|kernel BUG",
    "Disk Full":            r"No space left on device",
}

timestamp   = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
output_file = OUTPUT_DIR / f"log_report_{timestamp}.txt"

matches = {name: [] for name in PATTERNS}

print(f"Parsing: {LOG_FILE}")
with open(LOG_FILE, "r", errors="replace") as fh:
    for line_num, line in enumerate(fh, start=1):
        for name, pattern in PATTERNS.items():
            if re.search(pattern, line, re.IGNORECASE):
                matches[name].append((line_num, line.rstrip()))

with open(output_file, "w") as report:
    report.write(f"Log Parser Report - {timestamp}\n")
    report.write(f"Source: {LOG_FILE}\n")
    report.write("=" * 60 + "\n\n")
    for name, hits in matches.items():
        report.write(f"[{name}] - {len(hits)} occurrence(s)\n")
        for line_num, content in hits[:10]:   # cap preview at 10 lines
            report.write(f"  Line {line_num}: {content}\n")
        if len(hits) > 10:
            report.write(f"  ... and {len(hits) - 10} more.\n")
        report.write("\n")

print("\nSummary:")
for name, hits in matches.items():
    print(f"  {name}: {len(hits)} match(es)")
print(f"\nFull report saved to: {output_file}")

Key Skills Demonstrated

Tools & Technologies

• Python 3.8+
• Standard library modules including re, datetime, and pathlib
• Linux and application log sources
• Plain text output for ticket and analyst handoff use

Results & Outcomes

• Automated repetitive log review steps that are often done manually.
• Generated categorized reporting that can support incident notes and ticket updates.
• Improved flexibility by allowing new event types to be added quickly.
• Prepared the workflow for scheduled execution or on demand triage support.