Build a Python tool that turns log files into UML sequence diagrams

Logs record the order of events, but raw timestamps and status messages can be difficult to interpret when a request passes through several services. A Python utility that converts those events into UML sequence diagrams can reveal the interaction between clients, web servers, queues, databases and external APIs. This is useful for debugging distributed systems, documenting legacy applications and explaining production incidents.

The tool can remain small and portable: read a log file, extract participants and messages, then produce PlantUML or Mermaid syntax. A practical design should cope with inconsistent log formats, repeated requests, missing fields and sensitive information. For Australian development teams working across Sydney, Melbourne, Brisbane and Perth, clear timezone handling is particularly important when comparing events from systems hosted in different regions.

Define the log event model

Begin with an internal representation that is independent of any specific log format. Each event might contain a timestamp, correlation ID, sender, receiver, action, status and optional duration. A Python dataclass keeps the structure explicit:

from dataclasses import dataclass
from datetime import datetime

@dataclass
class Event:
    timestamp: datetime
    trace_id: str
    sender: str
    receiver: str
    message: str
    status: str | None = None

The correlation ID is central to the diagram. In a web application it may be called request_id, trace_id or transaction_id. If the source logs do not include one, the parser can group records by a short time window and client address, though this is less reliable. The tool should also normalise service names so that api-01, api-02 and api can be displayed as a single participant when required.

Parse common log formats safely

A regular expression can handle predictable text logs, such as Apache access records or application lines formatted as timestamp service action trace_id. For JSON logs, Python’s json.loads() is preferable because it avoids fragile field extraction. A parser interface allows multiple formats to share the same diagram-building code:

import json

def parse_json_line(line: str) -> Event:
    item = json.loads(line)
    return Event(
        timestamp=datetime.fromisoformat(item["timestamp"]),
        trace_id=item["trace_id"],
        sender=item["service"],
        receiver=item.get("target", "unknown"),
        message=item["event"],
        status=item.get("status"),
    )

Malformed lines should be reported with a line number rather than stopping the entire conversion. A command-line flag such as --strict can switch between warning mode and failure mode. This matters when analysing large files collected from a Linux server in Adelaide or a cloud workload serving customers throughout Australia, where one damaged record should not hide the rest of an incident.

Timestamps need careful treatment. Convert all values to timezone-aware datetime objects, then sort them in UTC internally. Displaying them in AEST, AEDT or the operator’s chosen zone can happen at the rendering stage. Storing naive local times can produce misleading sequences during daylight-saving changes, especially when logs from Melbourne and Perth are combined.

Build a sequence from correlated events

Once events are parsed, group them by trace ID and sort each group by timestamp. The simplest interpretation treats every event as a message from sender to receiver. More advanced logs can pair a request with its response using a span ID, allowing the generator to show activation periods, return arrows and elapsed time.

A sequence builder might produce an intermediate structure like this:

def build_sequence(events: list[Event]) -> dict:
    grouped = {}
    for event in events:
        grouped.setdefault(event.trace_id, []).append(event)

    for trace_id in grouped:
        grouped[trace_id].sort(key=lambda event: event.timestamp)

    return grouped

Avoid placing raw log text directly into diagram syntax. Escape quotes, newlines and control characters, and restrict participant names to safe identifiers. Long payloads should be shortened with a configurable limit. A --trace abc123 option is useful for isolating one request, while a --since and --until range can reduce a noisy production file to a manageable time interval.

Render PlantUML or Mermaid output

PlantUML is a strong target for a command-line tool because it supports participants, notes, loops, alternatives and many output formats. The renderer can turn each event into a message:

def render_plantuml(events: list[Event]) -> str:
    lines = ["@startuml"]
    participants = sorted({e.sender for e in events} | {e.receiver for e in events})

    for name in participants:
        lines.append(f'participant "{name}"')

    for event in events:
        label = event.message.replace('"', '\\"')
        lines.append(f"{event.sender} -> {event.receiver}: {label}")

    lines.append("@enduml")
    return "\n".join(lines)

Mermaid is convenient for Markdown documentation and browser-based viewers, while PlantUML suits generated reports and CI pipelines. The project can support both through separate renderers that consume the same event sequence. This separation keeps formatting decisions out of the parser and makes future formats easier to add.

A useful diagram includes a title, selected trace ID and time range. For repeated polling or retry behaviour, group identical messages into loops or add a retry counter instead of drawing dozens of indistinguishable arrows. When a request crosses an Australian payment provider, government service or external SaaS endpoint, clearly marking that boundary can make ownership and latency easier to understand.

Add a dependable command-line interface

Use argparse to expose the main operations: input path, format, output format, trace filter, timezone, maximum message length and strict parsing. A typical command could be:

python logseq.py access.log \
  --format json \
  --trace 7f31 \
  --output plantuml \
  --timezone Australia/Sydney

The program should write diagrams to standard output by default, allowing users to redirect them into a file or pipe them into another process. Exit codes should distinguish successful conversion, invalid arguments and parsing failures. Logging diagnostics to standard error keeps generated UML clean and suitable for automated workflows.

Testing should cover timestamp ordering, duplicate events, absent receivers, malformed records, daylight-saving transitions and escaping characters. Property-based tests can generate varied event combinations to check that every rendered message belongs to a known participant. A small fixture set representing API requests, database calls and failed retries provides useful regression coverage for an open-source release.

Protect data and package the utility

Log files frequently contain email addresses, IP addresses, session tokens and request bodies. Before generating a diagram, provide redaction rules for credentials, bearer tokens and personal information. Configuration can replace values with stable hashes, preserving relationships without exposing the original data. This is a practical safeguard for teams subject to the Australian Privacy Act and for consultants sharing incident diagrams with clients.

Package the tool with pyproject.toml, type hints and a clear licence. A --version option, man-page-style help and installation instructions make it easier for Linux users to adopt. Continuous integration can run tests across supported Python versions and verify that generated PlantUML remains valid.

A useful first release should favour predictable output over elaborate visual features. Once the parser, correlation logic and renderer are reliable, additions such as HTML reports, latency annotations, parallel branches and live log streaming can build on the same event model. The result is a focused Python development utility that turns opaque operational records into diagrams developers can inspect, share and maintain.