Building a Linux Daemon for Automatic Log Rotation and Compression

Managing logs is one of those quiet chores that keeps a server room in Brisbane or a home lab in Perth humming along. When a service is left unattended, log files balloon until they fill the disk, at which point everything from authentication to web serving starts misbehaving. A well-designed background process that handles rotation, retention, and compression on its own removes that risk and lets you focus on shipping features.

The Python ecosystem, the same community that produced tools like DenyHosts and Kodos, offers plenty of building blocks for crafting such a service. With a small amount of code, a handful of Linux conventions, and an eye on local compliance requirements like the Privacy Act 1988, you can deploy a reliable watcher that prunes and archives logs around the clock.

Understanding the Linux Log Lifecycle

Most distributions ship with logrotate, a venerable utility that handles the heavy lifting on Debian, Red Hat, and Arch-based systems alike. It is excellent for predictable schedules, yet it falls short when you need conditional behaviour, such as triggering a rotation only when a file exceeds a threshold tied to application health. That is where a custom daemon earns its keep.

Linux writes logs to a range of locations: /var/log/syslog on Debian-family boxes, /var/log/messages on older Red Hat systems, and journald's binary store on modern systemd hosts. Your daemon needs to know which paths to watch, whether the process runs as root, and how to interact with SELinux or AppArmor policies that may block writes. In Australian data centres, particularly those serving government clients under the Australian Government Information Security Manual, you may also need to retain logs for several years, so the design must accommodate long archival windows.

Crafting the Daemon Core

A simple structure built on Python's standard library is enough to get started. Use the daemon module or a lightweight alternative like python-daemon to fork off the parent process, redirect file descriptors, and write a PID file under /var/run. The main loop should poll watched files at a sensible interval, perhaps every sixty seconds, and compare current size against configurable limits.

Keeping configuration in a single TOML or YAML file makes deployment across a fleet of hosts painless, whether they sit in a Sydney colocation facility or a regional office in Hobart. Take inspiration from the design decisions behind DenyHosts' default deny thresholds, where careful tuning of limits turned a simple blocker into a dependable guardian. The same principle applies here: a rotation threshold that is too aggressive wastes CPU on compression, while one that is too lax risks filling the disk between scheduled runs.

Handling Compression and Storage Limits

Once a file crosses the rotation threshold, the daemon renames it with a timestamp suffix and spawns a compression worker. The choice of algorithm affects both storage cost and the time it takes to search archived logs during an incident, which is worth weighing carefully.

Common compression formats worth considering:

• gzip — universal compatibility, predictable CPU cost, ideal for small fleets.
• zstd — fast decompression, strong ratios, popular on AARNet-connected university clusters.
• bzip2 — excellent density for cold storage where read speed is irrelevant.
• xz — highest compression ratio, useful for monthly archives retained for years.

Retention policy deserves careful thought. A common pattern is to keep daily files for fourteen days, weekly for eight weeks, and monthly for two years, which aligns with typical guidance from the Office of the Australian Information Commissioner for breach investigations. Store older archives on a separate mount point or push them to object storage through the S3 protocol, which works with AWS S3 as well as local services like those offered by Canberra-based providers. Always verify that compressed archives preserve permissions and ownership if your compliance regime cares about chain of custody.

Running Safely with systemd

Modern Linux distributions expect services to integrate with systemd, and a custom daemon is no exception. Write a unit file that declares the user, restart policy, and resource limits, then enable it with systemctl. Hardening directives such as ProtectSystem, NoNewPrivileges, and PrivateTmp add layers of defence that match the Essential Eight maturity model promoted by the Australian Cyber Security Centre.

Hardening directives for the unit file:

• ProtectHome=true shields user directories from the daemon if it ever runs with broader permissions.
• RestrictAddressFamilies limits the socket families the process can open, reducing lateral movement risk.
• SystemCallArchitectures=native blocks exotic syscalls that an attacker might exploit after a compromise.
• CapabilityBoundingSet drops capabilities the daemon does not require, a small but valuable step.

Australians running personal servers on the NBN often appreciate how lightweight this approach is compared to container orchestration. Logging from the daemon itself should go through stdout when running under systemd, letting journald handle storage and letting you query recent activity with journalctl.

Observability and Long-term Maintenance

A daemon that works silently for months can still fail in subtle ways, so surface its behaviour through metrics. Expose counters for rotated files, compressed bytes, and errors over a Prometheus endpoint, and graph them in Grafana alongside disk usage. Alerting rules that fire when the rotation queue grows or when free space drops below a comfortable margin give you time to act before customers notice.

Testing deserves a slot in the development cycle too. Run the daemon against a synthetic workload that generates logs at varying rates, including a simulated disk-full condition, and verify that it recovers gracefully. Document the configuration file in a README that mentions common Australian deployment scenarios, such as running behind a Telstra-supplied modem or inside a corporate network managed by AARNet. With thoughtful design and steady maintenance, your background process will quietly keep logs tidy for years.