Securing Log Backups with Python's zipfile Module

As a senior software developer based in Melbourne, I've spent years building open-source tools that scratch my own itches. Projects like DenyHosts and Kodos came out of specific frustrations with existing software. When the compliance team flagged that our server logs were sitting around in plain text on a production box, the problem felt familiar: another gap that a small Python utility could close. The Notifiable Data Breaches scheme means a leak of logs containing user identifiers can trigger mandatory reporting, so at-rest encryption is not optional.

The temptation was to reach for a heavy-duty backup suite, but I wanted something I could read in one sitting and debug at 2am during a weekend on-call shift. Python's standard library has always been my first stop, and zipfile looked like the obvious candidate. It handles compression, produces archives any sysadmin can open with stock tools, and runs everywhere from a Raspberry Pi in a Brisbane colo rack to a beefy production server in Sydney.

The reality is that the built-in module historically supported only the ancient ZipCrypto stream cipher. That cipher has been considered broken for years, so leaning on it for anything beyond casual compression would have been negligent. I needed genuine AES-encrypted archives without abandoning the familiar ZIP container format, which meant layering a small dependency over the standard library.

The compliance trigger

The push to encrypt our log archives came after a routine review by our infosec lead. Australia's Notifiable Data Breaches scheme, which sits under the Privacy Act 1988, forces organisations to report incidents where personal information is likely to result in serious harm. Application logs often contain email addresses, IP addresses, and session tokens — exactly the kind of data the regulator cares about. Leaving months of those files on a server felt like an unforced error waiting to happen.

The team agreed on a two-part requirement: archives had to be encrypted at rest, and they had to remain portable enough that someone in another timezone could decrypt them with common tooling. ZIP was the format everyone already understood, which kept the human factor manageable.

Why zipfile was the obvious starting point

Python's zipfile module has been part of the standard library since the early days, and that longevity counts for something. It's well-documented, the API is stable, and it doesn't pull in a forest of transitive dependencies. For a backup job running nightly, predictable behaviour matters more than clever features.

There was a practical appeal, too. ZIP archives are understood by every operating system out of the box. If I got hit by a bus and someone else had to recover last Tuesday's logs, the on-call engineer shouldn't need to install a bespoke toolchain.

The encryption gotcha

The first script I wrote was deceptively simple: walk a log directory, add each file to a ZIP, set a password, and ship it off-site. Then I read the fine print. The ZipFile.setpassword() method uses ZipCrypto, a cipher from the early 1990s that modern hardware cracks in minutes. There is no encrypt=True flag, no AES support, and no clean way to rotate the encryption scheme.

The fix was small but important. I brought in pyzipper, a third-party library that writes ZIP files using the WinZip AES-256 specification. Behind the scenes, pyzipper extends the same conceptual workflow as zipfile — you open a target, write entries, close — but it negotiates the AES handshake for you. The resulting archive is still a standard ZIP, but the contents are encrypted with a cipher that has held up to scrutiny.

Building the wrapper script

The script itself ended up being shorter than the design document. A LogsArchiver class wraps pyzipper.AESZipFile, accepting a source directory and a destination path. The class walks the source using os.walk(), filters out files modified within the last hour to avoid grabbing half-written logs, and writes each entry with ZIP_DEFLATED compression. Compression level six has been a good compromise between CPU cost and archive size.

Error handling needed care. Disk full conditions, permission errors on locked log files, and corrupted source files all had to fail gracefully. I used a temporary file in the same directory as the destination, then atomically renamed it once the archive closed successfully. That way an interrupted run never produces an archive that looks valid but decrypts to garbage.

Automation and off-site syncing

The job runs nightly via cron, scheduled at a quiet time after the logs have rotated but before the morning shift starts checking dashboards. Cron's timezone handling is famously quirky, so I made the schedule explicit by setting the CRON_TZ environment variable to Australia/Sydney and embedding AEST in the filename. That keeps the archived logs grouped by the local business day, which matters when someone is hunting for an event from Tuesday arvo.

Once the archive lands on the local box, a small rsync over SSH pushes it to a secondary location. For Australian operations, that secondary is typically a different provider's data centre — often one in Sydney serving the east coast, with a secondary copy landing in Perth for geographic redundancy.

Verifying and rotating the archive pile

Encryption is only useful if you can also decrypt. Every archive gets a verification pass after creation: open it, read the first entry, hash it, compare against the original. If the digests don't match, the archive is quarantined and the on-call gets paged. This step catches the rare cases where a disk was reporting success while actually writing zeros.

For deeper integrity work across the archive pile, I lean on a Python verification utility I keep handy for exactly this purpose. It walks a directory of encrypted archives, runs sanity checks, and reports anything that looks stale or corrupted. Rotation is straightforward: daily archives kept for thirty days, weekly archives for a year, monthly archives shipped to long-term cold storage.

Key management for long-term archives

Encrypted archives that sit untouched for years are a liability if the encryption keys are lost. Storing the recovery key alongside the archive defeats the purpose, so the key lives in a separate secrets manager with audited access. A forgotten key in a vault somewhere is just as much a data breach as a leaked plaintext file — it just takes longer to notice.

Key rotation adds another wrinkle. When the AES key changes, older archives still need to be readable. I keep a small mapping of archive age ranges to key versions, and the decryption wrapper looks up the right key based on the archive timestamp. It is the kind of operational hygiene that separates a reliable archive strategy from a ticking compliance time bomb.