Build a periodic Python file purger by age and size
Old logs and build artefacts can quietly consume a server’s disk. To build a Python script that periodically purges old files based on age and size, begin with a deletion rule, a narrow root and a default dry run. The utility should report every candidate before anything disappears.
That fits the open-source focus of Phil Schwartz’s site. DenyHosts, Kodos and Scratchy solve narrow problems with Python and Linux tools, so a file purger should likewise offer source, usage notes, licence details and safe defaults rather than remain a hidden cron snippet.
Age is clearest as time since a file’s last modification, while size is its byte count. Both become risky when interpreted vaguely: “old” might mean yesterday, “large” might mean non-empty, and “delete” might hit an active database. Expose every threshold and keep the target directory explicit.
Australian operators will recognise the problem. A Sydney or Melbourne VPS can fill with rotated logs, while a Brisbane home lab may accumulate a heap of downloaded artefacts. Scheduling, permissions and local time matter as much as the file-selection rule.
Define a precise deletion policy
Choose whether age and size are combined or alternatives. The script below removes a regular file only when it is both older than the age threshold and larger than the size threshold. This conservative rule leaves tiny stale files alone unless deliberately included.
Modification time is only a policy signal, so document what it represents. A file still being written may have a recent timestamp but can still grow large. Schedule deletion after writers stop, or use an application marker such as a rotated suffix.
Implement the scanner in Python
A standard-library implementation is enough. It walks one root without following symlinked directories, ignores non-regular entries, validates arguments and separates planning from action with an explicit --execute flag.
#!/usr/bin/env python3
import argparse
import os
from datetime import datetime, timezone
from pathlib import Path
import stat
import time
parser = argparse.ArgumentParser()
parser.add_argument("root", type=Path)
parser.add_argument("--min-age-days", type=float, required=True)
parser.add_argument("--min-size-mb", type=float, required=True)
parser.add_argument("--execute", action="store_true")
args = parser.parse_args()
if args.min_age_days < 0 or args.min_size_mb < 0:
parser.error("thresholds cannot be negative")
root = args.root.expanduser().resolve(strict=True)
if not root.is_dir() or root == Path("/"):
parser.error("root must be a specific directory")
now = time.time()
min_age = args.min_age_days * 86400
min_size = args.min_size_mb * 1024 * 1024
for directory, _, filenames in os.walk(root, followlinks=False):
for filename in filenames:
path = Path(directory) / filename
try:
info = path.lstat()
if not stat.S_ISREG(info.st_mode):
continue
age = now - info.st_mtime
size = info.st_size
if age >= min_age and size >= min_size:
action = "DELETE" if args.execute else "WOULD_DELETE"
stamp = datetime.now(timezone.utc).isoformat(timespec="seconds")
print(
f"{stamp} {action} {path} "
f"age_days={age / 86400:.1f} "
f"size_mb={size / 1048576:.1f}"
)
if args.execute:
path.unlink()
except (FileNotFoundError, PermissionError) as exc:
print(f"SKIP {path}: {exc}")
Each match receives a UTC timestamp, age, size and action. Missing or unreadable files are reported and skipped, so one troublesome entry does not abort the run. The root check also refuses the filesystem root.
Test the dry run before deletion
Build a disposable test tree before touching production data. Create empty, small, large and recent files, set selected modification times with os.utime, and compare output with the expected result. Include nested directories and a symlink to confirm links are neither followed nor removed.
Run without --execute first and inspect the paths, thresholds and projected count. Repeat over several days so rotations and permission errors appear naturally. Once selection is correct, add --execute under a dedicated account rather than running it as root.
Schedule it around Australian operating hours
On Linux, a daily cron entry such as 15 3 * * * /usr/bin/python3 /home/phil/bin/purge-files.py /srv/logs/archive --min-age-days 30 --min-size-mb 100 --execute can call the script. Use an absolute interpreter path because cron has a sparse environment. Prevent overlap with a lock such as flock; when missed runs matter, a systemd timer can use Persistent=true.
Timezone handling deserves care in Australia. Sydney and Melbourne change daylight-saving time, while public holidays differ by state. Decide whether 03:00 means Sydney time, Melbourne time or UTC, state it in the timer, and avoid windows when local support teams or log writers are active.
Add safeguards for production use
Run the job as a dedicated unprivileged user with write permission only for the archive directory. Keep logs elsewhere, or the purger may remove its own audit trail. Retain a backup or quarantine volume briefly so an accidental selection can be recovered.
Avoid broad roots, mounted home directories and paths populated by active databases. Check for inode or device changes when files are replaced during a scan, and prefer a maintenance window for valuable data. A job-wide file lock also stops two timers selecting the same candidate.
Keep the utility open and useful
Publish the script with the transparency expected from DenyHosts and Kodos: source repository, licence, supported Python versions, sample configuration and concise examples. Version the policy separately so servers can pin a known-good threshold. Tests should cover threshold boundaries and permission failures.
Paid support can use Australian-dollar pricing and state whether GST applies; transparent licensing helps local teams choose suitable tools. Privacy is another concern, so keep logs in an approved Australian region when required. For contrast, the market comparisons in arbitrage betting explained depend on external prices; cleanup should depend only on recorded metadata.
Apply a practical cleanup checklist
Before enabling deletion, use a short verification routine:
- Restrict the root to a dedicated log or archive directory.
- Choose age and size limits from an observed retention cycle.
- Review a dry run and confirm the expected file count.
- Run under an account with directory-specific permissions.
At each scheduled execution, verify these controls:
- Log start time, candidates, deletions, errors and bytes released.
- Use a lock to prevent overlapping runs.
- Alert on failures or unexpectedly low reclaimed space.
Review results after a quiet Australian weekend window and then across a full business cycle. Measured growth should determine threshold changes, with every change documented. This keeps the script useful for a busy Melbourne studio or a small Canberra operation without turning a cleanup job into an unbounded delete command.