Using Python’s hashlib to verify files across multiple servers

A file copied between servers can look perfectly normal while containing a changed configuration, a damaged archive or an altered deployment artefact. Comparing filenames and modification times is useful for basic checks, but a cryptographic digest gives you a stronger way to determine whether the file’s contents are identical.

Python’s hashlib module makes this practical without adding third-party dependencies. You can calculate a digest on a reference machine, collect matching values from Linux hosts over SSH, and report differences in a format suitable for an operations log or CI pipeline. The approach works for a handful of servers in a Sydney office as well as a larger fleet spread across Australian cloud regions.

Select a suitable digest algorithm

A cryptographic hash converts a file into a fixed-length string. If even one byte changes, the resulting digest should change substantially. SHA-256 is a sensible default for integrity checks because it is widely supported, straightforward to explain and available through Python’s standard library.

MD5 and SHA-1 can still appear in legacy tooling, but they should not be chosen for new integrity workflows. Their collision weaknesses make them unsuitable where an attacker could deliberately create two different files with the same digest. SHA-512 is also available, although its longer output is rarely necessary for ordinary server file verification.

The following function reads the file in chunks rather than loading the entire object into memory:

from hashlib import sha256
from pathlib import Path

def file_digest(path, chunk_size=1024 * 1024):
    digest = sha256()

    with Path(path).open("rb") as file_obj:
        for chunk in iter(lambda: file_obj.read(chunk_size), b""):
            digest.update(chunk)

    return digest.hexdigest()

Reading one mebibyte at a time keeps memory use predictable for large database dumps, ISO images and log archives. Binary mode is essential: text-mode newline conversion would make the digest platform-dependent.

Build a trusted reference manifest

A comparison needs an expected value. Usually, that value is generated immediately after a package is built, a configuration bundle is approved or a release is deployed. Store the path, digest algorithm and expected hash in a manifest such as JSON:

import json
from pathlib import Path

files = ["/etc/ssh/sshd_config", "/opt/app/releases/current/app.tar.gz"]

manifest = {
    "algorithm": "sha256",
    "files": {
        name: file_digest(name)
        for name in files
    },
}

Path("manifest.json").write_text(
    json.dumps(manifest, indent=2) + "\n",
    encoding="utf-8",
)

The manifest itself must be protected. If an intruder can change both the target file and the expected digest, a verification script will report a false “match”. Keep the baseline in a restricted repository, sign it with a deployment key, or publish it through a trusted release system. For higher-assurance environments, verify a detached signature before using the manifest.

A digest proves that two byte streams match; it does not prove who created them. Access controls, secure key management, audit logs and a known-good build process provide that missing context. Australian organisations aligning server controls with the Essential Eight should treat hash verification as one control within a wider change-management and malware-defence process.

Collect hashes from remote Linux hosts

For a small fleet, SSH is often the least complicated transport. Python can execute a remote command with the standard ssh client, avoiding a dependency on a particular SSH library. The remote command runs a short Python expression, so the same hashing behaviour is used on every host.

import json
import subprocess

def remote_digest(host, path):
    script = (
        "import hashlib, sys\n"
        "h = hashlib.sha256()\n"
        "with open(sys.argv[1], 'rb') as f:\n"
        "    for chunk in iter(lambda: f.read(1048576), b''):\n"
        "        h.update(chunk)\n"
        "print(h.hexdigest())\n"
    )

    result = subprocess.run(
        ["ssh", host, "python3", "-c", script, path],
        capture_output=True,
        text=True,
        check=True,
    )
    return result.stdout.strip()

Use SSH keys, host-key verification and a restricted account rather than embedding passwords in a script. A deployment runner can iterate through a list of hosts and record whether each server is match, different, missing or unreachable.

For a larger estate, run a small agent or orchestration job on each machine and return structured JSON. Tools such as Ansible can distribute the check, while a central service stores results with timestamps and host identity. This is useful when servers sit across Sydney, Melbourne and Perth, where network latency and maintenance windows may vary.

Compare files safely and clearly

A verification result should distinguish a changed file from a failed connection. Treating every exception as a mismatch can trigger unnecessary incident response, while ignoring an unreachable host can create a dangerous blind spot.

def compare(expected, actual):
    if actual is None:
        return "unavailable"
    if expected is None:
        return "not-in-manifest"
    return "match" if expected == actual else "different"

For each host, include the server name, file path, expected digest, observed digest, UTC timestamp and error details where relevant. Avoid placing secret configuration contents in the report; a hash is generally safe to log, but paths and filenames can still reveal sensitive system structure.

Large files may change during hashing. A database export or active log can produce a digest that represents no stable point in time. Verify immutable release artefacts, pause writes where practical, use filesystem snapshots, or coordinate with the application so the source is consistent before calculation. This matters for busy workloads hosted on NBN-connected premises as well as cloud VMs.

Schedule checks across an Australian fleet

Periodic verification can detect drift after deployment, an unexpected package replacement or tampering between maintenance cycles. A nightly systemd timer or a scheduled CI job is usually enough for ordinary application files. Critical files may deserve checks after every deployment and at shorter intervals.

Australian operations teams also need to account for data location and business hours. An organisation serving customers in Queensland, New South Wales and Western Australia may schedule heavy scans outside each site’s peak period, while keeping timestamps in UTC for reliable comparison. If the manifest or results contain operational metadata covered by internal data-handling rules, choose storage in an approved Australian region, such as Sydney or Melbourne, when required by the organisation’s policy.

Use exit codes to integrate the checker with monitoring: zero for all matches, a distinct code for differences, and another for unavailable hosts. Alerting should identify the exact machine and path, then preserve the original result for investigation. A digest mismatch is evidence that contents changed, not automatic proof of compromise; compare it with deployment records, package updates and administrator activity before taking a host offline.