How I built a Python command-line tool to diff two JSON structures

Comparing JSON files sounds simple until the documents become large, deeply nested, or generated by different systems. A line-by-line comparison quickly becomes noisy: a reordered object appears changed, whitespace creates false differences, and a single missing value can be hidden among hundreds of unrelated lines.

I built a small Python command-line utility to compare JSON structures semantically rather than textually. The aim was to make configuration reviews, API troubleshooting, fixture checks, and deployment investigations faster for developers working in a terminal.

Defining the real comparison problem

The first design decision was to compare parsed data rather than raw files. Python’s json module converts an input document into dictionaries, lists, strings, numbers, booleans, and None. Once both files are represented as native objects, formatting and object-key order no longer affect the result.

That distinction matters in everyday development. A service in Sydney might emit compact JSON while a local test fixture in Melbourne uses four-space indentation. Those files can be structurally identical even though a traditional diff reports changes across nearly every line. The tool therefore treats JSON as data, with the file format acting only as the transport layer.

I also wanted the command-line interface to behave predictably in shell scripts. A clean comparison returns a success status when the structures match and a different exit code when they do not. That makes the utility useful in continuous integration, pre-deployment checks, and simple shell pipelines without forcing another program to interpret human-readable output.

Walking nested objects and arrays

The core algorithm recursively visits each value and carries a path describing its location. For an object, the path can use a familiar notation such as server.port or database.credentials.username. At every level, the program builds the union of keys found in both documents, then identifies additions, removals, and changed values.

Arrays need a deliberate policy because they are ordered collections. In the default mode, the item at index zero is compared with the item at index zero in the other document. This preserves meaningful changes in lists such as firewall rules, command arguments, or ordered middleware. A missing or extra element is reported with its array index, giving the developer a precise location rather than a vague “list differs” message.

There are cases where order is incidental, such as a collection of feature names returned by an API. I kept that behaviour optional rather than silently sorting every list. A --unordered mode can compare normalised members, but the user must opt into it because sorting complex dictionaries or duplicate values can hide meaningful changes.

Making differences useful at a glance

A diff is valuable only when its output helps someone decide what to do next. Each difference includes a type, a path, and the two relevant values. For example, a result might say that logging.level changed from "info" to "debug", that cache.ttl was added, or that users[3].email disappeared.

The default renderer is designed for terminal use. Added values are marked clearly, removed values are separated from changes, and long strings are escaped so newlines and tabs do not distort the display. I avoided dumping entire documents because that recreates the noise the tool is meant to remove.

Machine-readable output is equally important. A JSON or simple line-oriented mode allows another Python script, a CI job, or a monitoring process to consume the result. This also fits the wider collection of small development utilities, where focused tools are often more useful than a large framework with unrelated dependencies.

Handling numbers, nulls, and malformed input

JSON has fewer data types than Python, but comparisons still need care. false, 0, null, and an empty string are all distinct values and must never be collapsed into a single false-like result. The recursive comparator checks type and value explicitly, preventing a boolean from being treated as an integer simply because Python’s type hierarchy makes them related.

Floating-point values introduce another practical decision. Exact comparison is appropriate for many configuration files, while calculated API responses may need a tolerance. I kept exact equality as the default and added an optional numeric tolerance rather than applying rounding invisibly. A configuration review should not quietly overlook a changed timeout or threshold.

Malformed JSON receives a short, actionable error that includes the filename and parser location. The program exits with a failure status instead of producing a partial comparison. This is particularly useful when checking generated files on an NBN-connected development machine or in a build runner, where the real problem may be a truncated download or an interrupted export.

Testing the command across real workflows

Unit tests cover each recursive case: identical primitives, changed values, missing keys, additional keys, nested objects, empty arrays, reordered arrays, and arrays containing dictionaries. I also use property-style tests for invariants such as comparing a document with itself, which should always produce no differences.

Command tests verify exit statuses, standard output, standard error, and option handling. Temporary files make it possible to exercise realistic invocations without relying on a developer’s working directory. The same tests run on Linux and other supported platforms, keeping path handling and encoding assumptions visible.

Some fixtures reflect Australian operational realities. A configuration may contain an Australia/Sydney timezone, a GST rate represented as a decimal, or a list of state abbreviations used by a local service. These examples are ordinary JSON values, but they help ensure that the tool behaves sensibly with Unicode text, decimal-looking numbers, and region-specific settings used by Australian businesses.

Packaging it as a dependable Python tool

The utility is structured as a small package with a reusable comparison module and a thin CLI entry point. This separation means another Python program can call the diff engine directly, while terminal users get argument parsing, formatted output, and exit codes. Keeping dependencies minimal also makes installation easier on a fresh Linux server or a developer laptop.

The command supports standard input for one side of the comparison, allowing patterns such as piping an API response into the tool. Options control output format, array ordering, numeric tolerance, and whether unchanged paths are displayed. Help text includes examples, because command-line tools are often discovered months after they were written.

Licensing and documentation are part of the implementation rather than an afterthought. Australian teams handling customer or employee records must consider obligations under the Privacy Act 1988, so the tool should avoid logging complete sensitive documents by default. Clear warnings about output content, sensible file permissions, and an explicit licence make it safer to adopt in a commercial environment, whether the code is used by a small Brisbane consultancy or a larger Melbourne engineering group.