Advertisement
Open Source Projects by Phil Schwartz

Automating Missing Package Installation with Python

Python scripts often depend on software outside the Python runtime. A program may need git, curl, ffmpeg, a compiler, or a library such as libpq-dev before it can start. When that dependency is absent, an otherwise useful tool stops with an unclear error message.

A small dependency bootstrapper can detect missing commands, identify the operating system’s package manager, and install the required system packages. This approach is useful for Linux utilities, deployment scripts, test environments, and open-source projects distributed to different machines.

The safest design keeps application logic separate from package management. The script should inspect the host, explain what it intends to change, request permission, and report failures clearly. It should never assume that every machine uses Debian or that the current account has administrator access.

This matters in Australia, where a project might run on an Ubuntu VPS in Sydney, a Raspberry Pi in regional New South Wales, or a workstation managed by an organisation in Melbourne. Internet connections and hosting arrangements vary, so dependable diagnostics and repeatable installation matter more than a one-line command.

Define What the Program Actually Needs

Start with a clear dependency list. A shell command, a Python import, and a shared library are different types of requirement. For example, the command git usually maps to the Debian package git, while the Python import yaml maps to python3-yaml.

Keep distribution package names in a configuration structure rather than scattering them through application code:

DEPENDENCIES = {
    "commands": {
        "git": "git",
        "curl": "curl",
        "gcc": "build-essential",
    },
    "python_modules": {
        "yaml": "python3-yaml",
    },
}

This separation makes the script easier to extend. It also prevents a common mistake: installing a package named after an import when the distribution uses a different name.

Detect the Available Package Manager

Linux systems commonly provide apt-get, dnf, or pacman. Use shutil.which() to find an executable instead of relying on a hard-coded path. A package manager should be selected explicitly, with an informative error when the operating system is unsupported.

import shutil

def package_manager():
    for name in ("apt-get", "dnf", "pacman"):
        path = shutil.which(name)
        if path:
            return name, path
    raise RuntimeError(
        "No supported package manager was found. "
        "Install dependencies manually."
    )

Package repositories may be configured differently on a laptop, a cloud server, or a company network. A Sydney-hosted virtual machine may use an Australian mirror, while a regional office could have a filtering proxy. The installer should therefore report repository errors rather than pretending that a package was installed.

Find Missing Commands And Modules

Command detection is straightforward: shutil.which() returns a path when an executable is available. Python module detection can use importlib.util.find_spec() without importing and running the module.

import importlib.util
import shutil

def missing_dependencies():
    missing_packages = set()

    for command, package in DEPENDENCIES["commands"].items():
        if shutil.which(command) is None:
            missing_packages.add(package)

    for module, package in DEPENDENCIES["python_modules"].items():
        if importlib.util.find_spec(module) is None:
            missing_packages.add(package)

    return sorted(missing_packages)

A package can be installed while a command remains unavailable because the executable directory is absent from PATH. Likewise, a Python module may be installed into a virtual environment different from the interpreter running the script. Always test with the same Python executable that will run the application.

Build Safe Installation Commands

Never construct a shell command by joining untrusted strings and passing it to shell=True. Package names should come from a trusted internal mapping, and subprocess.run() should receive a list of arguments.

import os
import subprocess

def install_packages(manager, packages):
    if manager == "apt-get":
        command = ["apt-get", "update"]
        if os.geteuid() != 0:
            command.insert(0, "sudo")
        subprocess.run(command, check=True)

        command = ["apt-get", "install", "-y", *packages]
    elif manager == "dnf":
        command = ["dnf", "install", "-y", *packages]
    else:
        command = ["pacman", "-S", "--needed", "--noconfirm", *packages]

    if os.geteuid() != 0 and manager != "apt-get":
        command.insert(0, "sudo")

    subprocess.run(command, check=True)

The -y and --noconfirm flags make the process suitable for automation, but they also remove an interactive safety pause. A production script should display the package list and ask for confirmation unless it is running in a deliberately non-interactive deployment.

Handle Administrator Access Carefully

Installing operating-system packages changes the host and normally requires root privileges. On Ubuntu, the usual path is sudo apt-get install; on a controlled container or build runner, the process may already run as root. Do not silently collect passwords or attempt to bypass access controls.

A useful installer can offer an explicit --yes option and otherwise ask for approval:

def approved(packages, automatic=False):
    print("The following system packages are missing:")
    print("  " + " ".join(packages))

    if automatic:
        return True

    answer = input("Install them now? [y/N] ").strip().lower()
    return answer in {"y", "yes"}

Some Australian workplaces apply the Essential Eight security controls or restrict administrative rights on employee devices. In those environments, a script should produce a copyable command and exit cleanly when elevation is unavailable. That behaviour is more useful than repeatedly invoking sudo.

Support Distribution Differences

Package names are not universal. Debian and Ubuntu use apt, Fedora and newer Red Hat systems commonly use dnf, and Arch Linux uses pacman. macOS uses Homebrew rather than a Linux package manager, while Windows generally needs a different installation strategy.

Keep platform-specific logic behind a small interface. You might maintain separate dependency maps such as debian_packages, fedora_packages, and arch_packages, then select one after examining /etc/os-release. Avoid guessing based only on platform.system(), because that identifies the kernel family but not the distribution or release.

For tools deployed across Australia, this distinction is practical. A developer in Brisbane may use macOS, a Linux server in Perth may run Debian, and a Raspberry Pi used for monitoring in Tasmania may use Raspberry Pi OS. Clear unsupported-platform messages make those differences manageable.

Test Results And Preserve Useful Logs

The main program should call the bootstrapper before it imports optional modules or starts external commands. If installation fails, catch subprocess.CalledProcessError, include the command and exit status, and leave the original package-manager output visible.

def ensure_dependencies():
    missing = missing_dependencies()
    if not missing:
        return

    manager, _ = package_manager()
    if not approved(missing):
        raise SystemExit("Required packages were not installed.")

    try:
        install_packages(manager, missing)
    except subprocess.CalledProcessError as error:
        raise SystemExit(
            f"Package installation failed with exit code "
            f"{error.returncode}."
        )

Test on a clean virtual machine or container rather than only on a familiar workstation. Run the script twice to verify idempotence: the first run should install dependencies, while the second should report that nothing is missing. Record timestamps and package-manager output when the script runs in unattended jobs, particularly on cloud instances where logs may be the only evidence available.

A reliable Python dependency installer is deliberately modest. It detects known requirements, uses the host’s native package manager, respects administrator controls, and fails with enough information for a person to fix the environment. That makes it suitable for maintainable Linux tools rather than a fragile collection of shell shortcuts.