Advertisement
Open Source Projects by Phil Schwartz

How Scratchy Helped Me Identify a DDoS Attack on My Own Server

I first noticed something was wrong when a server that normally handled web traffic quietly began responding slowly. The symptoms were easy to dismiss: occasional timeouts, delayed SSH sessions, and a rising load average that did not match the number of visitors shown in my application logs.

The server was running Apache, so its access logs contained a detailed record of every request. The difficulty was volume. A busy log can hide an attack inside routine activity, especially when the requests resemble ordinary page loads. I needed a quick way to see patterns rather than inspect individual lines.

That was when I turned to Scratchy, my Apache log analyzer. Its summaries gave me a useful view of clients, requested resources, response codes, and request frequency. The results transformed a vague performance problem into evidence of a distributed denial-of-service attempt.

What the Server Looked Like Before

Before the incident, the machine had a predictable traffic profile. Most requests came from a modest group of addresses, and visitors followed recognizable paths through documentation pages, downloads, and project descriptions. Static files accounted for much of the bandwidth, while dynamic requests remained relatively limited.

The server load usually rose during software releases or when a project received attention from a large technical forum. Those events produced genuine traffic: different user agents, varied URLs, and requests spread across several minutes or hours. A normal spike still had structure.

The suspicious period looked different. CPU usage increased sharply, Apache workers remained occupied, and the number of active connections stayed high even though there was no corresponding announcement or referral source. The application itself showed little activity, which suggested that the pressure was arriving at the web server layer.

Signals Hidden in Apache Logs

I started with the access log instead of guessing at the cause. Scratchy grouped entries by useful dimensions, including remote address, requested path, status code, and user agent. That immediately exposed repetition that was difficult to recognize while scrolling through raw text.

Thousands of requests targeted the same small set of URLs. Many clients requested the same resource over and over, often with nearly identical timing. The source addresses were distributed across different networks, but the behavior was remarkably consistent.

The user-agent data was also revealing. A normal audience tends to include browsers, command-line tools, crawlers, and different operating-system signatures. During the event, a large portion of requests used blank, unusual, or duplicated user-agent strings. This did not prove malicious intent by itself, but it strengthened the pattern shown by request frequency.

Turning Raw Requests Into Evidence

The most valuable part of the analysis was the ability to compare proportions. A list of 50,000 log lines is intimidating; a summary showing that one endpoint received most of those requests is actionable. Scratchy helped me identify the paths and clients that were driving the abnormal load.

I also compared response codes. The server was returning many successful responses, so this was not simply a scan for missing files. The attackers were requesting real resources and forcing Apache to perform enough work to consume connections and bandwidth.

The timing mattered as well. Requests arrived in bursts, then continued at a sustained rate. That combination pointed away from a single misconfigured crawler and toward coordinated traffic. I saved the relevant log segment before making changes, preserving a baseline for later comparison.

Observation Normal Traffic Attack Traffic
Source addresses Varied, with repeat visitors Numerous addresses with similar behavior
Requested URLs Distributed across site content Concentrated on a few resources
User agents Mixed browsers, tools, and crawlers Repeated or suspicious signatures
Request timing Uneven browsing patterns Bursts and sustained high frequency
Server effect Predictable resource use Exhausted workers and slower responses

Separating Attack Traffic From Normal Spikes

The comparison kept me from treating every busy period as a DDoS attack. A popular release can produce thousands of legitimate downloads, and an automated package mirror can look repetitive. I checked referrers, URL diversity, cache behavior, and whether clients completed requests in a way that resembled real use.

The attack traffic lacked the variety expected from a real audience. It did not explore related pages, follow links, or show meaningful interest in project documentation. Instead, it repeatedly consumed the same server-side resources. The requests were distributed enough to evade a simple single-address block, but similar enough to reveal coordination.

This distinction was important because an overly broad response could have blocked legitimate users. Scratchy provided the aggregate view, while Apache configuration, connection statistics, and system monitoring supplied the surrounding context. No single metric was decisive; the pattern across several metrics was.

The Response That Reduced Pressure

I first made a copy of the logs and recorded the server state. Then I tightened access to the most heavily targeted resources and adjusted Apache settings to limit how many connections could remain active. These changes were intended to reduce exposure without taking the entire site offline.

I added temporary rules for the clearest abusive sources and reviewed them carefully before reloading the service. Address blocking alone was not a complete solution because a distributed attack can rotate through many networks, but it removed some of the immediate noise.

The larger improvement came from placing frequently requested static content behind caching and moving filtering closer to the network edge. That reduced the amount of work reaching Apache. After each change, I used new Scratchy reports to compare request volume, status codes, and URL concentration with the saved incident period.

Habits That Made the Investigation Faster

The event changed how I think about log analysis. A server does not need to be completely unavailable before an attack deserves attention. Rising connection counts, repeated requests, and an unusual concentration on a few endpoints can provide an early warning.

My practical checklist now includes these steps:

Scratchy remains useful because it turns a large text file into a concise traffic profile. It is also part of a broader collection of development utilities that reflect the value of small, focused tools: each one answers a specific operational question without requiring a complicated monitoring platform.

The investigation also reinforced the importance of preparation. A baseline collected during healthy operation makes an incident easier to recognize, while clear logs make a defensive response less dependent on guesswork. Even a lightweight analyzer can be decisive when it reveals the shape of an attack quickly.

If your Apache server is showing unexplained latency or connection pressure, download Scratchy, analyze a representative log period, and compare the results with normal traffic. The sooner unusual request patterns become visible, the sooner you can protect the server and keep legitimate users online.