Log File Merger

Interleave several log files into one timeline · per source timezone override · correlation window · silence gaps

Drop the log files

Or paste one

Merge

What This Tool Answers That a Chatbot Cannot

Ask a chatbot to merge three log files and it will give you a shell one liner, sort the lines, and move on. That answer is wrong in a specific way that is easy to miss, because the output still looks like a timeline. Shell sort compares the beginning of each line as text, so the order it produces depends on the first character of the timestamp and on nothing else. It cannot see a zone offset written halfway through an Apache line, and it has no idea what clock the machine that wrote the file believed it was on. When the files come from different services, different containers and different time zones, the merged file comes out sorted and still tells you the wrong story, which is worse than no file at all because it looks authoritative.

This tool parses the timestamp out of every line, converts each one to a real instant, and only then sorts. If a line carries its own offset, that offset wins. If it does not, the offset you set for that source is applied and the tool says so in the summary, so a guess never passes itself off as a reading. The result is a timeline you can check against something you already know, such as the moment a customer complained.

Why Sorting Text Is Not Sorting Time

Three separate problems hide behind the word merge. The first is that timestamps are written in several shapes. ISO 8601 with a trailing Z is unambiguous. An Apache combined line hides its time inside square brackets, followed by a numeric offset with no colon. A syslog line carries a month name, a day of month and a time with no year and no offset at all. A Unix epoch is a bare integer. A JSON log line may put the time under any of a handful of key names. Any tool that assumes one shape silently loses the rest, usually by dropping the lines it cannot read or by parking them all at the start of the file.

The second problem is the missing year. RFC 3164 syslog carries a month name and a day but no year, which is why every syslog file that spans New Year has a line that looks eleven months out of place. This tool assumes the current year for those lines and states the assumption in the source table, so you can correct it before trusting the order.

The third problem is the clock the writer thought it was on. An application that logs a naive local time with no offset writes something a reader cannot place on a timeline without help. Two containers, one configured for UTC and one left on Europe/Berlin, produce files whose lines look two hours apart when they are actually simultaneous. That is why each source here has its own offset box. Setting it correctly is the single most useful thing you can do with this tool.

Correlation, Not Just Sorting

A merged file answers what happened in what order. It does not by itself answer what else was happening at the same moment, which is the question people actually have during an incident. Click any line and every entry written by any source within the correlation window you set is highlighted, so a request that failed at one instant can be read next to the database error, the deploy line and the health check that stopped passing around the same second. Without that window, finding the related lines means scrolling and comparing timestamps by eye, and with a few thousand lines nobody does it.

The window is a number rather than a guess because the right value depends on your system. A request that completes in forty milliseconds belongs in a window of a few seconds. A batch job that runs every fifteen minutes belongs in a window of minutes. The default is five seconds, which suits request logs, and the field takes anything up to an hour.

Silence Is a Signal

The most useful row in a merged timeline is often the one that is missing. When a service logs nothing at all for a minute while its neighbours flood the file, that hole is either the cause or the first symptom. Set a threshold and every interval longer than it inside each source is marked in place, with the exact duration, so a forty seven second gap in the worker log lines up with the errors in the application log instead of hiding between two files.

Gaps are computed per source, not across the merged file, because only a per source gap is meaningful. A window with no lines at all in a file that is being written every second is a gap. The same window with no lines in a file that logs once a day is not.

Formats This Tool Recognises

Detection runs on the first timestamp-shaped run of characters in each line, and the format it picked is shown for every source so you can see when it guessed. Nothing is dropped silently: lines whose timestamp could not be read are counted per source and listed, because a merge that quietly loses lines is the failure mode you are trying to avoid.

Log timestamp formats detected by the merger, with an example and whether the offset is written in the line
FormatExampleOffset in the text
ISO 8601 / RFC 33392024-01-15T10:30:00.123ZYes
ISO 8601 with offset2024-01-15T11:30:00+01:00Yes
Apache or nginx combined[15/Jan/2024:10:30:00 +0000]Yes
AWS ALB access log[15/Jan/2024:10:30:00.123456 +0000]Yes
Apache error log2024/01/15 10:30:00No
Syslog RFC 3164Jan 15 10:30:00No, and no year
Unix epoch seconds1705312200Implied UTC
Unix epoch milliseconds1705312200123Implied UTC
JSON linestime, ts, timestamp, asctimeWhatever the value says

Every source can be forced to a single format if detection picks the wrong one, which happens with custom application logs that happen to start with a bracketed number.

How It Works

Everything runs in this page. Files are read with the browser file reader, split into lines, and each line is matched against the format list in order until one produces a timestamp. Lines that match nothing are kept in the source but excluded from the timeline, and reported. The surviving entries are sorted on their epoch value with the source order and line number as tie breakers, so two entries stamped with the same millisecond always come out in a fixed order and the export matches the screen. No request carries your logs anywhere after the page loads.

Large files are trimmed for display rather than for correctness. The merged array always holds every parsed line, and the export always writes all of them, but the on screen list is paged so a two hundred thousand line file does not lock the tab. A count tells you how many rows are hidden.

Pairs Well With

  • Timestamp Converter — turn any single instant here into Unix time, ISO or another zone.
  • Text Diff — compare two merged timelines and see exactly what a deploy changed.
  • JSON Formatter — tidy JSON lines before feeding them in.
  • Regex Tester — pull a request id out of the merged file and follow it across every source.
Do my log files get uploaded anywhere?

No. Files are read in the page with the browser file reader and the parsing runs on your machine. Once the page has finished loading there is no request that carries your log content to any server, there is no account, and nothing is stored between visits. That matters more here than on most tools, because logs are the one kind of input that routinely contains tokens, internal hostnames, user identifiers and request payloads.

Why do the lines come out in the wrong order when I do not set an offset?

Because a line with no offset written in it carries no information about which clock produced it. A syslog line reading Jan 15 10:30:00 is a different instant depending on whether the writing machine was on UTC or on Central European Time, and nothing in the line can tell you. This tool reads such a line as if it were already UTC, which is the common default, and tells you in the source table that the offset was assumed. Set the box next to that source to the machine's actual offset and the timeline snaps into place.

Why use this instead of sort on the command line?

Sort compares text, and it is genuinely the right tool when every line starts with the same fixed width timestamp in the same zone. It stops being the right tool the moment one file hides its time inside square brackets with an offset in the middle, one file writes a bare epoch, and one file omits the year entirely. Sort also cannot tell you which lines it could not read, cannot apply a per file offset, cannot highlight a correlation window, and cannot mark the gaps. For one clean file it is faster. For three files from three different systems during an incident, this page gives you an answer you can check.

How big a file can it handle?

The parser is written for ordinary log files and handles thousands of lines comfortably in the browser. Nothing is truncated in the stored data: every line that yields a timestamp goes into the merged array and into every export, no matter how large the file is. What is limited is the on screen list, which is paged so that a very large file stays responsive, and the browser memory ceiling that every local tool has. If a file is tens of megabytes it is still worth trying, but splitting or tailing it first will feel faster.

What happens to lines it cannot read?

They are not merged and they are not thrown away. Each source reports how many lines it contributed to the timeline and how many were skipped, so a file where a third of the lines were unreadable cannot pass as a clean merge. Skipped lines stay in the source's count and in the format note, which is the honest place to surface a detection failure.

What is the correlation window for?

It is the half width, in seconds, of the highlight you get when you click a line. With the default of five seconds, clicking an error in the application log also lights up whatever nginx, the database proxy and the deploy script wrote in the five seconds on either side of it. Set it to zero to turn the highlight off, or widen it for slower systems where a request takes minutes rather than milliseconds.

Does it change the timestamps in the exported file?

The export writes a new first column with the resolved instant in ISO 8601 UTC, then the source name, then the level, then the original line unchanged. The original text is never rewritten, so you still have the exact bytes your application wrote. Every timestamp your application printed is also kept, which means you can always undo the merge and check it against the source files.

Can I merge logs from two systems that log completely different things?

Yes, and that is the normal case. The sources only have to agree on what a line is, not on what it means. An nginx access log, an application JSON log and a syslog stream from the same period merge into one timeline, which is exactly the comparison you cannot make by eye while staring at three separate terminals.

Comments & Ratings

Be the first to comment.

References