What “ATOMIC_MOVE” means for your file transfer pipeline

· ~4 minute read

Picture a nightly job that watches an SFTP inbox. Every time a file lands in /inbound/, the job picks it up, parses it, and loads it into a warehouse. It runs every 30 seconds, and it has one fatal assumption baked in: that any file it sees is a whole file.

Now picture a vendor uploading a 400 MB export over a flaky connection. If the SFTP server writes those bytes straight into /inbound/report.csv, then for the two minutes the transfer is in flight, that path exists — and it's half a file. Your job wakes up, grabs 180 MB of a 400 MB CSV, and loads a truncated row into the warehouse. Nobody gets an error. The data is just wrong.

ATOMIC_MOVE is the mechanism that makes that impossible. This is a short explainer on what it actually guarantees, why the guarantee only holds inside a single filesystem, and what Stylus SFTP Server does when it doesn't.

The guarantee: all-or-nothing visibility

An atomic move is a rename that either fully happened or didn't happen at all — there is no in-between state an observer can catch. On disk, renaming a file within one filesystem doesn't touch the file's bytes; it only rewrites a directory entry to point at data that's already fully written. That pointer swap is a single operation the operating system will not interrupt. A downstream consumer polling the directory sees the old state or the new state, never a partial one.

That's why Stylus SFTP Server never writes uploads directly to their final name. Every upload is written to a staging file first, and only when the client cleanly closes the channel is it renamed into place. The rename is the moment the file “appears.” In Java terms it's one call:

Files.move(stagingFile, target,
           StandardCopyOption.ATOMIC_MOVE,
           StandardCopyOption.REPLACE_EXISTING);

Your nightly job now only ever sees complete files. The in-progress upload is a .tmp staging file it never looks at.

The catch: “atomic” stops at the volume boundary

Here's the part that trips people up. That instant pointer-swap only works because the staging file and the target file live on the same filesystem. A directory entry can only point at data on its own volume. The moment you ask the OS to “move” a file from the C: drive to the F: drive — or across two Linux mount points — it can't just rewrite a pointer. It has to physically copy every byte to the new volume and then delete the original.

A copy-then-delete is not atomic. While the copy runs, the destination exists and is growing. That's exactly the half-a-file window we were trying to eliminate, back again. Java is explicit about this: request ATOMIC_MOVE across volumes and it throws AtomicMoveNotSupportedException rather than silently doing a non-atomic copy.

The one-line rule: an atomic rename needs the staging area and the destination on the same filesystem volume. Same disk, same mount, same drive letter.

What SSS does at the boundary

Stylus SFTP Server keeps every staging file in a centralized .staging/ directory under the writable data root, and it mirrors the target's full path underneath — a file bound for F:\homes\john\report.csv stages at .staging\F\homes\john\report.csv.<uuid>.tmp. When the data root and the user's home directory sit on the same volume — the default single-drive install — the commit is a true atomic rename and downstream consumers get the all-or-nothing guarantee for free.

If an operator deliberately puts home directories on a different volume from the data root, the server doesn't fail the upload. It tries the atomic move, catches the cross-volume exception, and falls back to a plain move — leaving a warning in the log so the split is visible:

WARN  move: ATOMIC_MOVE failed (cross-volume?), falling back: ...

That warning is the signal. It means the file still lands correctly, but there is now a brief window where a fast-polling downstream job could catch a partial write. If you see it, the fix is to co-locate: keep the .staging/ data root on the same volume as the home directories it serves. (This is also why the atomic rename operations in the local storage backend deliberately don't carry a fallback — a rename that must be atomic should fail loudly rather than quietly copy across volumes.)

The takeaway

ATOMIC_MOVE isn't a performance tweak — it's a correctness contract with whatever reads your files next. It promises that a file either isn't there or is complete, with nothing in between. That promise is what lets a downstream pipeline treat “file exists” as “file is ready.” Keep your staging and your targets on one filesystem and you get it for nothing. Split them across volumes and you trade it for a copy-and-delete window — which is fine for a human dropping files by hand, and a data-integrity bug waiting to happen for an automated consumer.

See atomic staging in action

Free evaluation key. Install on Windows or Linux, upload a large file, and watch it appear in one atomic step — never half-written.

Request Evaluation Key More articles ›