Almost every SFTP server in the wild treats “the filesystem” as the place files go. You configure a home directory per user, the SSH daemon translates SFTP read/write commands into open(), read(), write() on real files, and the operating system handles the rest. It's the default model, it's been the default model for thirty years, and it works for most installations.
Some installations, it really doesn't. This post is about the cases where storing transferred files in a database instead of on a filesystem is the right call — what it buys you operationally, what it costs you in throughput, and what the design actually looks like when you build it. The short version: if your file transfer system has to play nicely with backup, replication, geographic redundancy, or transactional consumers downstream, a database storage backend earns its keep. If you're moving a million files a day between a vendor and a partner, just put them on a disk.
What "files on the filesystem" actually means in production
It looks simple from the SFTP server's perspective. From operations, it doesn't.
Your SFTP home directory lives on a volume. That volume needs:
- Backups. Snapshot, rsync, or a real backup agent. Whatever it is, it has to know how to walk the directory tree, capture file state, handle in-progress writes, and not destroy the volume during the run.
- Replication. If you have a hot standby SFTP server, you need DRBD, an NFS export, a SAN, an object store, or some other story for getting the same bytes on two machines.
- Quotas and orphan handling. The filesystem will happily fill up. Half-written
.tmpfiles from killed transfers don't clean themselves up. Per-user disk quotas are an OS feature, not an SFTP feature, and the two don't always agree. - Geographic redundancy. “Files in both data centres” means some form of object replication. Filesystem-level replication tools are not gentle.
- Downstream consumers. If a workflow engine, a virus scanner, or a data pipeline needs to know “a new file arrived,” you're either polling the directory or watching
inotifyevents. Polling has latency.inotifyhas its own footguns (the watch limit, missed events under load, the queue overflow event).
None of these are deal-breakers. They're tractable problems with mature tools. But every one of them is a thing you have to design, monitor, and pay for separately. The filesystem is doing one job — storing bytes — and you're bolting four or five other systems on top to make those bytes safely usable.
What you get when you put files in a database instead
If the file content goes into a database table — chunked into reasonable rows, transactionally committed on close — then the bytes inherit everything the database already does for everything else in the schema. That's a meaningful list:
- One backup. Your DBA already takes a backup of the database. The files are part of it. There is no second backup story.
- One replica. SQL Server Always On, PostgreSQL streaming replication, MySQL InnoDB Cluster — they all replicate every byte you write. The files come along for free.
- One transaction boundary. A file row appears in the table, atomically, at the moment the SFTP client closes the upload cleanly. No
.tmpfile races, no rename-on-close anxiety, no half-written-file edge cases for downstream consumers to worry about. - One query interface. “Has user X uploaded anything to /inbound/ in the last 10 minutes?” is a
SELECTstatement, not a directory scan. “What's the size distribution of files this customer has sent us this month?” is aGROUP BY, not a 20-line script. - One change feed. Triggers, CDC streams, or transactional outbox tables: any pattern your team already uses to react to inserts works for incoming files. The polling-vs-inotify dilemma vanishes.
- One permission model. Row-level security, schema permissions, and role-based access can apply to files just like rows. The SFTP server still does its own authentication; the DB just enforces the storage-layer policies you'd otherwise build by hand.
None of that is magic — it's just “files are data, treat them like data.” But the operational simplification is real. If your existing operations team is already great at running a database (and not great at running NFS or DRBD), you get to do file transfers in the half of your stack you already know.
What it costs
Three things, all manageable, all real.
Throughput, sometimes
A modern relational engine can absorb 200-300 MB/s of inserts on commodity hardware. That's faster than most SFTP clients can transmit. So for ordinary 1-100 MB transfers, the database backend keeps up with the wire. Where it starts to hurt is sustained multi-gigabyte transfers, because the database writes the bytes twice — once to the data file, once to the transaction log — and SQL engines aren't tuned the same way a filesystem is tuned for raw streaming throughput.
The mitigations: chunk the file into a row per N kilobytes (so the log records are bounded), use a relational engine with built-in page compression on the chunks table (cheap for text and XML payloads), and configure the database to use SIMPLE recovery mode if you don't need point-in-time recovery of the file content. With those three tweaks, a 10 GB upload to a database-backed SFTP server lands in roughly the same wall-clock time as the same upload to a local disk.
Operational footprint
The database becomes the system of record for file content, not just metadata. That means the database has to be sized for actual file volume, not just relational state. If your SFTP server moves a TB a month, the database is a TB a month bigger. This is a thing to plan for, not a thing to be surprised by.
Mental model shift for the team
People expect to be able to ls on the SFTP home directory and see real files. With a database backend, the “files” are rows. You still see them through the SFTP protocol, the Web Portal, WebDAV, FTP, whatever your clients use. But there is no ls /home/jsmith/inbound/ on the server's shell that shows them. The operations team has to be willing to query the database for “what's there.”
What we built
Stylus SFTP Server ships with a pluggable storage backend. The default is the local filesystem — the model everyone expects. The alternative is the JDBC backend, which writes file content into the relational database of your choice (H2, SQL Server, MySQL, or PostgreSQL).
The backend exposes a single interface, StorageBackend, with the operations you'd expect: openForRead, openForWrite, listFiles, exists, delete, rename. SFTP, FTP, FTPS, WebDAV, and the browser Portal all route through one of these backends. One implementation covers every protocol.
The JDBC implementation uses three tables:
sftp_file_storage -- one row per file: path, name, owner, size, mtime
sftp_chunk_data -- the actual bytes, chunked, ref_counted for dedup
sftp_file_chunks -- which chunks make up which file (ordered)The split into chunks lets identical chunks be deduplicated across files (the ref_count on sftp_chunk_data tracks how many files reference each chunk). The split also bounds the transaction log: instead of one transaction holding the whole file, each chunk write commits its own row, so the log can be truncated more aggressively. A long-running multi-gigabyte upload doesn't pin the entire log.
Configuration is one block in filesystem-config.xml:
<storage-backend>database</storage-backend>
<storage-database>
<url>jdbc:sqlserver://localhost:1433;databaseName=sftpfiles</url>
<username>sftp</username>
<password>...</password>
<max-pool-size>10</max-pool-size>
</storage-database>That's it. Tables are auto-created on first boot. Every SFTP client, every web browser hitting the Portal, every WebDAV mount — they all transparently land in the database.
When you should not pick it
Be honest about your workload before turning it on.
- If your SFTP server moves millions of small files per day at high sustained rates, the filesystem will outperform any database on raw throughput.
- If your downstream pipeline expects files on the local disk (it tails directories, it shells out to convert, it has hardcoded paths), the database backend means a fetch step you didn't have before.
- If you have no DBA and don't want one, running a database for files is overhead you don't need.
- If the files are massive videos, raw camera footage, or backup tarballs (think 100+ GB single files), neither model is great, but the filesystem is less bad.
The sweet spot
Database storage is the right call when files are transactional. When the file is the message: an EDI document, an invoice, a daily report, a CSV that drives a workflow. When you'd want to roll back the file alongside the database row that represents it. When the file's audit trail and the file's bytes need to live in the same backup window.
It's also the right call when you have a strong operational pattern around “the database is the source of truth.” If your team already runs a beautifully-tuned SQL Server Always On cluster, putting your inbound file exchange into the same cluster gives you DR for free. If you're already replicating Postgres to a second region for failover, your file transfers replicate too, no extra story.
And it's the right call when the alternative is filesystem-level features you don't actually want to operate: NFS exports, distributed filesystems, object-storage abstractions that introduce their own failure modes. Sometimes the simplest infrastructure decision is “use one fewer kind of system.”
The take-home
Default to the filesystem. The filesystem is fast, well-understood, and has a thirty-year tooling head start. If your operations team is happy with disk and a backup tape, don't change anything.
Reach for database storage when the operational story around files-as-files is the part that's hurting. When backup, replication, transactional consistency, or downstream querying are the actual pain. When you'd rather solve those problems once, in the part of your stack that already solves them for everything else.
The point of giving an SFTP server a pluggable storage backend isn't “database is better.” It's “put the bytes wherever fits your operations.”