MFT Orchestration: Connecting What You Already Have

Here's a pattern I've seen play out at least a dozen times: a team has cron jobs pulling files from a vendor SFTP, a Windows Scheduled Task pushing reports to a client server, a couple of shell scripts someone wrote three years ago that move nightly batches between environments, and maybe a Robocopy task that syncs a shared folder to somewhere nobody remembers.

Each one works. Individually. The problem is nobody can see all of them at once, nobody knows what happens when one fails, and nobody can answer the question "is our data moving correctly right now?" without logging into four different machines and checking three different log formats.

The instinct is to rip it all out and start fresh with a single MFT platform. That's the IBM Sterling sales pitch: replace everything with one governed system. It works, if you have eight months and a six-figure budget. Most teams don't.

The Orchestration Problem

File transfer sprawl isn't a tool problem. It's an orchestration problem. The transfers themselves are fine. Cron runs scp. SFTP works. rsync does its job. What's missing is the layer between all of them that provides:

• Visibility — a single view of every transfer, regardless of how it runs
• Monitoring — alerting when something fails or stalls, without checking log files manually
• Retry logic — automatic reconnection and backoff when a network blip kills a transfer
• Audit trail — who transferred what, when, and whether it succeeded
• Scheduling — coordinated timing so transfer B doesn't start until transfer A finishes

You can bolt some of this together with Nagios checks and a shared spreadsheet. I've seen teams try. The spreadsheet lasts about two weeks before it's stale and the Nagios alerts get ignored because they fire too often.

What Orchestration Actually Looks Like

An orchestration layer sits above your existing transfers. It doesn't replace them — it wraps them. Here's what that means in practice:

1. Register, don't replace

Your cron job that pulls vendor files at 2 AM? Keep it. But register it with the orchestration layer so the system knows it exists, when it's supposed to run, and what success looks like. Now when someone asks "did the vendor file arrive?", you have an answer that doesn't involve SSHing into a box.

2. Monitor the outcome, not the process

Most teams monitor whether a script ran, not whether it did what it was supposed to. An orchestration layer checks the outcome: did the file land? Is it the right size? Did it arrive within the expected window? If not, alert. This is the difference between "cron ran successfully" and "the transfer actually worked."

3. Chain transfers into workflows

When transfer A finishes, start transfer B. If B fails, retry twice with backoff, then alert. If B succeeds, run the validation script. This is where most free tools fall apart — cron can't do dependencies, and Windows Task Scheduler can't chain across machines.

4. Centralise credentials

Instead of SSH keys scattered across six servers and passwords baked into batch files, the orchestration layer holds credentials in one place. Rotate a key once, every transfer that uses it gets the update. This alone is worth the price of admission for compliance audits.

Orchestration vs. Replacement

I'm not saying rip-and-replace is always wrong. If your infrastructure is a security nightmare with plaintext FTP and hardcoded credentials everywhere, you probably should start fresh. But most teams I've worked with are somewhere in the middle: their transfers work, they just can't see them, can't monitor them, and can't prove to an auditor that they work.

For those teams, orchestration is the right first step. Get visibility. Get monitoring. Get audit trails. Then, over time, migrate individual transfers into the platform as they come up for renewal or break.

What This Looks Like in MFTPlus

We built MFTPlus with this exact pattern in mind. You don't have to migrate everything on day one.

mftctl registers existing transfers as jobs. A job can be an SFTP pull, an HTTP download, a local file copy, or a shell command. The agent runs on your server, executes the job on schedule, reports success or failure back to the dashboard, and handles retries automatically.

The dashboard shows every job across every server in a single view. Green means it worked. Red means it didn't. You can drill into any job to see transfer history, file sizes, duration, and error logs.

Credentials live in the platform, not in scripts. When you rotate an SSH key, you update it once in MFTPlus and every job that uses that credential picks it up on the next run.

Making it simpler

We're also working on natural language job creation. Instead of filling in form fields for protocol, host, port, credentials, and schedule, you describe what you want in plain English: "Pull the daily sales CSV from vendor.example.com every weekday at 6 AM and put it in /data/incoming." The system builds the job configuration from that description.

It's early. But the goal is to cut the time between "I need a new transfer" and "it's running and monitored" from hours to minutes.

When Orchestration Isn't Enough

There are cases where wrapping existing transfers won't cut it:

• Plaintext FTP — there's no safe way to orchestrate unencrypted transfers. Replace these first.
• Hardcoded credentials in scripts — you can monitor the script, but the credential exposure remains until you migrate the job into the platform.
• Transfers with no error handling — if a script fails silently and nobody notices for three days, monitoring helps you notice faster, but the root cause is still a brittle script. Migrating it to a managed job with retry logic fixes the cause.

The point isn't that orchestration solves everything. It's that orchestration solves most of the problem immediately, and gives you a foundation to fix the rest over time instead of in a single high-risk migration.

Getting Started

If you're running file transfers across multiple servers with multiple tools and no central view, start here:

1. Inventory what you have. List every transfer: source, destination, schedule, protocol, credentials. If you can't do this in an afternoon, you have shadow transfers. We're building a discovery tool for exactly this situation.
2. Register them as jobs. Use mftctl to create a job for each transfer. The agent runs alongside your existing setup — nothing breaks if the registration goes wrong.
3. Add monitoring first, migration second. Get everything visible on the dashboard before you start migrating transfers into managed jobs. You'll learn a lot about what actually runs on your network just by watching.
4. Migrate the worst offenders. Start with the transfers that cause the most pain: the ones that fail silently, the ones with hardcoded credentials, the ones nobody wants to touch because the original author left three years ago.