I want to know if too much reliance on pure FTP for file transfer in a 24x7 production environment is risky, problematic (in performance and reliability) and not scalable.
Data is transferred hourly and per minute. Sub-minute data transfers may even be necessary in the future. We seem to be FTPing too much, and it is growing.
Would database-database interoperability (e.g., SQL Server to Oracle via Oracle Gateway), text file gen via SQL SELECTs generally be more robust and scalable, considering all the functional/processing power available in enterprise database environments?
Would database-database interoperability be less hit on net traffic/load than FTP?
FTP is a wonderful way to move data between systems. It is simple and can usually do what you need to do.
It does have its issues, though. Performance, security, integrity, availability and scalability are some of them. Concurrent data transfers can put a significant strain on FTP servers. FTP is not inherently secure. You open yourself up when you allow this type of access. FTP Servers cannot balance many connections across a path of redundant servers. They also typically do not scale well.
You may have to upgrade your hardware if demands on the systems become huge. If you can use your database to transfer data, then I would certainly lean towards this method, though you may open yourself up to other problems. An incorrectly scripted SQL statement can kill the I/O you have on your system. You also leave yourself open to database corruption and/or access violations on the data side. Though neither is appealing, I would rather someone break into my OS than my database.
You may also want to look into other FTP-type servers that may give you some of what you need. Here's a URL that does comparisons of various types of FTP Servers that might be helpful to you:
Dig deeper on Linux network administration
Related Q&A from Kenneth Milberg
Unix-to-Linux migration expert Ken Milberg describes how virtualization, support, clustering and more fit into the migration of an IT infrastructure ...continue reading
A reader new to Linux wonders about which distribution is recommended for installing Nagios and what Nahant and Tikanga mean.continue reading
Documentation for Red Hat Enterprise Linux 5 covering checking system performance, tuning, kernel configuration and extending the file system exists ...continue reading
Have a question for an expert?
Please add a title for your question
Get answers from a TechTarget expert on whatever's puzzling you.