Filemail UDP Transfer Acceleration
On the internet today there are two main protocols that are used: TCP and UDP. TCP is the workhorse of the Internet. UDP transfer acceleration is the thoroughbred, made for speed.
What Is TCP?
This protocol has reliability mechanisms built into it, meaning that the sender will automatically re-transmit a TCP package to the receiver if it got somehow lost on the way.
TCP: The Backbone Of The Internet
Firewalls, Routers, and Switches are heavily optimized for TCP traffic, which makes TCP the natural choice for most tasks done on the internet today. Web browsers, Email clients, Facebook, Instagram, Snapchat, Stock tickers, Spotify, Slack, and FTP Clients are some of the applications which typically use TCP. If you are using a file transfer solution via a browser, then more than likely it is using TCP.
Ensuring Reliability: The Role Of Ack Messages In TCP
When a recipient receives a TCP package, an ACK (Acknowledge) message is sent back, to confirm to the sender that the package has been correctly received. This relatively simple mechanism makes the TCP protocol very reliable and easy-to-use for developers.
Sending ACK messages to the sender can seriously limit performance (since the sender must wait for the ACK of package-1 before sending package-2). To alleviate some of the issues plauging TCP, many improvements have been made to the protocol through the years.
Scalable Window Size
Enables the sender to send multiple packages and wait for multiple ACK’s at the same time. The data sent but not yet ACK-ed is often referred to as Data In Flight. The sender has a limit on how much data it can have in flight, meaning that the sender does not send data to the receiver faster than the receiver can handle.
Bandwidth throttling
This essentially enables a 10 Mbit client to communicate with a 1000 Mbit server without being overwhelmed with traffic from the server. In TCP it is based on the idea that there should only be a certain number of data in flight at any given time.
Selective ACKs
Enable the recipient to receive ACK packages in any order. If package-1 got lost then it does not hinder package-2 to be sent and ACK-ed before resending package-1.
The Drawbacks of TCP
When sending TCP traffic over a large geographical distance the performance drops significantly. As traffic (and ACKs) must travel through more routers and network infrastructure to get to the recipient it takes more time.
The time it takes is referred to as latency. If you are browsing a website hosted in the same country the latency is sometimes as low as 10 milliseconds. When browsing a website hosted on another continent you’ll immediately see that the pages take longer to load due to increased latency.
The latency problem of TCP becomes extremely apparent when transferring files across continents. Both the sender and the recipient might be on amazing 1000/1000 megabit connections but get only 10 megabit effective network bandwidth when transferring files due to latency that prevents them from communicating effectively when using TCP.
What is UDP?
UDP is the sibling of TCP. The main difference is that a UDP package is never ACK-ed. The sender has no idea if the UDP package made it to the recipient so it is not a reliable protocol. But it is blazing fast since there is no ACK-ing.
UDP: Prioritising Speed Over Reliability
UDP is great at video calls and online games. In these cases, it does not matter if a few UDP packages are dropped. The UDP packages are continuously sent so you won’t notice it in the video stream. Same with online games you’ll get a new UDP package with the latest updates within a few milliseconds. Losing a few UDP packages now and then isn’t a problem. It’s more important to transfer data quickly with the lowest latency possible.
Making UDP Reliable For File Transfers
UDP initially might seem like a horrible choice for file transfer. The total lack of reliability would mean corrupt files all over the place because of dropped UDP packages. The lack of bandwidth throttling would lead to clients being flooded with traffic and essentially going offline. For UDP to be useful for transferring files it needs to be reliable in the same way TCP is. For a start, the dropped packages need to be resent and bandwidth throttling must be possible. Security needs to be addressed as well.
Filemail UDP Transfer Acceleration Protocol
The Filemail UDP Acceleration Protocol (UAP) is a custom-made file transfer protocol built on top of UDP that offers blazing-fast transfer speeds even in high latency environments. It easily outperforms all TCP-based protocols such as HTTP and FTP – especially when sending files across large geographical distances where the latency creeps above 50 ms.
Up To 200 Times Faster
Benchmarks show that Filemail UDP transfer acceleration is sometimes up to 200 times faster than FTP, HTTP, and other transfer methods based on TCP.
Filemail UAP is built from the bottom up with one goal in mind: transferring files extremely fast from A to B in the most secure and reliable way possible. Below are some of the characteristics of this revolutionizing protocol.
Effective transfer bandwidth when transferring files from Europe to Australia across 21 network hops. Both ends are connected on a 1000/1000 Mbit connection.
Unofficial comparisons to UDP-based transfer solutions by IBM Aspera and Signiant show that Filemail is faster in pretty much all cases. Some of the performance differences might be due to Filemail utilizing hardware acceleration as well – by specifically using the AES-NI instruction set. Also, the protocols of IBM Aspera and Signiant use a separate TCP channel (SSH) to do bandwidth throttling, ACK’ing, etc. Filemail UAP does not rely on the slower TCP protocol – it uses UDP for transferring data, ACK’ing, bandwidth throttling, and encryption.
Security
All UDP data traffic is protected by AES Galois/Counter Mode (AES-GCM) encryption. This is the gold standard within encryption as of today and it ensures that the data being transferred can’t be intercepted by a third party. Each transfer session has a separate key, the key is exchanged between the client and server using Rivest-Shamir-Adleman (RSA) encryption.
Hardware Accelerated Encryption
Filemail UAP utilizes the AES-NI instruction set introduced by AMD and Intel a few years back. This instruction set enables Filemail UAP to use dedicated hardware components for the encryption and decryption of data. This makes a huge difference when it comes to transferring speeds. AES-NI is supported on operating systems such as Microsoft Windows, macOS, Linux, iOS, and Android. Hardware manufacturers such as Intel, AMD, ARM, VIA, Atmel, Samsung, Qualcomm, NXP, and Broadcom support it as well.
Software & Hardware Requirements
Filemail UAP is written in low-level C++ and can run on virtually all devices and operating systems. Binary releases are currently available for Windows, macOS, and Linux. Filemail Desktop is our flagship transfer tool that utilizes UAP. Console applications are also available. The Filemail UAP has been heavily optimized and has an extremely low memory footprint of only a few megabytes, while still maximizing bandwidth throughput. It also typically uses less than 5% CPU when sending files at max speed.
Bi-Directional Transfer Acceleration
Filemail UAP is used both when uploading and downloading files with Filemail Desktop. This means that you AND your customers and business partners can take advantage of this technology.
Getting Started With Filemail UDP Transfer Acceleration
Filemail UDP Transfer Acceleration is integrated into our Filemail Desktop application and is enabled automatically when sending and downloading files. UDP Transfer Acceleration is available on our Business and Enterprise plans.
