Transferring Research Data
Secure and efficient data transfer to and from CARC systems can be achieved with a variety of useful tools, the choice of which depends on whether the storage location is a personal computer or an external site (e.g., cloud storage). The choice also depends heavily on the requirement of data sensitivity as well as the familiarity of the user. There are three methods of data transfer that we recommend for use with CARC systems: command-line tools, graphical tools, and the Globus service.
Due to security risks, please be mindful of the type of information being transferred. Where possible, omit all information that may be considered confidential. For examples of confidential information that requires additional consideration, visit https://sites.usc.edu/trojansecure/information-data-security/.
Transfer speeds are determined by a number of factors, such as the network speed at your location, router and firewall settings, etc. If you experience slower than expected transfers, try to troubleshoot these issues first. If you continue to experience slow transfer speeds, submit a help ticket.
0.0.1 Use cases
Below are four example scenarios that provide some insight into which data transfer method you might use for a given situation:
| Start Location | End Location | Example Scenario | Transfer Method |
|---|---|---|---|
| Personal Computer | CARC file system for small-to-medium transfers | When transferring files from a personal computer to your CARC project folder that takes a moderate amount of time. | GUI, OnDemand, CLI |
| Personal Computer | CARC file system for large or secure transfers | When transferring files from a personal computer to your CARC project directory that takes a large amount of time or needs to be encrypted. | Globus |
| Amazon Web Services (AWS) | Any CARC file system | When transferring files from an AWS server to your CARC project directory. | CLI |
| Other HPC centers | Any CARC file system | When transferring files from another university or research institution to your CARC project directory. | Globus, CLI |
0.0.2 Command-line tools
Information on how to use command-line tools and which transfer scenarios they are best suited for.
0.0.3 Graphical tools
Instructions on how to use applications such as Cyberduck, FileZilla, and WinSCP, which provide a graphical user interface (GUI) to transfer data.
0.0.4 Globus service
Instructions on how to use Globus for large data transfers and the best transfer speeds.
0.0.5 Rclone
Information on how to use rclone to transfer data to and from cloud storage.