Data Management

Data management on CARC systems involves managing, sharing, and transferring files. All research projects should have a data management plan to maximize research productivity on CARC systems.

Currently, CARC systems do not support the use or storage of sensitive data. If your research work handles sensitive data, including but not limited to HIPAA-, FERPA-, or CUI-regulated data, see our Secure Computing user guide or contact us at before using our systems.

Storage file systems

CARC provides multiple storage file systems for various purposes. For more information on the available file systems for both the Discovery and Endeavour computing clusters, see the Storage File Systems user guide.

Managing files

File management involves organizing, sharing, backing up, and archiving files. On CARC systems, this can be achieved using command-line tools or graphical tools.

For more information on using command-line tools, see the Managing Files Using the Command Line user guide.

For graphical tool options, see the Transferring Files Using a Graphical User Interface user guide. The same graphical tools used to transfer files can also be used to manage files on CARC systems, but command-line tools may be better suited for more advanced tasks.

Transferring files

Secure and efficient data transfer to and from CARC systems can be achieved with a variety of useful tools, including command-line tools, graphical tools, and the Globus service. The choice of which tool to use depends on a few factors such as whether the storage location is a personal computer or an external site (e.g., cloud storage) and the transfer size. The choice also depends heavily on the requirement of data sensitivity, as well as the familiarity of the user.

For more information, see the File Transfer Overview user guide.

User Guides

  • Storage File Systems
  • Managing Files Using the Command Line
  • File Input/Output
  • File Transfer Overview
  • Transferring Files Using the Command Line
  • Transferring Files Using a Graphical User Interface
  • Transferring Files Using Globus
  • Transferring Files Using rclone
  • Related Links

    Some links
    Some links
    Some links