You are here

Efficient file transfers with gcp

gcp is a grid enabled version of the scp copy command.

The gcp command is meant to emulate as closely as possible the scp command while using globus-url-copy for file transfers for performance reasons. gcp allows recursive transfers of directories and wildcards.

Since gcp is a grid tool using globus-url-copy it needs to use grid certificates for authentication. This is accomplished by first checking whether a valid grid proxy certificate exists. If that is not the case, gcp will attempt to run grid-proxy-init. Failing that gcp will contact the myproxy.westgrid.ca proxy server in order to obtain a proxy certificate. If that fails as well, gcp issues an error, explaining how to upload certificates to the myproxy server. The whole authentication process is setup such that it is not required that the user has knowledge of grid authentication.

In most cases the syntax for gcp is the same as for scp.

Some useful options:

  • -r : recursive transfer to recursively copy the contents of all directories.  By default gcp will not copy directories.

    -v : verbose output; give additional information on what it is doing.

    -i : interactive; print a list of source files and destination files and (in the case of -r) destination directories that it will create on the destination host. It then will ask whether you want to proceed at which point you can answer 'n' and no files will be transfered.

Detailed information about gcp is available through its man page: type man gcp.

Examples

General examples

gcp file1 orcinus:document

will transfer the file file1 in the current directory to orcinus under the name document in your orcinus home directory.

gcp -r dir1 bugaboo:scratch

transfer the directory dir1 and all its files, subdirectories to bugaboo into the directory scratch/dir1.

gcp -r docs* programs/*.exe progdir bugaboo:new

copy recursively all files and directories that start with docs, all files that match programs/*.exe, and the directory progdir into the directory new in your home directory on bugaboo.

Example of how to move your data to Silo

This command will copy your whole directory to your home directory on Silo:

gcp -r dirname silo.westgrid.ca:.

This command will copy your data from silo to the system you are logged into. The directory dirname on SIlo is renamed to dirname1 on the current system in the current directory:

gcp -r silo.westgrid.ca:dirname dirname1