User Manual — UFTP Docs (2024)

The UFTP standalone Client is a Java-based commandline client for UFTP. It allows to listremote directories, copy files (with many options such as wildcards or multiple threads),and more. It can be used with either a UFTP Authentication Server ora UNICORE server to authenticate and initiate UFTP transfers.



  • Java 11 or later (OpenJDK preferred)

  • Access to a UFTP authentication service (either an Auth Server or aUNICORE/Xserver) and to the corresponding UFTPD Server.

To use the client, you need to know the address of the UFTP authentication service.You also need to have the valid credentials for the UFTP authentication.

Installation and Configuration

The UFTP client distribution packages are available at

If using the zip or tar archive, unpack it in a location of your choice. Add the bindirectory to your path. Alternatively, you canlink or copy the bin/uft script to a directory that is already onyour path, in this case edit the script and setup the required directories.

If you use the rpm or deb package, install it using the packagemanager of your Linux distribution.


By default, the UFTP client will use the current username ($USER) with SSH keyauthentication to authenticate to the Auth Server.

You can set a different default username via the UFTP_USER environment variable. This isuseful if your remote username is not the same as your local username.

The UFTP client supports various means of authentication. Dependingon the server you want to access, you can choose from

  • user name with SSH key (default)

  • user name with password

  • oidc-agent

  • manual specification of the HTTP Authorization header value

To explicitely specify the remote username, use the -u <username> option, e.g.

$ uftp ls -u username https://localhost:9000/rest/auth/TEST:/home/demo/

The credentials can be given in multiple ways.

  • On the command line -u username:password

    $ uftp ls -u username:password ...
  • You can tell the uftp client to query the password interactively by giving the -P option,e.g.

    $ uftp ls -u username -P ...
  • If no password is given, the client will attempt to use an SSH key for authentication, this hasto be configured on the authentication server accordingly. If you have multiple keys, use the-i option to select one. Otherwise, the client will check ~/.uftp/ and ~/.ssh/ foruseable keys. The SSH agent is supported, too.

  • The very useful oidc-agent tool is also directlysupported via -O <account_name>. In this case no username is required.

    $ uftp ls -O hbp ...
  • You can directly specify a value for the HTTP Authorization header withthe -A option. This allows to use an OIDC bearer token for authorization, e.g.-A "Bearer <oidc_token>. In this case no username is required.

    $ uftp ls -A "Bearer <oidc_token>" ...
  • If you explicitely DON’T want to send any authentication info, use -u anonymous.


In the following usage examples, the authentication service is locatedat localhost:9000/rest/auth/ and the user name is username.Replace these values by the correct ones for your installation.

Listing a directory: the ls command

$ uftp ls https://localhost:9000/rest/auth/TEST:/home/demo/

will list the /home/demo directory.

Copying data: the cp command

The cp command is used to copy local data to a remote server or viceversa. Remote locations are indicated by the https:// prefix, and youneed your user name, and the URL of the authentication server.

It has a number of features, which will be shown in the following.

Basic usage

  • Downloading a single file:

    $ uftp cp https://localhost:9000/rest/auth/TEST:/home/demo/ .

    will download the /home/demo/ file to the current directory

  • Download files using wildcards:

    $ uftp cp https://localhost:9000/rest/auth/TEST:/home/demo/data/* .

    will download all files in the /home/demo/test directory to the current directory

Similar commands work for upload.

  • Uploading files using wildcards:

    $ uftp cp "/data/*" https://localhost:9000/rest/auth/TEST:/home/demo/data/ .

The wildcards should be escaped to avoid the shell doing the expansion, which will also work,but generally be slower.

The recurse flag, -r, tells uftp to also copy subdirectories.

Piping data

The cp command can read/write from the console streams, which is great for integrating uftpinto Unix pipes. The - is used as a special file name to indicate that data should beread/written using the console.

Transferring with tar and zip

For example to tar the contents of a directory and upload the tar file using uftp

$ tar cz dir/* | uftp cp - https://localhost:9000/rest/auth/TEST:/archive.tgz

The UFTPD server can also unpack tar and zip streams, this is very useful toefficiently transfer many small files. To enable this, add the -a option, and DO NOT compressthe tar stream.

$ tar c dir/* | uftp cp -a - https://localhost:9000/rest/auth/TEST:/target_location/

or, using zip

$ zip -r - dir/* | uftp cp -a - https://localhost:9000/rest/auth/TEST:/target_location/


Zip will compress data, so might be slower or faster than tar, depending on network bandwithand processing speed.

Similarly, - can be used to write data to standard output.As an example, consider this

$ uftp cp https://localhost:9000/rest/auth/TEST:/archive.tgz - | tar tz

Or use uftp to cat a remote file

$ uftp cp https://localhost:9000/rest/auth/TEST:/foo.txt -

Using multiple FTP connections

When transferring large files (or many files) over a high-performance network, performancecan be vastly improved by using multiple FTP connections. (NOTE this is different fromthe multiple TCP streams as set via the -n option).

Use the -t option to set the desired number of streams. Note that the server may havea limit on the allowed number of concurrent connections, if in doubt, ask your serveradministrator.

$ uftp cp -t 2 https://localhost:9000/rest/auth/TEST:/home/demo/* .

To split up files larger than a certain size and transfer them in chunks,you need to specify a “split size” using the -T option. For example,to split up files larger than 1MB

$ uftp cp -t 2 -T 1M https://localhost:9000/rest/auth/TEST:/home/demo/* .


NOTE: uftp client versions before 1.7.0 have an automatic split size set to 512MB,with 1.7.0 and later you MUST use “-T …” to enable file splitting.

Byte ranges

To copy just part of a file, a byte range can be given with the -Boption. Counting starts at zero. For example to download only thefirst 1024 bytes of file (i.e. the byte range 0 - 1023), you would do

$ uftp cp -B 0-1023 https://localhost:9000/rest/auth/TEST:/home/demo/ .

As an additional feature, you can use the additional -p flag, whichwill write also only the given range. For example

$ uftp cp -B 1024-2047-p https://localhost:9000/rest/auth/TEST:/home/demo/ .

will write bytes 1024-2047 of the remote file to the local file,starting at offset 1024. The local file will have length 2048.

The same thing works for remote files!

Number of bytes to transfer

A simplified version of the byte range specification is to just givethe amount of data to transfer (implying the start of the range is at byte 0)

This is handy for quick performance tests:

$ uftp cp -B 10G https://localhost:9000/rest/auth/TEST:/dev/zero /dev/null

Encryption and compression

The cp command supports the -E and -C options, which enable dataencryption and compression (during transfer) respectively.

  • Data encryption uses a symmetric algorithm, which nonethelessdrastically lowers the performance.

  • Data compression uses the gzip algorithm.

Compression and encryption can be combined.

Resuming a failed transfer

If a copy command was terminated prematurely, it can be resumed usingthe -R option. If the -R option is present, the UFTP client willcheck if the target file exists, and will append only the missingdata.

So if your inital copy operation

$ uftp cp -u username https://localhost:9000/rest/auth/TEST:/home/demo/ .

did not finish correctly, you can resume it with

$ uftp cp -R https://localhost:9000/rest/auth/TEST:/home/demo/ .

Performance testing

For performance testing, you can use /dev/zero and /dev/null as data source / sink.

For example to transfer 10 gigabytes of zeros from the remote server:

$ uftp cp -B 0-10G https://localhost:9000/rest/auth/TEST:/dev/zero /dev/null

This can also be combined with the multi-connection option -t. To use two connections eachtransferring 5 gigabytes

$ uftp cp -B 0-10G -t 2 https://localhost:9000/rest/auth/TEST:/dev/zero /dev/null

Computing checksums for remote files

To compute a checksum for a remote file, use the checksum command:

$ uftp checksum https://localhost:9000/rest/auth/TEST:/data/*.dat

A number of different hashing algorithms are available, which can be selectedusing the -a option (MD5, SHA-1, SHA-256, SHA-256). For example

$ uftp checksum -a SHA-256 https://localhost:9000/rest/auth/TEST:/data/*.dat

Synchronizing a file: the sync command

Note that sync only supports single files, i.e. no directories or wildcards!The syntax is

$ uftp sync <master> <slave>

For example, to synchronize a local file with a remote master file:

$ uftp sync https://localhost:9000/rest/auth/TEST:/master.file local.file

To synchronize a remote file with a local master file:

$ uftp sync master.file https://localhost:9000/rest/auth/TEST:/remote.file

Data sharing

Data sharing enables users to create access to their datasets forother users via UFTP, even if those users do not have Unix-levelaccess to the data.

Data sharing works as follows:

  • when you share a file (or directory), the Auth Server will store informationabout the path, the owner and the Unix user ID used to access the file in a database

  • the targetted user can now access this file via the Auth Server, and the Authserver will use the owner’s Unix user ID to access the file.

By default, files will be shared for anonymous access. This willallow anyone who knows the sharing link to access the file usingnormal HTTP tools like wget or curl.

Shares can also be limited to certain users.

Depending on the type of share, access to the files is possible withthe UFTP protocol or plain HTTPs.

Shares can be deleted by their owner, i.e. the user who created them.


Not all UFTP installations support data sharing. You can check if a server hasthe sharing feature enabled by running uftp info --server ...

Server URL

If not given via the --server argument, the URL of the Auth Server willbe taken from the environment variable UFTP_SHARE_URL

$ export UFTP_SHARE_URL=https://localhost:9000/rest/share/TEST$ uftp share --list

Anonymous (https) access

For anonymous access via HTTP you need to use the correct URL. If you create (or list) shares,the UFTP client will show the required links. You can download the file e.g. using wget.

In case the share is a directory, wget will return a directory listing.

Server-to-server copy

REQUIRES UFTPD 3.2.0 or later (at least on one side)

The rcp command is used to instruct a remote UFTPD serverto copy data from another UFTPD server. The client authenticates to both sides.

Basic usage

The basic syntax is similar to the normal uftp cp command:

$ uftp rcp <options> <source1> ... <sourceN> <target>

If the same means of authentication can be used for both source and target sides,both source and target are normal UFTP URLs. If source and target require differentauthentication, you need to use the uftp auth command first to authenticate tothe one side (usually the source)

$ uftp auth <options> <source_URL>

and give the resulting host:port and one-time password to the rcp command viacommandline options:

$ uftp rcp --server <host:port> --one-time-password <pwd> <source_file> <target>

Other supported features

The rcp command supports byte ranges via the -B option.

Reversing the copy direction

By default, the target side is instructed to download data from the source side.This can be reversed, if necessary, for example if only the source supportsserver-to-server copy. To do this, an environment variable can be set:

$ export UFTP_RCP_USE_SEND_FILE=true$ uftp rcp ...

This will result in the source side uploading the file to the target side, andthe --server and --one-time-password options will refer to the target side.

Known issues

There is no way to monitor or abort a running server-to-server transferfrom the client.

Wildcards are not supported.

Using a proxy server (EXPERIMENTAL)

The uftp client has support for some types of FTP and HTTPs proxies.

This is configured via enviroment settings. I.e. in your shell you can define

  • FTP proxy

    export UFTP_PROXY=proxy.yourorg.eduexport UFTP_PROXY_PORT=21
  • HTTP proxy

    export UFTP_HTTP_PROXY=proxy.yourorg.eduexport UFTP_HTTP_PROXY_PORT=80

FTP proxying was tested with the DeleGate/9.9.13 and frox proxiesand requires UFTPD server version 2.8.1 or later to work.

If this does not work for you, or if you require support for adifferent type of proxy, please contact us via a support ticket or via email.


🤷‍♂️ How can I get more detailed logging?

👉 In the client's conf directory you'll find a logging.propertiesfile that allows you to increase the log levels.

🤷‍♂️ I get “Invalid server response 500” and “Exception…. Authenticationfailure”

👉 Probably you gave a wrong username or password. Contact your siteadministrator if in doubt! If using a password, make sure you give the -P flag.

🤷‍♂️ I get “Invalid server response 405 Unable to connect to server for listing”

👉 Check the remote URL that you use. Maybe you have a typo in the/rest/auth/<servername> part.

User Manual — UFTP Docs (2024)
Top Articles
Latest Posts
Article information

Author: Jonah Leffler

Last Updated:

Views: 5675

Rating: 4.4 / 5 (65 voted)

Reviews: 80% of readers found this page helpful

Author information

Name: Jonah Leffler

Birthday: 1997-10-27

Address: 8987 Kieth Ports, Luettgenland, CT 54657-9808

Phone: +2611128251586

Job: Mining Supervisor

Hobby: Worldbuilding, Electronics, Amateur radio, Skiing, Cycling, Jogging, Taxidermy

Introduction: My name is Jonah Leffler, I am a determined, faithful, outstanding, inexpensive, cheerful, determined, smiling person who loves writing and wants to share my knowledge and understanding with you.