ARCHER logo ARCHER banner

The ARCHER Service is now closed and has been superseded by ARCHER2.

  • ARCHER homepage
  • About ARCHER
    • About ARCHER
    • News & Events
    • Calendar
    • Blog Articles
    • Hardware
    • Software
    • Service Policies
    • Service Reports
    • Partners
    • People
    • Media Gallery
  • Get Access
    • Getting Access
    • TA Form and Notes
    • kAU Calculator
    • Cost of Access
  • User Support
    • User Support
    • Helpdesk
    • Frequently Asked Questions
    • ARCHER App
  • Documentation
    • User Guides & Documentation
    • Essential Skills
    • Quick Start Guide
    • ARCHER User Guide
    • ARCHER Best Practice Guide
    • Scientific Software Packages
    • UK Research Data Facility Guide
    • Knights Landing Guide
    • Data Management Guide
    • SAFE User Guide
    • ARCHER Troubleshooting Guide
    • ARCHER White Papers
    • Screencast Videos
  • Service Status
    • Detailed Service Status
    • Maintenance
  • Training
    • Upcoming Courses
    • Online Training
    • Driving Test
    • Course Registration
    • Course Descriptions
    • Virtual Tutorials and Webinars
    • Locations
    • Training personnel
    • Past Course Materials Repository
    • Feedback
  • Community
    • ARCHER Community
    • ARCHER Benchmarks
    • ARCHER KNL Performance Reports
    • Cray CoE for ARCHER
    • Embedded CSE
    • ARCHER Champions
    • ARCHER Scientific Consortia
    • HPC Scientific Advisory Committee
    • ARCHER for Early Career Researchers
  • Industry
    • Information for Industry
  • Outreach
    • Outreach (on EPCC Website)

You are here:

  • ARCHER

ARCHER User Guide

  • 1. Introduction
  • 2. Connecting and Transferring Data
  • 3. Resource Management
  • 4. Application Development Environment
  • 5. Running Jobs
  • 6. Using Python on ARCHER
  • 7. Linux Help
  • User Guides & Documentation
  • Essential Skills
  • Quick Start Guide
  • ARCHER User Guide
  • ARCHER Best Practice Guide
  • Scientific Software Packages
  • UK Research Data Facility Guide
  • Knights Landing Guide
  • Data Management Guide
  • SAFE User Guide
  • ARCHER Troubleshooting Guide
  • ARCHER White Papers
  • Screencast Videos

Contact Us

support@archer.ac.uk

Twitter Feed

Tweets by @ARCHER_HPC

ISO 9001 Certified

ISO 27001 Certified

2. Connecting and Data Transfer

  • 2.1 Interactive access
    • 2.1.1 Interactive access to Post Processing nodes
  • 2.2 Non-interactive SSH access
  • 2.3 Transferring data to and from ARCHER
    • 2.3.1 Performance considerations
  • 2.4 Making access more convenient using the SSH configuration file

On the ARCHER system interactive access can be achieved via SSH, either directly from a command line terminal or using an SSH client. In addition data can be transferred to and from the ARCHER system using scp from the command line or by using a file transfer client.

To access ARCHER, you need to use two credentials: your password and an SSH key pair protected by a passphrase. You can find more detailed instructions on how to set up your credentials to access ARCHER from Windows, macOS and Linux in the "Logging on to ARCHER" guide:

  • Logging on to ARCHER

In the rest of this section, we cover the basic connection and data transfer methods, along with presenting some performance considerations. Finally, we briefly introduce the SSH config file which can make access a bit more convenient.

Note:Whenever this section talks about connecting to or transferring data to/from ARCHER you will be required to use both your password and SSH key.

2.1 Interactive access

To log into ARCHER you should use the "login.archer.ac.uk" address:

ssh [userID]@login.archer.ac.uk

Windows users will need an SSH client. If you have a modern version of Windows 10 you can use the SSH client included with Powershell. We also find the following tools useful for connecting from Windows:

  • MobaXterm
  • PuTTY

Note on using MobaXterm on ARCHER:

When you log in to a system using MobaXterm, it also tries to set up a SFTP session. On ARCHER this cannot happen automatically so you will receive a prompt for SFTP browser password.

Two possible solutions are:

  • Turn off the SFTP browser
  • Re-enter your password when prompted

2.1.1 Interactive access to Post Processing nodes

The ARCHER post processing (PP) nodes are provided for compute, memory, or data-intensive operations that do not require access to parallel, compute nodes. /home, /work and the RDF filesystems are all mounted on the PP nodes.

The PP nodes can be accessed in two ways:

  1. Via the serial queues: see the description in the Post Processing Jobs section of this User Guide.
  2. Via direct interactive access: this is described below.

To connect to the PP nodes you must first be logged in to the ARCHER login nodes as described above. Once on the ARCHER login nodes you connect to one of the two PP nodes using the ssh command and one of the following host names:

  • espp1
  • espp2

For example, to connect to the espp1 PP node you would first log in to ARCHER and then use the command:

ssh espp1

You will be prompted for a password - this is your ARCHER password.

If you wish to export the PP node display back to your local workstation (for example you are using an application with a GUI). Then you should first log into ARCHER using the "-X" (or "-Y") option to ssh and then add this option to your ssh command to access the PP nodes. For example:

ssh -X espp1

Note: compiling programs for the PP nodes is a slightly different procedure than compiling for the compute nodes. See the Compiling for Post Processing nodes section in this User Guide. You can use the PP nodes for performing long compilations for the compute nodes using the standard compiling commands outlined in this User Guide.

Note: you must be logged in to the ARCHER login (or MOM/job launcher nodes via a batch job) before you can log in to the PP nodes.

2.2 Non-interactive SSH access

Disconnected tty/terminal processes will be terminated, which means that non-interactive SSH sessions will be terminated. To avoid this, use the -t flag (you may need more than one depending on how you are starting ssh). For example

ssh -t [userID]@login.archer.ac.uk 'echo start; sleep 60; echo finish'

2.3 Transferring data to and from ARCHER

The ARCHER systems are connected to the outside world via the UK academic SuperJANET5 network.

  • JANET Website

The simplest way of transferring data to and from ARCHER is using the scp command. Here are some examples:

Example 1:

The following command on your local system, will transfer the file source.f to your home directory on the ARCHER system.

scp ./source.f [username]@login.archer.ac.uk:
Example 2:

The following command will copy the file input_data.tar.gz from your local system to the RUN5 sub-directory on your work directory.

scp ./input_data.tar.gz \
  [username]@login.archer.ac.uk:/work/[project]/[group]/[username]/RUN5
Example 3:

The following command will copy the sub-directory RESULTS from your work directory to the current directory on your local system (note the use of the -r option).

scp -r [username]@login.archer.ac.uk:/work/[project>]/[group]/[username]/RESULTS ./
Example 4:

If your local system supports ssh logins, you may also run the scp commands from the ARCHER system. For example, the same transfer of the RESULTS sub-directory on ARCHER to the home directory of your local system could also be accomplished by running the following command on the ARCHER system:

scp -r /work/[project]/[group]/[username]/RESULTS \
  local_username@machine_name.institution.ac.uk:~

2.3.1 Performance considerations

ARCHER is capable of generating data at a rate far greater than the rate at which this can be downloaded over SuperJANET5. In practice, it is expected that only a portion of data generated on ARCHER will be required to be transferred back to users' institutions - the rest will be, for example, intermediate or checkpoint files required for subsequent runs. However, it is still essential that all users try to transfer data to and from ARCHER as efficiently as possible. The most obvious ways to do this are:

  1. Only transfer those files that are required for subsequent analysis, visualisation and/or archiving. Do you really need to download those intermediate result or checkpointing files? Probably not.
  2. Combine lots of small files into a single tar file, to reduce the overheads associated in initiating data transfers.
  3. Compress data before sending it, e.g. using gzip or bzip2.
  4. Consider doing any pre- or post-processing calculations on ARCHER. Long running pre- or post- processing calculations should be run via the batch queue system, rather than on the login nodes. Such pre- or post-processing codes could be serial or OpenMP parallel applications running on a single node, though if the amount of data to be processed is large, a MPI application able to use multiple nodes may be preferable.

Note that the performance of data transfers between ARCHER and your local institution may differ depending upon whether the transfer command is run on ARCHER or on your local system.

2.4 Making access more convenient using the SSH configuration file

Typing in the full command to login or transfer data to ARCHER can become tedious as it often has to be repeated many times. You can use the SSH configuration file, usually located on your local machine at ".ssh/config" to make things a bit more convenient.

Each remote site (or group of sites) can have an entry in this file which may look something like:

Host archer
  HostName login.archer.ac.uk
  User username

(remember to replace "username" with your actual username!).

The Host archer line defines a short name for the entry. In this case, instead of typing ssh login.archer.ac.uk to access the ARCHER login nodes, you could use ssh archer instead. The remaining lines define the options for the "archer" host.

  • Hostname login.archer.ac.uk - defines the full address of the host
  • User username - defines the username to use by default for this host (replace "username" with your own username on the remote host)

Now you can use SSH to access ARCHER without needing to enter your username or the full hostname every time:

-bash-4.1$ ssh archer

You can set up as many of these entries as you need in your local configuration file. Other options are available. See the ssh_config man page (or "man ssh_config" on any machine with SSH installed) for a description of the SSH configuration file. You may find the "IdentityFile" option useful if you have to manage multiple SSH key pairs for different systems as this allows you to specify which SSH key to use for each system.

1. Introduction | Contents | 3. Resource Management

Copyright © Design and Content 2013-2019 EPCC. All rights reserved.

EPSRC NERC EPCC