ARCHER logo ARCHER banner

The ARCHER Service is now closed and has been superseded by ARCHER2.

  • ARCHER homepage
  • About ARCHER
    • About ARCHER
    • News & Events
    • Calendar
    • Blog Articles
    • Hardware
    • Software
    • Service Policies
    • Service Reports
    • Partners
    • People
    • Media Gallery
  • Get Access
    • Getting Access
    • TA Form and Notes
    • kAU Calculator
    • Cost of Access
  • User Support
    • User Support
    • Helpdesk
    • Frequently Asked Questions
    • ARCHER App
  • Documentation
    • User Guides & Documentation
    • Essential Skills
    • Quick Start Guide
    • ARCHER User Guide
    • ARCHER Best Practice Guide
    • Scientific Software Packages
    • UK Research Data Facility Guide
    • Knights Landing Guide
    • Data Management Guide
    • SAFE User Guide
    • ARCHER Troubleshooting Guide
    • ARCHER White Papers
    • Screencast Videos
  • Service Status
    • Detailed Service Status
    • Maintenance
  • Training
    • Upcoming Courses
    • Online Training
    • Driving Test
    • Course Registration
    • Course Descriptions
    • Virtual Tutorials and Webinars
    • Locations
    • Training personnel
    • Past Course Materials Repository
    • Feedback
  • Community
    • ARCHER Community
    • ARCHER Benchmarks
    • ARCHER KNL Performance Reports
    • Cray CoE for ARCHER
    • Embedded CSE
    • ARCHER Champions
    • ARCHER Scientific Consortia
    • HPC Scientific Advisory Committee
    • ARCHER for Early Career Researchers
  • Industry
    • Information for Industry
  • Outreach
    • Outreach (on EPCC Website)

You are here:

  • ARCHER

Can't list the navigation items at the moment. Please try again later.

  • ARCHER Community
  • ARCHER Benchmarks
  • ARCHER KNL Performance Reports
  • Cray CoE for ARCHER
  • Embedded CSE
  • ARCHER Champions
  • ARCHER Scientific Consortia
  • HPC Scientific Advisory Committee
  • ARCHER for Early Career Researchers

Contact Us

support@archer.ac.uk

Twitter Feed

Tweets by @ARCHER_HPC

ISO 9001 Certified

ISO 27001 Certified

Tuning FHI-Aims for complex simulations on Cray HPC platforms

eCSE01-018

Key Personnel

PI/Co-I: Dr Scott Woodley, Prof Richard Catlow, Dr Alexey Sokol, Dr Andrew Logsdail - University College London

Technical: Dr Matthew Farrow - University College London

Relevant documents

eCSE Technical Report: Tuning FHI-Aims for complex simulations on CRAY HPC platforms

Project summary

This project was concerned with porting and optimising the electronic structure code FHI-aims for use on the UK national supercomputer, ARCHER. The FHI-aims software has been developed specifically for highly accurate modelling of a variety of system types, including solids and nanoparticles, using state-of-the-art Density Functional Theory (DFT), which focuses on the behaviour of electrons and atoms in a chemical compound. The methodology adopted in the FHI-aims code uses numerical atomic orbitals, or electron trajectories, which, in theory, are extremely efficient for calculations on many processors simultaneously. Therefore, the code is very advantageous for usage on massive supercomputers such as ARCHER. Our work thus enables and encourages the undertaking of new research using current and emerging UK computer facilities in a wide range of fields including materials and life sciences, chemistry, physics, and engineering.

We have compiled and benchmarked the software on ARCHER and created a module for current users to load conveniently an optimised executable. This work has immense benefit for potential academic and industrial user communities. Currently, users of FHI-aims have to install the software themselves, which requires specialist knowledge and is time consuming. Moreover, the users are unlikely to be using the latest version of the software which is more efficient and has previously-discovered problems resolved. This module allows all users to be using the same code base, which is important for scientific reproducibility.

test systems modelled by the software

Graphical representation of test systems modelled by the software

In our project, we have used profiling tools to determine which portions of the software take the longest time. For example, the performance analysis flagged one routine that took an unexpected time to complete, which we have addressed, and this resulted in an optimised routine.

We highlight the newly available self-consistent meta-GGA density functionals, which allows for very accurate chemical calculations and was previously implemented in a post-processing, and thus approximate, manner. Hence, new predictive studies can now be expected in the description of complex chemical reactions.

In the last part of the project and in support of KLMC runs, FHI-aims has been optimised to facilitate cheaper pre-screening on DFT energy landscapes when more traditional interatomic potentials are not available. Making meta-GGA functionals available in ChemShell enables new scientific applications on one hand and speeds up significantly more challenging reactivity problems. However, there is still a significant scope for further code optimisation in FHIaims for the embedding approach implemented within ChemShell. Support for further work in both directions would be very beneficial for various applications and, therefore, will form the basis of future proposals.

Achievement of objectives

The main objective was to "tune FHI-aims for optimal efficiency on ARCHER's hardware, both as a standalone code and as a key component within two software packages - KLMC and ChemShell - for the benefit of members of the Materials Chemistry Consortium (MCC) and other new or current users. Improving the scalability for a wide range of applications to enable new science is the overarching target".

We fulfilled this objective by porting an optimal installation of the FHI-aims code to the ARCHER platform, which is available to both KLMC and ChemShell, and by the creation of a module that future users can exploit by simply typing the phrase "module load aims". This command will provide future users of ARCHER, who have a license to run FHI-aims, access to the latest version of the code. To optimise the FHI-aims code for the ARCHER platform, we have tuned the optimal compiler settings; these settings are published on the FHI-aims wiki page for users of FHI-aims and were used in the compilation of the executable that is made available to ARCHER users via the "module load aims" command.

As typically found for codes newly ported onto a high-end compute (HEC) platform, the FHI-aims software also suffered from a number of bottlenecks when initially ported onto ARCHER. There are different types of jobs the user can use FHI-aims to perform; therefore, the bottlenecks are of course dependent upon the chosen task. We identified the bottlenecks of FHI-aims when used to perform typical simulations that are of interest to the materials chemistry community. These bottlenecks were brought to the attention of the core developers; patches to the code that addresses these bottlenecks will be made available in future releases of the software.

A large proportion of ARCHER resource is consumed by hybrid DFT based calculations and, in particular, performed using VASP. The objective in the largest of the four work packages was to provide a more efficient approach. The implementation of meta-generalised gradient approximation (meta-GGA) density functionals is an important and essential enhancement of current capabilities, which resulted from the eCSE and EPSRC support of this work. These functionals are an order of magnitude cheaper than hybrid functionals and offer an improved accuracy compared to the more traditional functionals that hybrid functionals have replaced.

The relationship between the core developers of FHI-aims and the UCL group has also been strengthened by the success of this project. Further performance enhancements are still underway at UCL within our group; we are finding and implementing new algorithmic solutions to the basic SCF code infrastructure. The eCSE work packages were exploratory in nature and ambitious, and have opened new avenues to pursue, for which we will seek further eCSE support.

Summary of the software

The software is held in a git repository accessible from:

https://aimsclub.fhi-berlin.mpg.de/index.php

On ARCHER, access can be obtained for the "aims" module by providing the ARCHER helpdesk with proof of licence. In general, the FHI-aims software is obtained directly from the developer (https://aimsclub.fhi-berlin.mpg.de) and a licence is required for its use.

Support for FHI-aims usage on ARCHER will be provided in the future by our team via the Materials Chemistry Consortium (MCC) framework.

Copyright © Design and Content 2013-2019 EPCC. All rights reserved.

EPSRC NERC EPCC