ARCHER logo ARCHER banner

The ARCHER Service is now closed and has been superseded by ARCHER2.

  • ARCHER homepage
  • About ARCHER
    • About ARCHER
    • News & Events
    • Calendar
    • Blog Articles
    • Hardware
    • Software
    • Service Policies
    • Service Reports
    • Partners
    • People
    • Media Gallery
  • Get Access
    • Getting Access
    • TA Form and Notes
    • kAU Calculator
    • Cost of Access
  • User Support
    • User Support
    • Helpdesk
    • Frequently Asked Questions
    • ARCHER App
  • Documentation
    • User Guides & Documentation
    • Essential Skills
    • Quick Start Guide
    • ARCHER User Guide
    • ARCHER Best Practice Guide
    • Scientific Software Packages
    • UK Research Data Facility Guide
    • Knights Landing Guide
    • Data Management Guide
    • SAFE User Guide
    • ARCHER Troubleshooting Guide
    • ARCHER White Papers
    • Screencast Videos
  • Service Status
    • Detailed Service Status
    • Maintenance
  • Training
    • Upcoming Courses
    • Online Training
    • Driving Test
    • Course Registration
    • Course Descriptions
    • Virtual Tutorials and Webinars
    • Locations
    • Training personnel
    • Past Course Materials Repository
    • Feedback
  • Community
    • ARCHER Community
    • ARCHER Benchmarks
    • ARCHER KNL Performance Reports
    • Cray CoE for ARCHER
    • Embedded CSE
    • ARCHER Champions
    • ARCHER Scientific Consortia
    • HPC Scientific Advisory Committee
    • ARCHER for Early Career Researchers
  • Industry
    • Information for Industry
  • Outreach
    • Outreach (on EPCC Website)

You are here:

  • ARCHER

Can't list the navigation items at the moment. Please try again later.

  • ARCHER Community
  • ARCHER Benchmarks
  • ARCHER KNL Performance Reports
  • Cray CoE for ARCHER
  • Embedded CSE
  • ARCHER Champions
  • ARCHER Scientific Consortia
  • HPC Scientific Advisory Committee
  • ARCHER for Early Career Researchers

Contact Us

support@archer.ac.uk

Twitter Feed

Tweets by @ARCHER_HPC

ISO 9001 Certified

ISO 27001 Certified

Implementation of multi-level contact detection in granular LAMMPS to enable efficient polydisperse DEM simulations

eCSE12-09

Key Personnel

PI/Co-I: Tom Shire (U. Glasgow) and Kevin Hanley (U. Edinburgh)

Technical: Kevin Stratford (EPCC)

Relevant documents

eCSE Technical Report: Implementation of multi-level contact detection in LAMMPS

Summary: Summary (pdf)

Project summary

Granular materials occur in many physical and industrial settings, such as geomaterials, avalanches and landslides, crushing of mining ores, food processing and pharmaceuticals. In many cases the grains/granules cover a wide range of particle sizes. For example, the granular filter material used to construct the Bennett Dam in Canada contains particles ranging from 0.08 to 75 mm.

Discrete element modelling (DEM) is a computational tool for predicting how such granular materials will respond during loading, flowing or other processes found in nature or in industry. Each grain in a system is modelled as an individual element, and the elements interact to model to overall system response. The most time-consuming part of a DEM simulation is calculating which grains are in contact with each other and communicating this information between processors when running simulations in parallel. In most DEM codes these stages can become prohibitive when a wide range of particle sizes is employed, as the number of potential inter-particle contacts dramatically increases.

This eCSE project has implemented an improved contact detection scheme for DEM simulations of materials with a wide range of particle sizes into LAMMPS, a widely used open-source software which is especially efficient for massively parallel simulations which can be run using supercomputers such as ARCHER. The new scheme limits the search range over which particle-contacts are checked by defining different ranges depending on the size of particles: e.g., potential contacts between two large particles use a larger search area than between a small and large or two small particles.

For a typical DEM model in which the largest grain diameter is 100 times the smallest, such as that shown below, the newly implemented algorithm allows the simulation to run over 400 times faster and use significantly less RAM memory.

The new implementation also scales well when used in parallel. We are working to incorporate the implementation into the main version of LAMMPS run by Sandia National Laboratories in the USA, which will ensure its long-term sustainability. This new implementation will allow previously intractable problems in materials with a wide range of particle sizes to be simulated using DEM, such as the study of liquefaction of sands, but also will be of interest in other areas such as simulation of colloid/solvent interactions in particle suspensions.

Achievement of objectives

The objectives of the work were to:

  1. Adapt existing contact detection in LAMMPS.
    A new multi-level algorithm "neighbor bytype" has been implemented and produces the same results as existing code in LAMMPS. "The same results" here encompasses explicit checks on the number and identity of neighbours, and global measures such as the pair energy for standard test configurations.
  2. Quantify improvements in simulation time.
    For bi-disperse cases the new algorithm improves performance by up to three orders of magnitude for 100:1 size ratios compared with "neighbour multi".
    For polydisperse cases, improvements of up to a factor of 3 are recorded across a wide range of size ratios compared with "neighbor multi". (Comparisons with "neighbour bin", if practical at all, would be likely to produce very much larger improvements.)
  3. Release a well-documented implementation
    While no release is available at the time of writing, we are working with the owners at Sandia to incorporate the changes. Ultimately, the timing of any release in the main LAMMPS code will be determined by Sandia.
  4. Demonstrate scalability of highly polydisperse DEM simulations on ARCHER.
    For bidisperse cases, the new contact detection algorithm, combined with an improved communication scheme, demonstrates improved scalability on up to 32 (768 MPI tasks) nodes for representative benchmark problems.
  5. Provide advice to users on the optimum parameters for contact detection.
    A number of common polydisperse cases are considered, and optimum parameters are identified. However, for general particle size distributions, some caution is warranted in making sweeping statements.

Summary of the Software

LAMMPS [1] is a flexible and popular code performing classical molecular dynamics. It is developed by Sandia and is publicly available under GPL [2]. While the developments made by this work are not yet incorporated into the main version of LAMMPS, we hope they will be so in the near future. In the meantime, the relevant updates can be made available on request.

[1] https://lammps.sandia.gov
[2] https://github.com/lammps/lammps

Scientific Benefits

We believe the improvements represent a potentially very significant increase in the efficiency of the LAMMPS code in cases where there is a large size discrepancy in the particles in the system. Such systems are common in granular media, and will allow new kinds of system to be studied in appropriate detail.

Copyright © Design and Content 2013-2019 EPCC. All rights reserved.

EPSRC NERC EPCC