Supercomputing world record

Computing Impact

Dr. Giuseppe Barca
Dr. Giuseppe Barca

Dr Giuseppe Barca and his team have set a new world record for quantum chemical calculations, using a supercomputer to predict the chemical reactions and physical properties of molecular systems with tens of thousands of atoms.

This new chemical modelling technology will be used on supercomputers worldwide, leading to new technological leaps in renewable energy, medicine, and advanced manufacturing.

The previous record was also held by Barca and his team. “The 2020 record was mainly about achieving unprecedented molecular sizes,” said Barca, a Lecturer in the ANU School of Computing.

“This new algorithm can not only model significantly larger molecular scales than before, but it can do it with a much higher accuracy than its predecessor.”

The previous code was 99% accurate, which might sound pretty good.

“Unfortunately, it has been known for over 50 years that 99% accuracy is not enough, by any means, to predict chemical reactions!” Barca said. “If we want to predict any chemical transformation, we need the additional 1%.”

Barca achieved “the additional 1%” by using electronically correlated methods that are so demanding from a computational perspective that they hadn’t been possible at large scale until the past year as the newest supercomputers came online.

Opening new frontiers for energy, medicine, nanotechnology

The higher accuracy means scientists will have the ability to model the breaking of chemical bonds, which will have a major impact on progress in chemical, physical, biological and engineering sciences.

The record was obtained as part of the Exascale Computing Project launched in 2016 as part of U.S. president Barack Obama’s National Strategic Computing initiative, a “whole-of-nation effort” to maximise the benefits of exascale computing systems, which are anticipated in 2022.

Exascale supercomputers will be capable of calculating more than 10¹⁸ (1 quintillion) floating point operations per second, which is 5 to 8 times more powerful than the fastest supercomputers operating today.

Barca serves as a Partner investigator for the ECP’s General Atomic and Molecular Electronic Structure System (GAMESS) team, which includes quantum chemists and HPC specialists charged with devising algorithms and methods to fully exploit the impending exascale supercomputers.

“These kinds of projects require highly multidisciplinary expertise with contributions from material science, chemistry, biology, quantum mechanics, supercomputing and artificial intelligence,” Barca said. “This record would have been unfeasible without my partners.”

The GAMESS team is led by renowned quantum chemist Dr Mark S. Gordon of Iowa State University and includes his students Jorge L. Galvez Vallejo, David L. Poole, and Melisa Alkan. In Australia, Barca is joined by his student at the ANU School of Computing, Ryan Stocks, and Barca’s mentor at ANU, Dr Alistair P. Rendell, who is now Dean of Engineering at Flinders University in Adelaide.

Barca is in charge of the software development on Graphics Processing Units (GPUs), which are the kinds of processors exascale machines, and many present-day supercomputers, use. He describes their mission as “building general tools for advancing science using the most advanced computational technology on the planet”. They will present their findings in November at Supercomputing 2021 Conference, the preeminent event for HPC.

The record-breaking calculation was run by the Summit supercomputer at the Oak Ridge lab in Tennessee—currently the world’s second fastest supercomputer. Using 27,600 GPUs, Summit required just over 11 minutes to simulate a protein including over 45,000 atoms and 180,000 electrons. It is the largest ab initio correlated quantum chemistry calculation to date, with speed, accuracy, and resolution surpassing all previous computational experiments.

Practical applications for “the additional 1%”

The paradigm shift will come as computer simulations take the place of traditional chemical experimentation while maintaining the same level of accuracy.

These calculations are expected to solve some of the world’s most challenging problems, streamlining technological advancement by shortening the time and cost of R&D processes, “or entirely replacing them when unfeasible,” Barca said.

Applications include (1) the development of more efficient catalysts for the production of second-generation biofuels, (2) the design of high-thermal-efficiency materials for low-carbon-emission buildings, (3) the design and production of new nanotechnologies to cure bacterial infections and improve cancer treatments, and (4) low-cost strategies for the industrial production of nanomaterials.

Nanomaterials are chemical substances or materials that are used at a very small scale. Barca said nanomaterials have “a myriad of applications,” including nanocrystals and quantum dots, battery materials, drug delivery vehicles and medical devices, supramolecular assemblies, and soft matters.

A 99%-accurate computational approach is already in use for the testing and screening of pharmaceuticals, including drugs for the treatment of COVID-19. This would be “incredibly expensive, inefficient and impossible without using high-performance computational chemistry,” Barca explained.

“Typically, a drug can have an effect if it binds to a biological receptor or enzyme. Now you can imagine that, for a given disease one can come up with an astronomically large list of candidate drugs, all with different chemical structures. Testing whether each one of these drugs binds to a receptor in the lab would be impractical,” he said.

HPC allows for simulations to take the place of experiments. “Since we know the physics and chemistry equations governing the binding, we can simulate the binding and screen out the drugs which are unlikely to be beneficial.”

After such screening, the “the additional 1%” becomes crucial for safe and effective innovation.

“If the prediction is incorrect, not only have we wasted money, time, equipment, and our reputation, but it may even lead to something much worse than that, especially if the lab R&D process is automated. It could lead to a chain reaction that causes an explosion or the release of a lethal gas,” Barca said.

“In order to predict the correct outcome of a chemical reaction, we need to be ~99.99% accurate. This is because whether or not a chemical reaction actually happens depends on the difference of energies between reactants and products. If my reactants have energy 1000 and my product 999.9, the difference of energy between the products and reactants is only -0.1. Sadly, this difference is enough to decide the outcome of our reaction and how fast it will happen.”

Cross-disciplinary collaboration in a pandemic

Before the pandemic, Barca had been travelling to the United States an average of 8 times per year, staying for at least 2 weeks each time. When case numbers there skyrocketed and travel restrictions were imposed, Barca was forced to manage the project remotely.

“I have weekly meetings with my ECP partners and my students in the US, and I meet fortnightly with the PaCER people,” he said, referring to a related project in conjunction with the Pawsey Center for Extreme-scale Readiness in Western Australia.

ANU is the only Australian partner in the ECP project. In the U.S., Barca coordinates with experts at the Ames National Lab in Iowa, the University of Texas El Paso, the Georgia Institute of Technology, Old Dominion University, and EP Analytics in addition to the aforementioned Iowa State University and Oak Ridge National Lab. ECP also enlists the expertise of America’s leading HPC corporations, NVIDIA, Intel, AMD, Cray.

To ease the communication, Barca set up two Slack channels where information can be shared across institutions, disciplines and continents. Participants are invited to “post news, chat and interact, discussing aspects of the project as well as whatever they would like to chat about”.

“Bringing these people together from different academic fields and backgrounds, giving them visibility on the projects’ recent developments, and stimulating them to have a say about future development directions, is a key aspect of our success,” Barca said.

arrow-left bars search caret-down plus minus arrow-right times arrow-up