Cyfronet – for the first time three supercomputers from one Polish computing centre are on the TOP500 list of the fastest supercomputers in the world
On May 30, 2022, during the ISC High-Performance conference in Hamburg, the latest TOP500 list of supercomputers with the highest computing power in the world was announced. For the first time in history, the list simultaneously includes three supercomputers from one Polish computing centre. These are those operating in Cyfronet: Athena, Ares, and Prometheus.
Athena is currently the fastest supercomputer in Poland and ranks 105th place. In addition, Cyfronet's 3 supercomputers were included in the Green500 list - the greenest supercomputers, including Athena in the top ten of Green500 in 9th position.
Athena - the fastest supercomputer in Poland
Athena achieves the theoretical computing power of over 7.7 PetaFlops (7709 TeraFlops), which gave the machine 105th place on the June TOP500 list and makes it the fastest supercomputer in Poland at present. The system installed in Cyfronet in 2021 provides the Polish scientific community and the economy with the most modern computing resources based on the latest generation GPGPU processors and accelerators and the necessary data storage subsystem based on high-speed flash memory. Athena's configuration includes 48 servers with AMD EPYC processors, 1 TB of RAM (6144 CPU cores in total), and 384 NVIDIA A100 GPGPU cards.
An indispensable element enabling the use of such high computing power effectively is to provide a high-performance internal network of a supercomputer (Infiniband HDR with a bandwidth of 4 x 200 Gb/s per server) and a high-speed disk subsystem. It’s built on the basis of the Lustre open-source software (also used in the disk systems of the Prometheus and Zeus supercomputers) and dedicated disk servers equipped with flash memory in the NVMe standard. The system was installed in the existing Cyfronet data centre and integrated with the PLGrid infrastructure.
This type of infrastructure meets the needs of users of Cyfronet supercomputers, who use computing infrastructure both to perform standard high-performance scientific simulations (HPC) and to apply artificial intelligence (AI) and machine learning (ML) methods in research in the field of medicine, pharmacology, biology, chemistry, physics, and many other areas of science. Athena's computing power for AI computing is almost 240 PetaFlops!
The expected effect of delivering specialized computing resources of Athena will be the expansion of the scope of research work, the possibility of undertaking advanced simulations and analyses, and increasing the possibilities of processing continuously incoming data from laboratories around the world. The direct expected results of the work will be articles and scientific studies, patents, and in the longer term, innovative solutions that may be the basis for developing new solutions in the economy.
The Athena supercomputer
Ares - a new figure in the Polish scientific arena
The total theoretical performance of the Ares' CPU part is over 3.5 PetaFlops (3510 TeraFlops), and of the GPU part is over 500 TFlops. Ares is also supported by a disk system with a capacity of over 11 PB. An InfiniBand EDR network is used for data transfer. The supercomputer has 37 824 computing cores and 147.7 TB of RAM. On the June TOP500 list, Ares took 290th position
Launched in 2021, the Ares supercomputer is built of computing servers with Intel Xeon Platinum processors, divided into three partitions:
• 532 servers equipped with 192 GB of RAM,
• 256 servers with 384 GB of RAM each,
• 9 servers with 8 NVIDIA Tesla V100 cards each.
Ares supplemented Cyfronet's computing resources with newer generation processors and the use of servers with more available memory. Its availability shortened the time of calculating scientific tasks and made it possible to analyze problems that so far could not be run on a large scale due to insufficient memory on previously available resources. In addition, the location of Ares in the Backup Data Center – in a geographically different area than Prometheus – guarantees the continuity of the provision of computing services in the event of sudden random events.
The Ares supercomputer
Prometheus – fast and energy-saving
Prometheus, working at the ACC Cyfronet AGH, has been consistently listed on the TOP500 list of the world's fastest supercomputers since its launch in 2015. In the latest edition, it took 475th place. This position is ensured by the computing power of over 2.3 PetaFlops (2349 TeraFlops), which was achieved mainly thanks to the use of efficient servers of the HP Apollo8000 platform and connecting them with the super-fast InfiniBand network with a capacity of 56 Gb/s. The supercomputer has 53 748 computing cores (energy-saving and efficient Intel Haswell and Intel Skylake processors) and 283.5 TB of RAM in DDR4 technology. Its computing servers are cooled directly by the liquid, which ensures high energy efficiency. Prometheus comes with two file systems with a total capacity of 10 PB and access speed of 180 GB/s. It also comes with NVIDIA Tesla cards with GPGPU graphics processors.
The architecture of the Prometheus supercomputer responds to the diverse needs of scientists by providing resources organized in partitions:
• classic computing servers with high-performance Intel Xeon Haswell and Intel Xeon Gold processors,
• a set of servers with NVIDIA K40 XL graphics processors,
• an acceleration partition with NVIDIA K80 GPGPU cards and Intel Xeon Phi and Nallatech FPGA accelerators,
• a partition dedicated to computing related to artificial intelligence, equipped with GPGPU NVIDIA Tesla V100 graphics accelerators.
The Prometheus supercomputer (A. Frączek, KSAF AGH)
All supercomputers have been installed in Cyfronet's modern computer rooms, specially adapted to the operation of such machines. The proper functioning of supercomputers is ensured by essential elements of the technical infrastructure, such as a guaranteed power supply system with an additional power generator and modern technological air-conditioning and gas extinguishing systems.
Certificates confirming places on TOP500:
The world's top energy efficiency
Parallel to the announcement of the TOP500 list of supercomputers with the highest computing power, the Green500 list of the most ecological supercomputers is published. The main criterion (energy efficiency) is calculated as the ratio of the number of floating-point operations per second (computing power of a supercomputer) to energy consumption: Gflops/W. Athena's 9th place on the Green500 list of the world's most energy-efficient supercomputers is a particular success. This position proves the excellent ratio of the provided computing power to the electricity consumption. Ares and Prometheus were also on the Green500 list. Ares took 83rd place, and Prometheus ranked 162nd position. Both systems are cooled directly with the liquid, facilitating such high energy efficiency.
Scientific calculations in various fields
Scientists conducting research using Cyfronet's infrastructure represent many fields. Advanced modeling and numerical calculations are mainly used in physics, chemistry, biology, medicine and material technology, as well as astronomy, geology, and environmental protection. Research topics include dynamic analysis of ceramic composite materials, modeling and optimization of an energy-saving hydrogen fuel cell prototype, simulation of molecular dynamics of electrolytes for ion cells or using quantum methods for image analysis. Scientists also research gene expression in the Baltic Sea cod, study the interaction of influenza virus fusion peptides with lipid membranes, analyze the impact of underground fluid injection on stress drops induced by seismic tremors, and study microphysical processes in space plasma. Supercomputers in Cyfronet available as part of the PLGrid infrastructure are also used for high-energy physics (ATLAS, LHCb, ALICE and CMS projects), astrophysicists (CTA, LOFAR), Earth sciences (EPOS), European Spallation Source (ESS), biologists (WeNMR) and humanities and social scientists (CLARIN).
The World’s and European winners
Frontier, installed at the Oak Ridge National Laboratory (ORNL) in the USA, was announced the fastest supercomputer in the world. Frontier is the first supercomputer which achieved more than 1 ExaFlop (1018 floating point operations per second), and its theoretical computing power is 1685.65 PFlops (1.69 EFlops).
In turn, the fastest European supercomputer was the LUMI supercomputer, built thanks to the cooperation of 10 European countries, including Poland. In the TOP500 list, LUMI was ranked 3rd place, even though the GPU partition was not yet fully installed. The current theoretical computing power of the supercomputer is 537 PFlops. Polish scientists can access the supercomputer via the PLGrid infrastructure. ACC Cyfronet AGH coordinates activities in this area. More information: https://cyfronet.pl/lumi.
Polish supercomputers on the TOP500 list
In the current edition of the TOP500 list, two other supercomputers from Poland were mentioned apart from Athena, Ares and Prometheus. The complete list of achieved places with the values of computing power is as follows:
- 105th – Academic Computer Centre CYFRONET, Athena (7709 TFlops),
- 145th – Poznan Supercomputing and Networking Centre, Altair (5880 TFlops),
- 290th – Academic Computer Centre CYFRONET, Ares (3510 TFlops),
- 462nd – Centre of Informatics Tricity Academic Supercomputer and networK, Tryton Plus (2824 TFlops),
- 475th - Academic Computer Centre CYFRONET, Prometheus (2349 TFlops).
Use the power of supercomputers in your research
Supercomputers make it possible to significantly reduce the time of performing calculations, which using single computers would often take many years (in specific real cases, over 150, 700 or even 1000 years). Thanks to supercomputers, the calculations can most often be performed in just a few hours and for large tasks – in a few days. Cyfronet’s users benefit from professional support – from training sessions to individual consultations with experts. Information about Cyfronet's resources is available at https://cyfronet.pl. The list of services within the PLGrid infrastructure is available in the Application Catalog. We cordially invite you to register in the PLGrid Portal and use many unique services.