On November 13, 2023, during The International Conference for High Performance Computing, Networking, Storage, and Analysis in Denver, US, the latest TOP500 list of supercomputers with the highest computing power in the world was announced. The list includes three supercomputers operating in Cyfronet: Athena, Helios and Ares.
The autumn TOP500 list (top500.org) is the eighth one which includes more than one Cyfronet's supercomputer. Additionally, Cyfronet's machines appeared in the Green500 list of most energy-efficient supercomputers.
Athena – the fastest supercomputer in Poland
Athena is currently a flagship system of Cyfronet. It achieves the theoretical computing power of over 7.7 PetaFlops (7709 TeraFlops), giving the machine 155th place in the November TOP500 list and making it the fastest supercomputer in Poland. The system installed in Cyfronet in 2021 provides the Polish scientific community and the economy with the most modern computing resources based on the latest generation CPU processors and GPGPU accelerators and the necessary data storage subsystem found on high-speed flash memory. Athena's configuration includes 48 servers with AMD EPYC processors (6144 CPU cores) and 384 NVIDIA A100 GPGPU cards.
An indispensable element enabling the use of such high computing power effectively is to provide a high-performance internal network of a supercomputer (Infiniband HDR with a bandwidth of 4 x 200 Gb/s per server) and a high-speed disk subsystem. It’s built based on the Lustre open-source software and dedicated disk servers equipped with flash memory in the NVMe standard.
This type of infrastructure meets the needs of users of Cyfronet supercomputers, who use computing infrastructure both to perform standard high-performance scientific simulations (HPC) and to apply artificial intelligence (AI) and machine learning (ML) methods in research in the field of medicine, pharmacology, biology, chemistry, physics, and many other areas of science. Athena's computing power for AI computing is almost 240 PetaFlops!
Helios – a new way for innovation in Polish science and economy
The Helios supercomputer is a new system installed at ACC Cyfronet AGH, created as result of work performed in the National Supercomputing Infrastructure for EuroHPC - EuroHPC PL project coordinated by Cyfronet. The supercomputer was built according to Cyfronet's design by Hewlett-Packard Enterprise based on the HPE Cray EX4000 platform. It will ultimately consist of three computing partitions: 1) CPU equipped with 75,264 AMD Zen4 computing cores and 200 TB of DDR5 RAM, 2) GPU equipped with 440 NVIDIA Grace Hopper GH200 superchips, and 3) INT for interactive work, equipped with 24 NVIDIA H100 accelerators and fast local NVMe memory.
Due to the current system installation, only a general-purpose processor (CPU) partition with a computing power of 3.4 PetaFlops was submitted for testing for TOP500. Nevertheless, Helios was included in the TOP500 list and ranked 291st position. After complete system installation, Helios is expected to achieve approximately 35 PetaFlops of theoretical computing power when all partitions are launched. On the current list, this power would allow Helios to rank among the 50 fastest supercomputers in the world.
The Helios disk subsystem consists of two types of Luster file systems: scratch with a capacity of 1.5 PB and a speed of over 1.8 TB/s and project with a capacity of 16 PB and a speed of almost 200 GB/s. All supercomputer components are connected via the Slingshot network with a speed of 200 Gb/s. The HPE Cray EX4000 platform, based on which Helios was built, is also used in constructing the fastest supercomputers in the world (Frontier) and Europe (LUMI). Thanks to direct liquid cooling of the CPU and GPU partitions, it is possible to achieve a very low PUE (Power Usage Effectiveness) for the system, increasing its energy efficiency and reducing operating costs. Additionally, thanks to the recovery of waste heat produced by Helios, it will be possible to use it for heating.
Already in the first half of 2024 - after completion of the installation and the stage of thorough infrastructure testing - Helios will be available to both scientific units and economic and public administration entities. The expected effect is the development of innovative research and the acceleration of introducing technologies based on it to the market.
Ares – the warrior for science
The Ares supercomputer offers a total computing power of over 4 PetaFlops (the theoretical performance of the CPU part is over 3.5 PetaFlops and the GPU part is over 0.5 PetaFlops). Ares is composed of computing servers with Intel processors (Xeon Platinum model), divided into three partitions:
- 532 servers equipped with 192 GB of RAM each,
- 256 servers equipped with 384 GB of RAM each,
- 9 servers equipped with 8 NVIDIA Tesla V100 cards each.
The supercomputer has 37,824 computing cores. Ares is also supported by a disk system with a capacity of over 11 PB. An InfiniBand EDR network is used for data transfer. On the November TOP500 list, Ares took the 404th position.
The world's top energy efficiency
Parallel to the announcement of the TOP500 list of supercomputers with the highest computing power, the Green500 list of the most ecological supercomputers is published. The main criterion (energy efficiency) is the ratio of the number of floating-point operations per second (computing power of a supercomputer) to energy consumption. Athena's 22th place on the Green500 list of the world's most energy-efficient supercomputers is a particular success. This position proves the excellent ratio of the provided computing power to the electricity consumption. Ares was also on the list and took 114st place. In turn, Helios – CPU took position no. 344 on the Green500 list.
Scientific calculations in various fields
Scientists conducting research using Cyfronet's infrastructure represent many fields. Advanced modelling and numerical calculations are mainly used in chemistry, biology, physics, medicine and material technology, astronomy, geology, and environmental protection. Supercomputers in Cyfronet, which are available as part of the PLGrid infrastructure, are also used for high-energy physics (ATLAS, LHCb, ALICE and CMS projects), astrophysics (CTA, LOFAR), Earth sciences (EPOS), European Spallation Source (ESS), biology (WeNMR) and humanities and social sciences (CLARIN).
The World’s and European winners
Frontier, installed at the Oak Ridge National Laboratory (ORNL) in the US, was announced as the fastest supercomputer in the world. Frontier is the first supercomputer which achieved more than 1 ExaFlops (1018 floating point operations per second), and its theoretical computing power is 1.68 ExaFlops. The global TOP10 includes 6 machines from the US, 3 from Europe and 1 from Japan.
The fastest European supercomputer is again the LUMI supercomputer, built thanks to the cooperation of 10 European countries, including Poland. In the TOP500 list, LUMI was ranked the 5th place. The current theoretical computing power of the supercomputer is 531 PetaFlops. Polish scientists can access the supercomputer via the PLGrid infrastructure. ACC Cyfronet AGH coordinates activities in this area — more information: https://cyfronet.pl/lumi.
Polish supercomputers on the TOP500 list
The current TOP500 list includes 4 supercomputers from Poland, including 3 from ACC Cyfronet AGH. The complete list of achieved places with the values of computing power is as follows:
- 155 – Academic Computer Centre CYFRONET, Athena (7710 TeraFlops)
- 221 – Poznan Supercomputing and Networking Centre, Altair (5880 TeraFlops),
- 291 – Academic Computer Centre CYFRONET, Helios (3400 TeraFlops).
- 404 – Academic Computer Centre CYFRONET, Ares (3510 TeraFlops).
Use the power of supercomputers in your research
Supercomputers make it possible to significantly reduce the time of performing calculations, which using single computers would often take many years (in specific real cases, over 150, 700 or even 1000 years). Thanks to supercomputers, calculations can most often be performed in just a few hours and for large tasks – in a few days. Cyfronet’s users benefit from professional support – from training sessions to individual expert consultations.
Information about Cyfronet's resources is available at https://cyfronet.pl.
The list of services within the PLGrid infrastructure is available in the PLGrid guides.
We cordially invite you to register in the PLGrid Portal and use many unique services.
