Prometheus supercomputer
Prometheus, operating at ACK Cyfronet AGH, has been listed 15 times in a row on the TOP500 list of the world's fastest supercomputers since its launch in 2015, with the highest position at 38. Its high placement on the TOP500 list was ensured by its computing power of 2.65 PFlops (PetaFlops), which was achieved primarily by using high-performance servers of the HP Apollo 8000 platform and connecting them with a 56 Gbps superfast InfiniBand network. The supercomputer has 53,748 computing cores (energy-efficient and high-performance Intel Haswell and Intel Skylake processors) and 283.5 TB of DDR4 technology operating memory. The Prometheus supercomputer has two file systems with a total capacity of 10 PB and an access speed of 180 GB/s. It also features NVIDIA Tesla cards with GPGPUs. Prometheus was built by Hewlett-Packard according to the assumptions developed by Cyfronet experts.
The Prometheus architecture responds to the diverse needs of researchers by providing computing resources organised in partitions:
- classical cluster of computing servers with highly efficient CPU nodes equipped with Intel Xeon Haswell and Intel Xeon Gold processors,
- cluster of servers equipped with graphic accelerators GPGPU NVIDIA Tesla K40 XL,
- acceleration partition including GPGPU NVIDIA K80 cards as well as Intel Xeon Phi 7120P and Nallatech FPGA accelerators,
- partition dedicated to calculations related to artificial intelligence, equipped with GPGPU NVIDIA Tesla V100 graphics accelerators. It is worth mentioning that this partition is a system with computing power over 4 PFlops for tensor operations and 256 TFlops for standard calculations performed on double precision numbers, which made it the fastest dedicated solution for artificial intelligence available for the needs of scientists in Poland at the time.
Prometheus was distinguished by an innovative liquid cooling system. Thanks to the technology used, in its first years of use it was one of the most energy-efficient supercomputers of its class in the world. To maintain the appropriate temperature of the cooling liquid in our climate, dry-coolers (heat exchangers), which are cheaper to operate, are sufficient, instead of chilled water generators consuming large amounts of electricity and classic process air conditioners. The liquid is cooled by both CPUs, GPGPUs and memory modules using a special airtight system of heat pipes and contact heat transfer between the servers and the system in which the liquid circulates. The servers remain “dry” - they can be removed anytime without the danger of liquid leakage in the cooling system. Liquid cooling made it possible to achieve a very high (compared to other machines running at a similar time) installation density - as many as 144 computing servers in a single rack.
Prometheus has been installed in a high-tech computing room, exclusively adapted for its operation. The supercomputer's proper functioning is additionally supported by the accompanying infrastructure, including a guaranteed power supply with an additional generator, modern air-conditioning and gas extinguishing.
The supercomputer is a part of the PLGrid Infrastructure.