Modularity is a key architectural concept in building highly efficient HPC systems which match the computational and data processing needs of modern HPC applications and workflows. Accelerated Clusters replicate one node architecture (usually CPUs plus a number of accelerators, such as GPUs) and thus provide a fix ratio of CPU and accelerator resources; applications which require different ratios, or accelerators not present in the “master” node, will run with reduced performance, or leave significant parts of the nodes it runs on unused.
Modularity
The diagram shows a modular supercomputer with three compute modules and a separate storage module. Such a system is in operation at Jülich Supercomputing Centre for the DEEP-SEA project.
Potential future upgrades by quantum computing and/or neuromorphic modules are possible.
The Modular Supercomputer Architecture (MSA) on the other hand combines tightly coupled modules, each one consisting of a large number of interconnected accelerators or CPUs into a single system. Applications can now run on arbitrary combinations and numbers of CPUs and accelerators, which intelligent runtime systems compose into an execution system which perfectly matches the application requirements. Allocation and composition/decomposition can happen dynamically during runtime, matching a sequence of phases with tailored execution systems. This is of particular benefit to “mixed” workflows, which combine techniques from HPC, AI and Big Data Analytics. Given a reasonable match between workload characteristics and system configuration, MSA can improve time to solution, energy to solution, and throughput; it also addresses scalability limitations by running highly scalable code parts on highly parallel, very efficient accelerator modules, and parts with limited parallelism on CPUs with high single-thread performance.
This was invented in close collaboration between ParTec AG and the Jülich Supercomputing Centre (JSC) and was first demonstrated in production form in Cluster/Booster systems in 2015. In the DEEP projects, the basic MSA architecture, network federation, runtime systems and programming paradigms and tools were developed and perfected. MSA systems are now in use in several high-profile European HPC systems, and MSA is the architectural blueprint for future European Exascale and post-Exascale systems. In the DEEP projects, the basic MSA architecture, network federation, runtime systems and programming paradigms and tools were developed and perfected. MSA systems are now in use in several high-profile European HPC systems, and MSA is the architectural blueprint for future European Exascale and post-Exascale systems.
CLUSTER-BOOSTER CONCEPT
Cluster-Booster concepts have been known for years. However, they used static approaches, i.e., a static assignment of CPUs and GPUs. ParTec’s modular approach, on the other hand, allows dynamic assignments and thus high resource utilization. This results in a reduced need for nodes for predefined algorithms or increased performance with the same number
of nodes.
GENERALIZATION TO MODULARITY
The key advantage of the Modular System Architecture (MSA) is its match with the modern needs of HPC. These requirements are reliability, efficiency, and scalability.
ON-GOING TOPICS
HPC meets Quantum Computing
Quantum computing is a paradigm shift that enables the extension of binary computing to a multitude of states whether 0 or 1, represented by so-called “Q-Bits”. The integration of quantum modules into the Modular Supercomputing Architecture promises to be the next step in the evolution of computing.
ON-GOING TOPICS
Artificial Intelligence and HPC
Traditional HPC and innovative AI techniques will converge to pave the way for accelerating scientific discoveries and advancing applications, taking into account the heterogeneity of the Modular Supercomputing Architecture as well as the heterogeneity and complexity of application workflows.