Modularity is a key architectural concept in building highly efficient HPC systems which match the computational and data processing needs of modern HPC applications and workflows. Accelerated Clusters replicate one node architecture (usually CPUs plus a number of accelerators, such as GPUs) and thus provide a fix ratio of CPU and accelerator resources; applications which require different ratios, or accelerators not present in the “master” node, will run with reduced performance, or leave significant parts of the nodes it runs on unused.
The dynamic Modular Supercomputer Architecture (dMSA) on the other hand combines tightly coupled modules, each one consisting of a large number of interconnected accelerators or CPUs into a single system. Applications can now run on arbitrary combinations and numbers of CPUs and accelerators, which intelligent runtime systems compose into an execution system which perfectly matches the application requirements. Allocation and composition/decomposition can happen dynamically during runtime, matching a sequence of phases with tailored execution systems. This is of particular benefit to “mixed” workflows, which combine techniques from HPC, AI and Big Data Analytics. Given a reasonable match between workload characteristics and system configuration, dMSA can improve time to solution, energy to solution, and throughput; it also addresses scalability limitations by running highly scalable code parts on highly parallel, very efficient accelerator modules, and parts with limited parallelism on CPUs with high single-thread performance.
This was invented in close collaboration between ParTec AG and the Jülich Supercomputing Centre (JSC) and was first demonstrated in production form in Cluster/Booster systems in 2015. In the DEEP projects, the basic dMSA architecture, network federation, runtime systems and programming paradigms and tools were developed and perfected. dMSA systems are now in use in several high-profile European HPC systems, and dMSA is the architectural blueprint for future European Exascale and post-Exascale systems.
The diagram above shows a modular supercomputer with three compute modules and a separate storage module. Such a system is in operation at Jülich Supercomputing Centre for the DEEP-SEA project.
Potential future upgrades by quantum computing and/or neuromorphic modules are possible.
Please find further details in our white paper:
