Logo: to the web site of Uppsala University

This work studies three multigrid variants for matrix-free finite-element computations on locally refined meshes: geometric local smoothing, geometric global coarsening (both h-multigrid), and polynomial global coarsening (a variant of p-multigrid). We have integrated the algorithms into the same framework-the open source finite-element library deal.II-, which allows us to make fair comparisons regarding their implementation complexity, computational efficiency, and parallel scalability as well as to compare the measurements with theoretically derived performance metrics. Serial simulations and parallel weak and strong scaling on up to 147,456 CPU cores on 3,072 compute nodes are presented. The results obtained indicate that global-coarsening algorithms show a better parallel behavior for comparable smoothers due to the better load balance, particularly on the expensive fine levels. In the serial case, the costs of applying hanging-node constraints might be significant, leading to advantages of local smoothing, even though the number of solver iterations needed is slightly higher. When using p- and h-multigrid in sequence (hp-multigrid), the results indicate that it makes sense to decrease the degree of the elements first from a performance point of view due to the cheaper transfer.

Approximately 75% of the raw material and 50% of the products in the chemical industry are granular materials. The discrete element method (DEM) provides detailed insights of phenomena at particle scale, and it is therefore often used for modeling granular materials. However, because DEM tracks the motion and contact of individual particles separately, its computational cost increases nonlinearly O (n(p) log(n(p))) - O (n(2)) (depending on the algorithm) with the number of particles (n(p)). In this article, we introduce a new open-source parallel DEM software with load balancing: Lethe-DEM. Lethe-DEM, a module of Lethe, consists of solvers for two-dimensional and three-dimensional DEM simulations. Load balancing allows LetheDEM to significantly increase the parallel efficiency by approximate to 25-70% depending on the granular simulation. We explain the fundamental modules of Lethe-DEM, its software architecture, and the governing equations. Furthermore, we verify LetheDEM with several tests including analytical solutions and comparison with other software. Comparisons with experiments in a flat-bottomed silo, wedge-shaped silo, and rotating drum validate Lethe-DEM. We investigate the strong and weak scaling of Lethe-DEM with 1 <= n(c) <= 192 and 32 <= n(c) <= 320 processes, respectively, with and without load balancing. The strong-scaling analysis is performed on the wedge-shaped silo and rotating drum simulations, while for the weak-scaling analysis, we use a dam-break simulation. The best scalability of Lethe-DEM is obtained in the range of 5000 <= n(p)/n(c) <= 15,000. Finally, we demonstrate that large-scale simulations can be carried out with Lethe-DEM using the simulation of a three-dimensional cylindrical silo with n(p) = 4.3 x 10(6) on 320 cores.

We present an implementation of a stage-parallel preconditioner for Radau IIA type fully implicit Runge–Kutta methods, which approximates the inverse of the Runge–Kutta matrix AQ from the Butcher tableau by the lower triangular matrix resulting from an LU decomposition and diagonalizes the system with as many blocks as stages. For the transformed system, we employ a block preconditioner where each block is distributed and solved by a subgroup of processes in parallel. For combination of partial results, we use either a communication pattern resembling Cannon’s algorithm or shared memory. A performance model and a large set of performance studies (including strong-scaling runs with up to 150k processes on 3k compute nodes) conducted for a time-dependent heat problem, using matrix-free finite element methods, indicate that the stage-parallel implementation can reach higher throughputs near the scaling limit. The achievable speedup increases linearly with the number of stages and is bounded by the number of stages. Furthermore, we show that the presented stage-parallel concepts are also applicable to the case that AQ is directly diagonalized, which requires either complex arithmetic or solutions of two-by-two blocks, both exposing about half the parallelism. Alternatively to distributing stages and assigning them to distinct processes, we discuss the possibility of batching operations from different stages together.

This contribution presents an efficient algorithm for resolving hanging-node constraints on the fly for high-order finite-element computations on adaptively refined meshes, using matrix-free implementations. We concentrate on unstructured hex-dominated meshes and on multi-component elements with nodal Lagrange shape functions in at least one of their components. The application of general constraints is split up into two distinct operators, one specialized in the hanging-node part and a generic one for the remaining constraints, such as Dirichlet boundary conditions. The former implements in-face interpolations efficiently by a sequence of 1D interpolations with sum factorization according to the refinement configuration of the cell. We discuss ways to efficiently encode and decode such refinement configurations. Furthermore, we present distinct differences in the interpolation step on GPU and CPU, as well as compare different vectorization strategies for the latter. Experimental comparisons with a state-of-the-art algorithm that does not exploit the tensor-product structure show that, on CPUs, the additional costs of cells with hanging-node constraints can be reduced by a factor of 5-10 for a Laplace operator evaluation with high-order elements (k = 3) and affine meshes. For non-affine meshes, the costs for the application of hanging-node constraints can be completely hidden behind the memory transfer. The algorithm has been integrated into the open-source finite-element library deal.II.

This paper describes in detail the implementation of a finite element technique for solving the compressible Navier–Stokes equations that is provably robust and demonstrates excellent performance on modern computer hardware. The method is second-order accurate in time and space. Robustness here means that the method is proved to be invariant domain preserving under the hyperbolic CFL time step restriction, and the method delivers results that are reproducible. The proposed technique is shown to be accurate on challenging 2D and 3D realistic benchmarks.

This paper provides an overview of the new features of the finite element library deal.II, version 9.4.

We present a novel, highly scalable and optimized solver for turbulent flows based on high-order discontinuous Galerkin discretizations of the incompressible Navier-Stokes equations aimed to minimize time-to-solution. The solver uses explicit-implicit time integration with variable step size. The central algorithmic component is the matrix-free evaluation of discretized finite element operators. The node-level performance is optimized by sum-factorization kernels for tensor-product elements with unique algorithmic choices that reduce the number of arithmetic operations, improve cache usage, and vectorize the arithmetic work across elements and faces. These ingredients are integrated into a framework scalable to the massive parallelism of supercomputers by the use of optimal-complexity linear solvers, such as mixed-precision, hybrid geometric-polynomial-algebraic multigrid solvers for the pressure Poisson problem. The application problem under consideration are fluid dynamical simulations of the human respiratory system under mechanical ventilation conditions, using unstructured/structured adaptively refined meshes for geometrically complex domains typical of biomedical engineering.

This work presents the efficient, matrix-free finite-element library hyper deal for solving partial differential equations in two up to six dimensions with high-order discontinuous Galerkin methods. It builds upon the low-dimensional finite-element library deal. II to create complex low-dimensional meshes and to operate on them individually. These meshes are combined via a tensor product on the fly, and the library provides new special-purpose highly optimized matrix-free functions exploiting domain decomposition as well as shared memory via MPI-3.0 features. Both node-level performance analyses and strong/weak-scaling studies on up to 147,456 CPU cores confirm the efficiency of the implementation. Results obtained with the library hyper . deal are reported for high-dimensional advection problems and for the solution of the Vlasov-Poisson equation in up to six-dimensional phase space.

The well-known energy dissipation anomaly in the inviscid limit, related to velocity singularities according to Onsager, still needs to be demonstrated by numerical experiments. The present work contributes to this topic through high-resolution numerical simulations of the inviscid three-dimensional Taylor-Green vortex problem using a novel high-order discontinuous Galerkin discretisation approach for the incompressible Euler equations. The main methodological ingredient is the use of a discretisation scheme with inbuilt dissipation mechanisms, as opposed to discretely energy-conserving schemes, which - by construction - rule out the occurrence of anomalous dissipation. We investigate effective spatial resolution up to 8192(3) (defined based on the 2 pi-periodic box) and make the interesting phenomenological observation that the kinetic energy evolution does not tend towards exact energy conservation for increasing spatial resolution of the numerical scheme, but that the sequence of discrete solutions seemingly converges to a solution with non-zero kinetic energy dissipation rate. Taking the fine-resolution simulation as a reference, we measure grid-convergence with a relative L-2-error of 0.27% for the temporal evolution of the kinetic energy and 3.52% for the kinetic energy dissipation rate against the dissipative fine-resolution simulation. The present work raises the question of whether such results can be seen as a numerical confirmation of the famous energy dissipation anomaly. Due to the relation between anomalous energy dissipation and the occurrence of singularities for the incompressible Euler equations according to Onsager's conjecture, we elaborate on an indirect approach for the identification of finite-time singularities that relies on energy arguments.

This paper provides an overview of the new features of the finite element library deal . II, version 9.3.