1bcb2dfaeSJed Brown# Changes/Release Notes 2bcb2dfaeSJed Brown 3bcb2dfaeSJed BrownOn this page we provide a summary of the main API changes, new features and examples 4bcb2dfaeSJed Brownfor each release of libCEED. 5bcb2dfaeSJed Brown 6bcb2dfaeSJed Brown(main)= 7bcb2dfaeSJed Brown 8bcb2dfaeSJed Brown## Current `main` branch 9bcb2dfaeSJed Brown 10bcb2dfaeSJed Brown### Maintainability 11bcb2dfaeSJed Brown 12bcb2dfaeSJed Brown- Refactored preconditioner support internally to facilitate future development and improve GPU completeness/test coverage. 13bcb2dfaeSJed Brown 14bcb2dfaeSJed Brown(v0-9)= 15bcb2dfaeSJed Brown 16bcb2dfaeSJed Brown## v0.9 (Jul 6, 2021) 17bcb2dfaeSJed Brown 18bcb2dfaeSJed Brown### Interface changes 19bcb2dfaeSJed Brown 20bcb2dfaeSJed Brown- Minor modification in error handling macro to silence pedantic warnings when compiling with Clang, but no functional impact. 21bcb2dfaeSJed Brown 22bcb2dfaeSJed Brown### New features 23bcb2dfaeSJed Brown 24bcb2dfaeSJed Brown- Add {c:func}`CeedVectorAXPY` and {c:func}`CeedVectorPointwiseMult` as a convenience for stand-alone testing and internal use. 25bcb2dfaeSJed Brown- Add `CEED_QFUNCTION_HELPER` macro to properly annotate QFunction helper functions for code generation backends. 26bcb2dfaeSJed Brown- Add `CeedPragmaOptimizeOff` macro for code that is sensitive to floating point errors from fast math optimizations. 27bcb2dfaeSJed Brown- Rust support: split `libceed-sys` crate out of `libceed` and [publish both on crates.io](https://crates.io/crates/libceed). 28bcb2dfaeSJed Brown 29bcb2dfaeSJed Brown### Performance improvements 30bcb2dfaeSJed Brown 31bcb2dfaeSJed Brown### Examples 32bcb2dfaeSJed Brown 33bcb2dfaeSJed Brown- Solid mechanics mini-app updated to explore the performance impacts of various formulations in the initial and current configurations. 34bcb2dfaeSJed Brown- Fluid mechanics example adds GPU support and improves modularity. 35bcb2dfaeSJed Brown 36bcb2dfaeSJed Brown### Deprecated backends 37bcb2dfaeSJed Brown 38bcb2dfaeSJed Brown- The `/cpu/self/tmpl` and `/cpu/self/tmpl/sub` backends have been removed. These backends were intially added to test the backend inheritance mechanism, but this mechanism is now widely used and tested in multiple backends. 39bcb2dfaeSJed Brown 40bcb2dfaeSJed Brown(v0-8)= 41bcb2dfaeSJed Brown 42bcb2dfaeSJed Brown## v0.8 (Mar 31, 2021) 43bcb2dfaeSJed Brown 44bcb2dfaeSJed Brown### Interface changes 45bcb2dfaeSJed Brown 46bcb2dfaeSJed Brown- Error handling improved to include enumerated error codes for C interface return values. 47bcb2dfaeSJed Brown- Installed headers that will follow semantic versioning were moved to {code}`include/ceed` directory. These headers have been renamed from {code}`ceed-*.h` to {code}`ceed/*.h`. Placeholder headers with the old naming schema are currently provided, but these headers will be removed in the libCEED v0.9 release. 48bcb2dfaeSJed Brown 49bcb2dfaeSJed Brown### New features 50bcb2dfaeSJed Brown 51bcb2dfaeSJed Brown- Julia and Rust interfaces added, providing a nearly 1-1 correspondence with the C interface, plus some convenience features. 52bcb2dfaeSJed Brown- Static libraries can be built with `make STATIC=1` and the pkg-config file is installed accordingly. 53bcb2dfaeSJed Brown- Add {c:func}`CeedOperatorLinearAssembleSymbolic` and {c:func}`CeedOperatorLinearAssemble` to support full assembly of libCEED operators. 54bcb2dfaeSJed Brown 55bcb2dfaeSJed Brown### Performance improvements 56bcb2dfaeSJed Brown 57bcb2dfaeSJed Brown- New HIP MAGMA backends for hipMAGMA library users: `/gpu/hip/magma` and `/gpu/hip/magma/det`. 58bcb2dfaeSJed Brown- New HIP backends for improved tensor basis performance: `/gpu/hip/shared` and `/gpu/hip/gen`. 59bcb2dfaeSJed Brown 60bcb2dfaeSJed Brown### Examples 61bcb2dfaeSJed Brown 62bcb2dfaeSJed Brown- {ref}`example-petsc-elasticity` example updated with traction boundary conditions and improved Dirichlet boundary conditions. 63bcb2dfaeSJed Brown- {ref}`example-petsc-elasticity` example updated with Neo-Hookean hyperelasticity in current configuration as well as improved Neo-Hookean hyperelasticity exploring storage vs computation tradeoffs. 64bcb2dfaeSJed Brown- {ref}`example-petsc-navier-stokes` example updated with isentropic traveling vortex test case, an analytical solution to the Euler equations that is useful for testing boundary conditions, discretization stability, and order of accuracy. 65bcb2dfaeSJed Brown- {ref}`example-petsc-navier-stokes` example updated with support for performing convergence study and plotting order of convergence by polynomial degree. 66bcb2dfaeSJed Brown 67bcb2dfaeSJed Brown(v0-7)= 68bcb2dfaeSJed Brown 69bcb2dfaeSJed Brown## v0.7 (Sep 29, 2020) 70bcb2dfaeSJed Brown 71bcb2dfaeSJed Brown### Interface changes 72bcb2dfaeSJed Brown 73bcb2dfaeSJed Brown- Replace limited {code}`CeedInterlaceMode` with more flexible component stride {code}`compstride` in {code}`CeedElemRestriction` constructors. 74bcb2dfaeSJed Brown As a result, the {code}`indices` parameter has been replaced with {code}`offsets` and the {code}`nnodes` parameter has been replaced with {code}`lsize`. 75bcb2dfaeSJed Brown These changes improve support for mixed finite element methods. 76bcb2dfaeSJed Brown- Replace various uses of {code}`Ceed*Get*Status` with {code}`Ceed*Is*` in the backend API to match common nomenclature. 77bcb2dfaeSJed Brown- Replace {code}`CeedOperatorAssembleLinearDiagonal` with {c:func}`CeedOperatorLinearAssembleDiagonal` for clarity. 78bcb2dfaeSJed Brown- Linear Operators can be assembled as point-block diagonal matrices with {c:func}`CeedOperatorLinearAssemblePointBlockDiagonal`, provided in row-major form in a {code}`ncomp` by {code}`ncomp` block per node. 79bcb2dfaeSJed Brown- Diagonal assemble interface changed to accept a {ref}`CeedVector` instead of a pointer to a {ref}`CeedVector` to reduce memory movement when interfacing with calling code. 80bcb2dfaeSJed Brown- Added {c:func}`CeedOperatorLinearAssembleAddDiagonal` and {c:func}`CeedOperatorLinearAssembleAddPointBlockDiagonal` for improved future integration with codes such as MFEM that compose the action of {ref}`CeedOperator`s external to libCEED. 81bcb2dfaeSJed Brown- Added {c:func}`CeedVectorTakeAray` to sync and remove libCEED read/write access to an allocated array and pass ownership of the array to the caller. 82bcb2dfaeSJed Brown This function is recommended over {c:func}`CeedVectorSyncArray` when the {code}`CeedVector` has an array owned by the caller that was set by {c:func}`CeedVectorSetArray`. 83bcb2dfaeSJed Brown- Added {code}`CeedQFunctionContext` object to manage user QFunction context data and reduce copies between device and host memory. 84bcb2dfaeSJed Brown- Added {c:func}`CeedOperatorMultigridLevelCreate`, {c:func}`CeedOperatorMultigridLevelCreateTensorH1`, and {c:func}`CeedOperatorMultigridLevelCreateH1` to facilitate creation of multigrid prolongation, restriction, and coarse grid operators using a common quadrature space. 85bcb2dfaeSJed Brown 86bcb2dfaeSJed Brown### New features 87bcb2dfaeSJed Brown 88bcb2dfaeSJed Brown- New HIP backend: `/gpu/hip/ref`. 89bcb2dfaeSJed Brown- CeedQFunction support for user `CUfunction`s in some backends 90bcb2dfaeSJed Brown 91bcb2dfaeSJed Brown### Performance improvements 92bcb2dfaeSJed Brown 93bcb2dfaeSJed Brown- OCCA backend rebuilt to facilitate future performance enhancements. 94bcb2dfaeSJed Brown- Petsc BPs suite improved to reduce noise due to multiple calls to {code}`mpiexec`. 95bcb2dfaeSJed Brown 96bcb2dfaeSJed Brown### Examples 97bcb2dfaeSJed Brown 98bcb2dfaeSJed Brown- {ref}`example-petsc-elasticity` example updated with strain energy computation and more flexible boundary conditions. 99bcb2dfaeSJed Brown 100bcb2dfaeSJed Brown### Deprecated backends 101bcb2dfaeSJed Brown 102bcb2dfaeSJed Brown- The `/gpu/cuda/reg` backend has been removed, with its core features moved into `/gpu/cuda/ref` and `/gpu/cuda/shared`. 103bcb2dfaeSJed Brown 104bcb2dfaeSJed Brown(v0-6)= 105bcb2dfaeSJed Brown 106bcb2dfaeSJed Brown## v0.6 (Mar 29, 2020) 107bcb2dfaeSJed Brown 108bcb2dfaeSJed BrownlibCEED v0.6 contains numerous new features and examples, as well as expanded 109bcb2dfaeSJed Browndocumentation in [this new website](https://libceed.readthedocs.io). 110bcb2dfaeSJed Brown 111bcb2dfaeSJed Brown### New features 112bcb2dfaeSJed Brown 113bcb2dfaeSJed Brown- New Python interface using [CFFI](https://cffi.readthedocs.io/) provides a nearly 114bcb2dfaeSJed Brown 1-1 correspondence with the C interface, plus some convenience features. For instance, 115bcb2dfaeSJed Brown data stored in the {cpp:type}`CeedVector` structure are available without copy as 116bcb2dfaeSJed Brown {py:class}`numpy.ndarray`. Short tutorials are provided in 117bcb2dfaeSJed Brown [Binder](https://mybinder.org/v2/gh/CEED/libCEED/main?urlpath=lab/tree/examples/tutorials/). 118bcb2dfaeSJed Brown- Linear QFunctions can be assembled as block-diagonal matrices (per quadrature point, 119bcb2dfaeSJed Brown {c:func}`CeedOperatorAssembleLinearQFunction`) or to evaluate the diagonal 120bcb2dfaeSJed Brown ({c:func}`CeedOperatorAssembleLinearDiagonal`). These operations are useful for 121bcb2dfaeSJed Brown preconditioning ingredients and are used in the libCEED's multigrid examples. 122bcb2dfaeSJed Brown- The inverse of separable operators can be obtained using 123bcb2dfaeSJed Brown {c:func}`CeedOperatorCreateFDMElementInverse` and applied with 124bcb2dfaeSJed Brown {c:func}`CeedOperatorApply`. This is a useful preconditioning ingredient, 125bcb2dfaeSJed Brown especially for Laplacians and related operators. 126bcb2dfaeSJed Brown- New functions: {c:func}`CeedVectorNorm`, {c:func}`CeedOperatorApplyAdd`, 127bcb2dfaeSJed Brown {c:func}`CeedQFunctionView`, {c:func}`CeedOperatorView`. 128bcb2dfaeSJed Brown- Make public accessors for various attributes to facilitate writing composable code. 129bcb2dfaeSJed Brown- New backend: `/cpu/self/memcheck/serial`. 130bcb2dfaeSJed Brown- QFunctions using variable-length array (VLA) pointer constructs can be used with CUDA 131bcb2dfaeSJed Brown backends. (Single source is coming soon for OCCA backends.) 132bcb2dfaeSJed Brown- Fix some missing edge cases in CUDA backend. 133bcb2dfaeSJed Brown 134bcb2dfaeSJed Brown### Performance Improvements 135bcb2dfaeSJed Brown 136bcb2dfaeSJed Brown- MAGMA backend performance optimization and non-tensor bases. 137bcb2dfaeSJed Brown- No-copy optimization in {c:func}`CeedOperatorApply`. 138bcb2dfaeSJed Brown 139bcb2dfaeSJed Brown### Interface changes 140bcb2dfaeSJed Brown 141bcb2dfaeSJed Brown- Replace {code}`CeedElemRestrictionCreateIdentity` and 142bcb2dfaeSJed Brown {code}`CeedElemRestrictionCreateBlocked` with more flexible 143bcb2dfaeSJed Brown {c:func}`CeedElemRestrictionCreateStrided` and 144bcb2dfaeSJed Brown {c:func}`CeedElemRestrictionCreateBlockedStrided`. 145bcb2dfaeSJed Brown- Add arguments to {c:func}`CeedQFunctionCreateIdentity`. 146bcb2dfaeSJed Brown- Replace ambiguous uses of {cpp:enum}`CeedTransposeMode` for L-vector identification 147bcb2dfaeSJed Brown with {cpp:enum}`CeedInterlaceMode`. This is now an attribute of the 148bcb2dfaeSJed Brown {cpp:type}`CeedElemRestriction` (see {c:func}`CeedElemRestrictionCreate`) and no 149bcb2dfaeSJed Brown longer passed as `lmode` arguments to {c:func}`CeedOperatorSetField` and 150bcb2dfaeSJed Brown {c:func}`CeedElemRestrictionApply`. 151bcb2dfaeSJed Brown 152bcb2dfaeSJed Brown### Examples 153bcb2dfaeSJed Brown 154bcb2dfaeSJed BrownlibCEED-0.6 contains greatly expanded examples with {ref}`new documentation <Examples>`. 155bcb2dfaeSJed BrownNotable additions include: 156bcb2dfaeSJed Brown 157bcb2dfaeSJed Brown- Standalone {ref}`ex2-surface` ({file}`examples/ceed/ex2-surface`): compute the area of 158bcb2dfaeSJed Brown a domain in 1, 2, and 3 dimensions by applying a Laplacian. 159bcb2dfaeSJed Brown 160bcb2dfaeSJed Brown- PETSc {ref}`example-petsc-area` ({file}`examples/petsc/area.c`): computes surface area 161bcb2dfaeSJed Brown of domains (like the cube and sphere) by direct integration on a surface mesh; 162bcb2dfaeSJed Brown demonstrates geometric dimension different from topological dimension. 163bcb2dfaeSJed Brown 164bcb2dfaeSJed Brown- PETSc {ref}`example-petsc-bps`: 165bcb2dfaeSJed Brown 166bcb2dfaeSJed Brown - {file}`examples/petsc/bpsraw.c` (formerly `bps.c`): transparent CUDA support. 167bcb2dfaeSJed Brown - {file}`examples/petsc/bps.c` (formerly `bpsdmplex.c`): performance improvements 168bcb2dfaeSJed Brown and transparent CUDA support. 169bcb2dfaeSJed Brown - {ref}`example-petsc-bps-sphere` ({file}`examples/petsc/bpssphere.c`): 170bcb2dfaeSJed Brown generalizations of all CEED BPs to the surface of the sphere; demonstrates geometric 171bcb2dfaeSJed Brown dimension different from topological dimension. 172bcb2dfaeSJed Brown 173bcb2dfaeSJed Brown- {ref}`example-petsc-multigrid` ({file}`examples/petsc/multigrid.c`): new p-multigrid 174bcb2dfaeSJed Brown solver with algebraic multigrid coarse solve. 175bcb2dfaeSJed Brown 176bcb2dfaeSJed Brown- {ref}`example-petsc-navier-stokes` ({file}`examples/fluids/navierstokes.c`; formerly 177bcb2dfaeSJed Brown `examples/navier-stokes`): unstructured grid support (using PETSc's `DMPlex`), 178bcb2dfaeSJed Brown implicit time integration, SU/SUPG stabilization, free-slip boundary conditions, and 179bcb2dfaeSJed Brown quasi-2D computational domain support. 180bcb2dfaeSJed Brown 181bcb2dfaeSJed Brown- {ref}`example-petsc-elasticity` ({file}`examples/solids/elasticity.c`): new solver for 182bcb2dfaeSJed Brown linear elasticity, small-strain hyperelasticity, and globalized finite-strain 183bcb2dfaeSJed Brown hyperelasticity using p-multigrid with algebraic multigrid coarse solve. 184bcb2dfaeSJed Brown 185bcb2dfaeSJed Brown(v0-5)= 186bcb2dfaeSJed Brown 187bcb2dfaeSJed Brown## v0.5 (Sep 18, 2019) 188bcb2dfaeSJed Brown 189bcb2dfaeSJed BrownFor this release, several improvements were made. Two new CUDA backends were added to 190bcb2dfaeSJed Brownthe family of backends, of which, the new `cuda-gen` backend achieves state-of-the-art 191bcb2dfaeSJed Brownperformance using single-source {ref}`CeedQFunction`. From this release, users 192bcb2dfaeSJed Browncan define Q-Functions in a single source code independently of the targeted backend 193bcb2dfaeSJed Brownwith the aid of a new macro `CEED QFUNCTION` to support JIT (Just-In-Time) and CPU 194bcb2dfaeSJed Browncompilation of the user provided {ref}`CeedQFunction` code. To allow a unified 195bcb2dfaeSJed Browndeclaration, the {ref}`CeedQFunction` API has undergone a slight change: 196bcb2dfaeSJed Brownthe `QFunctionField` parameter `ncomp` has been changed to `size`. This change 197bcb2dfaeSJed Brownrequires setting the previous value of `ncomp` to `ncomp*dim` when adding a 198bcb2dfaeSJed Brown`QFunctionField` with eval mode `CEED EVAL GRAD`. 199bcb2dfaeSJed Brown 200bcb2dfaeSJed BrownAdditionally, new CPU backends 201bcb2dfaeSJed Brownwere included in this release, such as the `/cpu/self/opt/*` backends (which are 202bcb2dfaeSJed Brownwritten in pure C and use partial **E-vectors** to improve performance) and the 203bcb2dfaeSJed Brown`/cpu/self/ref/memcheck` backend (which relies upon the 204bcb2dfaeSJed Brown[Valgrind](http://valgrind.org/) Memcheck tool to help verify that user 205bcb2dfaeSJed Brown{ref}`CeedQFunction` have no undefined values). 206bcb2dfaeSJed BrownThis release also included various performance improvements, bug fixes, new examples, 207bcb2dfaeSJed Brownand improved tests. Among these improvements, vectorized instructions for 208bcb2dfaeSJed Brown{ref}`CeedQFunction` code compiled for CPU were enhanced by using `CeedPragmaSIMD` 209bcb2dfaeSJed Browninstead of `CeedPragmaOMP`, implementation of a {ref}`CeedQFunction` gallery and 210bcb2dfaeSJed Brownidentity Q-Functions were introduced, and the PETSc benchmark problems were expanded 211bcb2dfaeSJed Brownto include unstructured meshes handling were. For this expansion, the prior version of 212bcb2dfaeSJed Brownthe PETSc BPs, which only included data associated with structured geometries, were 213bcb2dfaeSJed Brownrenamed `bpsraw`, and the new version of the BPs, which can handle data associated 214bcb2dfaeSJed Brownwith any unstructured geometry, were called `bps`. Additionally, other benchmark 215bcb2dfaeSJed Brownproblems, namely BP2 and BP4 (the vector-valued versions of BP1 and BP3, respectively), 216bcb2dfaeSJed Brownand BP5 and BP6 (the collocated versions---for which the quadrature points are the same 217bcb2dfaeSJed Brownas the Gauss Lobatto nodes---of BP3 and BP4 respectively) were added to the PETSc 218bcb2dfaeSJed Brownexamples. Furthermoew, another standalone libCEED example, called `ex2`, which 219bcb2dfaeSJed Browncomputes the surface area of a given mesh was added to this release. 220bcb2dfaeSJed Brown 221bcb2dfaeSJed BrownBackends available in this release: 222bcb2dfaeSJed Brown 223*68e843eeSJed Brown| CEED resource (`-ceed`) | Backend | 224*68e843eeSJed Brown|--------------------------|-----------------------------------------------------| 225*68e843eeSJed Brown| `/cpu/self/ref/serial` | Serial reference implementation | 226*68e843eeSJed Brown| `/cpu/self/ref/blocked` | Blocked reference implementation | 227*68e843eeSJed Brown| `/cpu/self/ref/memcheck` | Memcheck backend, undefined value checks | 228*68e843eeSJed Brown| `/cpu/self/opt/serial` | Serial optimized C implementation | 229*68e843eeSJed Brown| `/cpu/self/opt/blocked` | Blocked optimized C implementation | 230*68e843eeSJed Brown| `/cpu/self/avx/serial` | Serial AVX implementation | 231*68e843eeSJed Brown| `/cpu/self/avx/blocked` | Blocked AVX implementation | 232*68e843eeSJed Brown| `/cpu/self/xsmm/serial` | Serial LIBXSMM implementation | 233*68e843eeSJed Brown| `/cpu/self/xsmm/blocked` | Blocked LIBXSMM implementation | 234*68e843eeSJed Brown| `/cpu/occa` | Serial OCCA kernels | 235*68e843eeSJed Brown| `/gpu/occa` | CUDA OCCA kernels | 236*68e843eeSJed Brown| `/omp/occa` | OpenMP OCCA kernels | 237*68e843eeSJed Brown| `/ocl/occa` | OpenCL OCCA kernels | 238*68e843eeSJed Brown| `/gpu/cuda/ref` | Reference pure CUDA kernels | 239*68e843eeSJed Brown| `/gpu/cuda/reg` | Pure CUDA kernels using one thread per element | 240*68e843eeSJed Brown| `/gpu/cuda/shared` | Optimized pure CUDA kernels using shared memory | 241*68e843eeSJed Brown| `/gpu/cuda/gen` | Optimized pure CUDA kernels using code generation | 242*68e843eeSJed Brown| `/gpu/magma` | CUDA MAGMA kernels | 243bcb2dfaeSJed Brown 244bcb2dfaeSJed BrownExamples available in this release: 245bcb2dfaeSJed Brown 246*68e843eeSJed Brown:::{list-table} 247*68e843eeSJed Brown:header-rows: 1 248*68e843eeSJed Brown:widths: auto 249*68e843eeSJed Brown* - User code 250*68e843eeSJed Brown - Example 251*68e843eeSJed Brown* - `ceed` 252*68e843eeSJed Brown - * ex1 (volume) 253*68e843eeSJed Brown * ex2 (surface) 254*68e843eeSJed Brown* - `mfem` 255*68e843eeSJed Brown - * BP1 (scalar mass operator) 256*68e843eeSJed Brown * BP3 (scalar Laplace operator) 257*68e843eeSJed Brown* - `petsc` 258*68e843eeSJed Brown - * BP1 (scalar mass operator) 259*68e843eeSJed Brown * BP2 (vector mass operator) 260*68e843eeSJed Brown * BP3 (scalar Laplace operator) 261*68e843eeSJed Brown * BP4 (vector Laplace operator) 262*68e843eeSJed Brown * BP5 (collocated scalar Laplace operator) 263*68e843eeSJed Brown * BP6 (collocated vector Laplace operator) 264*68e843eeSJed Brown * Navier-Stokes 265*68e843eeSJed Brown* - `nek5000` 266*68e843eeSJed Brown - * BP1 (scalar mass operator) 267*68e843eeSJed Brown * BP3 (scalar Laplace operator) 268*68e843eeSJed Brown::: 269bcb2dfaeSJed Brown 270bcb2dfaeSJed Brown(v0-4)= 271bcb2dfaeSJed Brown 272bcb2dfaeSJed Brown## v0.4 (Apr 1, 2019) 273bcb2dfaeSJed Brown 274bcb2dfaeSJed BrownlibCEED v0.4 was made again publicly available in the second full CEED software 275bcb2dfaeSJed Browndistribution, release CEED 2.0. This release contained notable features, such as 276bcb2dfaeSJed Brownfour new CPU backends, two new GPU backends, CPU backend optimizations, initial 277bcb2dfaeSJed Brownsupport for operator composition, performance benchmarking, and a Navier-Stokes demo. 278bcb2dfaeSJed BrownThe new CPU backends in this release came in two families. The `/cpu/self/*/serial` 279bcb2dfaeSJed Brownbackends process one element at a time and are intended for meshes with a smaller number 280bcb2dfaeSJed Brownof high order elements. The `/cpu/self/*/blocked` backends process blocked batches of 281bcb2dfaeSJed Browneight interlaced elements and are intended for meshes with higher numbers of elements. 282bcb2dfaeSJed BrownThe `/cpu/self/avx/*` backends rely upon AVX instructions to provide vectorized CPU 283bcb2dfaeSJed Brownperformance. The `/cpu/self/xsmm/*` backends rely upon the 284bcb2dfaeSJed Brown[LIBXSMM](http://github.com/hfp/libxsmm) package to provide vectorized CPU 285bcb2dfaeSJed Brownperformance. The `/gpu/cuda/*` backends provide GPU performance strictly using CUDA. 286bcb2dfaeSJed BrownThe `/gpu/cuda/ref` backend is a reference CUDA backend, providing reasonable 287bcb2dfaeSJed Brownperformance for most problem configurations. The `/gpu/cuda/reg` backend uses a simple 288bcb2dfaeSJed Brownparallelization approach, where each thread treats a finite element. Using just in time 289bcb2dfaeSJed Browncompilation, provided by nvrtc (NVidia Runtime Compiler), and runtime parameters, this 290bcb2dfaeSJed Brownbackend unroll loops and map memory address to registers. The `/gpu/cuda/reg` backend 291bcb2dfaeSJed Brownachieve good peak performance for 1D, 2D, and low order 3D problems, but performance 292bcb2dfaeSJed Browndeteriorates very quickly when threads run out of registers. 293bcb2dfaeSJed Brown 294bcb2dfaeSJed BrownA new explicit time-stepping Navier-Stokes solver was added to the family of libCEED 295bcb2dfaeSJed Brownexamples in the `examples/petsc` directory (see {ref}`example-petsc-navier-stokes`). 296bcb2dfaeSJed BrownThis example solves the time-dependent Navier-Stokes equations of compressible gas 297bcb2dfaeSJed Browndynamics in a static Eulerian three-dimensional frame, using structured high-order 298bcb2dfaeSJed Brownfinite/spectral element spatial discretizations and explicit high-order time-stepping 299bcb2dfaeSJed Brown(available in PETSc). Moreover, the Navier-Stokes example was developed using PETSc, 300bcb2dfaeSJed Brownso that the pointwise physics (defined at quadrature points) is separated from the 301bcb2dfaeSJed Brownparallelization and meshing concerns. 302bcb2dfaeSJed Brown 303bcb2dfaeSJed BrownBackends available in this release: 304bcb2dfaeSJed Brown 305*68e843eeSJed Brown| CEED resource (`-ceed`) | Backend | 306*68e843eeSJed Brown|--------------------------|-----------------------------------------------------| 307*68e843eeSJed Brown| `/cpu/self/ref/serial` | Serial reference implementation | 308*68e843eeSJed Brown| `/cpu/self/ref/blocked` | Blocked reference implementation | 309*68e843eeSJed Brown| `/cpu/self/tmpl` | Backend template, defaults to `/cpu/self/blocked` | 310*68e843eeSJed Brown| `/cpu/self/avx/serial` | Serial AVX implementation | 311*68e843eeSJed Brown| `/cpu/self/avx/blocked` | Blocked AVX implementation | 312*68e843eeSJed Brown| `/cpu/self/xsmm/serial` | Serial LIBXSMM implementation | 313*68e843eeSJed Brown| `/cpu/self/xsmm/blocked` | Blocked LIBXSMM implementation | 314*68e843eeSJed Brown| `/cpu/occa` | Serial OCCA kernels | 315*68e843eeSJed Brown| `/gpu/occa` | CUDA OCCA kernels | 316*68e843eeSJed Brown| `/omp/occa` | OpenMP OCCA kernels | 317*68e843eeSJed Brown| `/ocl/occa` | OpenCL OCCA kernels | 318*68e843eeSJed Brown| `/gpu/cuda/ref` | Reference pure CUDA kernels | 319*68e843eeSJed Brown| `/gpu/cuda/reg` | Pure CUDA kernels using one thread per element | 320*68e843eeSJed Brown| `/gpu/magma` | CUDA MAGMA kernels | 321bcb2dfaeSJed Brown 322bcb2dfaeSJed BrownExamples available in this release: 323bcb2dfaeSJed Brown 324*68e843eeSJed Brown:::{list-table} 325*68e843eeSJed Brown:header-rows: 1 326*68e843eeSJed Brown:widths: auto 327*68e843eeSJed Brown* - User code 328*68e843eeSJed Brown - Example 329*68e843eeSJed Brown* - `ceed` 330*68e843eeSJed Brown - * ex1 (volume) 331*68e843eeSJed Brown* - `mfem` 332*68e843eeSJed Brown - * BP1 (scalar mass operator) 333*68e843eeSJed Brown * BP3 (scalar Laplace operator) 334*68e843eeSJed Brown* - `petsc` 335*68e843eeSJed Brown - * BP1 (scalar mass operator) 336*68e843eeSJed Brown * BP3 (scalar Laplace operator) 337*68e843eeSJed Brown * Navier-Stokes 338*68e843eeSJed Brown* - `nek5000` 339*68e843eeSJed Brown - * BP1 (scalar mass operator) 340*68e843eeSJed Brown * BP3 (scalar Laplace operator) 341*68e843eeSJed Brown::: 342bcb2dfaeSJed Brown 343bcb2dfaeSJed Brown(v0-3)= 344bcb2dfaeSJed Brown 345bcb2dfaeSJed Brown## v0.3 (Sep 30, 2018) 346bcb2dfaeSJed Brown 347bcb2dfaeSJed BrownNotable features in this release include active/passive field interface, support for 348bcb2dfaeSJed Brownnon-tensor bases, backend optimization, and improved Fortran interface. This release 349bcb2dfaeSJed Brownalso focused on providing improved continuous integration, and many new tests with code 350bcb2dfaeSJed Browncoverage reports of about 90%. This release also provided a significant change to the 351bcb2dfaeSJed Brownpublic interface: a {ref}`CeedQFunction` can take any number of named input and output 352bcb2dfaeSJed Brownarguments while {ref}`CeedOperator` connects them to the actual data, which may be 353bcb2dfaeSJed Brownsupplied explicitly to `CeedOperatorApply()` (active) or separately via 354bcb2dfaeSJed Brown`CeedOperatorSetField()` (passive). This interface change enables reusable libraries 355bcb2dfaeSJed Brownof CeedQFunctions and composition of block solvers constructed using 356bcb2dfaeSJed Brown{ref}`CeedOperator`. A concept of blocked restriction was added to this release and 357bcb2dfaeSJed Brownused in an optimized CPU backend. Although this is typically not visible to the user, 358bcb2dfaeSJed Brownit enables effective use of arbitrary-length SIMD while maintaining cache locality. 359bcb2dfaeSJed BrownThis CPU backend also implements an algebraic factorization of tensor product gradients 360bcb2dfaeSJed Brownto perform fewer operations than standard application of interpolation and 361bcb2dfaeSJed Browndifferentiation from nodes to quadrature points. This algebraic formulation 362bcb2dfaeSJed Brownautomatically supports non-polynomial and non-interpolatory bases, thus is more general 363bcb2dfaeSJed Brownthan the more common derivation in terms of Lagrange polynomials on the quadrature points. 364bcb2dfaeSJed Brown 365bcb2dfaeSJed BrownBackends available in this release: 366bcb2dfaeSJed Brown 367*68e843eeSJed Brown| CEED resource (`-ceed`) | Backend | 368*68e843eeSJed Brown|-------------------------|-----------------------------------------------------| 369*68e843eeSJed Brown| `/cpu/self/blocked` | Blocked reference implementation | 370*68e843eeSJed Brown| `/cpu/self/ref` | Serial reference implementation | 371*68e843eeSJed Brown| `/cpu/self/tmpl` | Backend template, defaults to `/cpu/self/blocked` | 372*68e843eeSJed Brown| `/cpu/occa` | Serial OCCA kernels | 373*68e843eeSJed Brown| `/gpu/occa` | CUDA OCCA kernels | 374*68e843eeSJed Brown| `/omp/occa` | OpenMP OCCA kernels | 375*68e843eeSJed Brown| `/ocl/occa` | OpenCL OCCA kernels | 376*68e843eeSJed Brown| `/gpu/magma` | CUDA MAGMA kernels | 377bcb2dfaeSJed Brown 378bcb2dfaeSJed BrownExamples available in this release: 379bcb2dfaeSJed Brown 380*68e843eeSJed Brown:::{list-table} 381*68e843eeSJed Brown:header-rows: 1 382*68e843eeSJed Brown:widths: auto 383*68e843eeSJed Brown* - User code 384*68e843eeSJed Brown - Example 385*68e843eeSJed Brown* - `ceed` 386*68e843eeSJed Brown - * ex1 (volume) 387*68e843eeSJed Brown* - `mfem` 388*68e843eeSJed Brown - * BP1 (scalar mass operator) 389*68e843eeSJed Brown * BP3 (scalar Laplace operator) 390*68e843eeSJed Brown* - `petsc` 391*68e843eeSJed Brown - * BP1 (scalar mass operator) 392*68e843eeSJed Brown * BP3 (scalar Laplace operator) 393*68e843eeSJed Brown* - `nek5000` 394*68e843eeSJed Brown - * BP1 (scalar mass operator) 395*68e843eeSJed Brown * BP3 (scalar Laplace operator) 396*68e843eeSJed Brown::: 397bcb2dfaeSJed Brown 398bcb2dfaeSJed Brown(v0-21)= 399bcb2dfaeSJed Brown 400bcb2dfaeSJed Brown## v0.21 (Sep 30, 2018) 401bcb2dfaeSJed Brown 402bcb2dfaeSJed BrownA MAGMA backend (which relies upon the 403bcb2dfaeSJed Brown[MAGMA](https://bitbucket.org/icl/magma) package) was integrated in libCEED for this 404bcb2dfaeSJed Brownrelease. This initial integration set up the framework of using MAGMA and provided the 405bcb2dfaeSJed BrownlibCEED functionality through MAGMA kernels as one of libCEED’s computational backends. 406bcb2dfaeSJed BrownAs any other backend, the MAGMA backend provides extended basic data structures for 407bcb2dfaeSJed Brown{ref}`CeedVector`, {ref}`CeedElemRestriction`, and {ref}`CeedOperator`, and implements 408bcb2dfaeSJed Brownthe fundamental CEED building blocks to work with the new data structures. 409bcb2dfaeSJed BrownIn general, the MAGMA-specific data structures keep the libCEED pointers to CPU data 410bcb2dfaeSJed Brownbut also add corresponding device (e.g., GPU) pointers to the data. Coherency is handled 411bcb2dfaeSJed Browninternally, and thus seamlessly to the user, through the functions/methods that are 412bcb2dfaeSJed Brownprovided to support them. 413bcb2dfaeSJed Brown 414bcb2dfaeSJed BrownBackends available in this release: 415bcb2dfaeSJed Brown 416*68e843eeSJed Brown| CEED resource (`-ceed`) | Backend | 417*68e843eeSJed Brown|-------------------------|---------------------------------| 418*68e843eeSJed Brown| `/cpu/self` | Serial reference implementation | 419*68e843eeSJed Brown| `/cpu/occa` | Serial OCCA kernels | 420*68e843eeSJed Brown| `/gpu/occa` | CUDA OCCA kernels | 421*68e843eeSJed Brown| `/omp/occa` | OpenMP OCCA kernels | 422*68e843eeSJed Brown| `/ocl/occa` | OpenCL OCCA kernels | 423*68e843eeSJed Brown| `/gpu/magma` | CUDA MAGMA kernels | 424bcb2dfaeSJed Brown 425bcb2dfaeSJed BrownExamples available in this release: 426bcb2dfaeSJed Brown 427*68e843eeSJed Brown:::{list-table} 428*68e843eeSJed Brown:header-rows: 1 429*68e843eeSJed Brown:widths: auto 430*68e843eeSJed Brown* - User code 431*68e843eeSJed Brown - Example 432*68e843eeSJed Brown* - `ceed` 433*68e843eeSJed Brown - * ex1 (volume) 434*68e843eeSJed Brown* - `mfem` 435*68e843eeSJed Brown - * BP1 (scalar mass operator) 436*68e843eeSJed Brown * BP3 (scalar Laplace operator) 437*68e843eeSJed Brown* - `petsc` 438*68e843eeSJed Brown - * BP1 (scalar mass operator) 439*68e843eeSJed Brown* - `nek5000` 440*68e843eeSJed Brown - * BP1 (scalar mass operator) 441*68e843eeSJed Brown::: 442bcb2dfaeSJed Brown 443bcb2dfaeSJed Brown(v0-2)= 444bcb2dfaeSJed Brown 445bcb2dfaeSJed Brown## v0.2 (Mar 30, 2018) 446bcb2dfaeSJed Brown 447bcb2dfaeSJed BrownlibCEED was made publicly available the first full CEED software distribution, release 448bcb2dfaeSJed BrownCEED 1.0. The distribution was made available using the Spack package manager to provide 449bcb2dfaeSJed Browna common, easy-to-use build environment, where the user can build the CEED distribution 450bcb2dfaeSJed Brownwith all dependencies. This release included a new Fortran interface for the library. 451bcb2dfaeSJed BrownThis release also contained major improvements in the OCCA backend (including a new 452bcb2dfaeSJed Brown`/ocl/occa` backend) and new examples. The standalone libCEED example was modified to 453bcb2dfaeSJed Browncompute the volume volume of a given mesh (in 1D, 2D, or 3D) and placed in an 454bcb2dfaeSJed Brown`examples/ceed` subfolder. A new `mfem` example to perform BP3 (with the application 455bcb2dfaeSJed Brownof the Laplace operator) was also added to this release. 456bcb2dfaeSJed Brown 457bcb2dfaeSJed BrownBackends available in this release: 458bcb2dfaeSJed Brown 459*68e843eeSJed Brown| CEED resource (`-ceed`) | Backend | 460*68e843eeSJed Brown|-------------------------|---------------------------------| 461*68e843eeSJed Brown| `/cpu/self` | Serial reference implementation | 462*68e843eeSJed Brown| `/cpu/occa` | Serial OCCA kernels | 463*68e843eeSJed Brown| `/gpu/occa` | CUDA OCCA kernels | 464*68e843eeSJed Brown| `/omp/occa` | OpenMP OCCA kernels | 465*68e843eeSJed Brown| `/ocl/occa` | OpenCL OCCA kernels | 466bcb2dfaeSJed Brown 467bcb2dfaeSJed BrownExamples available in this release: 468bcb2dfaeSJed Brown 469*68e843eeSJed Brown:::{list-table} 470*68e843eeSJed Brown:header-rows: 1 471*68e843eeSJed Brown:widths: auto 472*68e843eeSJed Brown* - User code 473*68e843eeSJed Brown - Example 474*68e843eeSJed Brown* - `ceed` 475*68e843eeSJed Brown - * ex1 (volume) 476*68e843eeSJed Brown* - `mfem` 477*68e843eeSJed Brown - * BP1 (scalar mass operator) 478*68e843eeSJed Brown * BP3 (scalar Laplace operator) 479*68e843eeSJed Brown* - `petsc` 480*68e843eeSJed Brown - * BP1 (scalar mass operator) 481*68e843eeSJed Brown* - `nek5000` 482*68e843eeSJed Brown - * BP1 (scalar mass operator) 483*68e843eeSJed Brown::: 484bcb2dfaeSJed Brown 485bcb2dfaeSJed Brown(v0-1)= 486bcb2dfaeSJed Brown 487bcb2dfaeSJed Brown## v0.1 (Jan 3, 2018) 488bcb2dfaeSJed Brown 489bcb2dfaeSJed BrownInitial low-level API of the CEED project. The low-level API provides a set of Finite 490bcb2dfaeSJed BrownElements kernels and components for writing new low-level kernels. Examples include: 491bcb2dfaeSJed Brownvector and sparse linear algebra, element matrix assembly over a batch of elements, 492bcb2dfaeSJed Brownpartial assembly and action for efficient high-order operators like mass, diffusion, 493bcb2dfaeSJed Brownadvection, etc. The main goal of the low-level API is to establish the basis for the 494bcb2dfaeSJed Brownhigh-level API. Also, identifying such low-level kernels and providing a reference 495bcb2dfaeSJed Brownimplementation for them serves as the basis for specialized backend implementations. 496bcb2dfaeSJed BrownThis release contained several backends: `/cpu/self`, and backends which rely upon the 497bcb2dfaeSJed Brown[OCCA](http://github.com/libocca/occa) package, such as `/cpu/occa`, 498bcb2dfaeSJed Brown`/gpu/occa`, and `/omp/occa`. 499bcb2dfaeSJed BrownIt also included several examples, in the `examples` folder: 500bcb2dfaeSJed BrownA standalone code that shows the usage of libCEED (with no external 501bcb2dfaeSJed Browndependencies) to apply the Laplace operator, `ex1`; an `mfem` example to perform BP1 502bcb2dfaeSJed Brown(with the application of the mass operator); and a `petsc` example to perform BP1 503bcb2dfaeSJed Brown(with the application of the mass operator). 504bcb2dfaeSJed Brown 505bcb2dfaeSJed BrownBackends available in this release: 506bcb2dfaeSJed Brown 507*68e843eeSJed Brown| CEED resource (`-ceed`) | Backend | 508*68e843eeSJed Brown|-------------------------|---------------------------------| 509*68e843eeSJed Brown| `/cpu/self` | Serial reference implementation | 510*68e843eeSJed Brown| `/cpu/occa` | Serial OCCA kernels | 511*68e843eeSJed Brown| `/gpu/occa` | CUDA OCCA kernels | 512*68e843eeSJed Brown| `/omp/occa` | OpenMP OCCA kernels | 513bcb2dfaeSJed Brown 514bcb2dfaeSJed BrownExamples available in this release: 515bcb2dfaeSJed Brown 516bcb2dfaeSJed Brown| User code | Example | 517*68e843eeSJed Brown|-----------------------|-----------------------------------| 518*68e843eeSJed Brown| `ceed` | ex1 (scalar Laplace operator) | 519*68e843eeSJed Brown| `mfem` | BP1 (scalar mass operator) | 520*68e843eeSJed Brown| `petsc` | BP1 (scalar mass operator) | 521bcb2dfaeSJed Brown``` 522