1# Changes/Release Notes 2 3On this page we provide a summary of the main API changes, new features and examples for each release of libCEED. 4 5(main)= 6 7## Current `main` branch 8 9### Interface changes 10 11- Added {c:func}`CeedOperatorSetName` for more readable {c:func}`CeedOperatorView` output. 12- Added {c:func}`CeedBasisCreateProjection` to facilitate interpolation between nodes for separate `CeedBases`. 13 14### New features 15 16- Update `/cpu/self/memcheck/*` backends to help verify `CeedQFunctionContext` data sizes provided by user. 17- Added `CeedInt_FMT` to support potential future use of larger interger sizes. 18- Added CEED_QFUNCTION_ATTR for setting compiler attributes/pragmas to CEED_QFUNCTION_HELPER and CEED_QFUNCTION 19- OCCA backend updated to latest OCCA release; DPC++ and OMP OCCA modes enabled. 20Due to a limitation of the OCCA parser, typedefs are required to use pointers to arrays in QFunctions with the OCCA backend. 21This issue will be fixed in a future OCCA release. 22 23### Bugfix 24 25- Fix bug in setting device id for GPU backends. 26- Fix storing of indices for `CeedElemRestriction` on the host with GPU backends. 27- Fix `CeedElemRestriction` sizing for {c:func}`CeedOperatorAssemblePointBlockDiagonal`. 28- Fix bugs in CPU implementation of {c:func}`CeedOperatorLinearAssemble` when there are different number of active input modes and active output modes. 29 30### Examples 31 32- Added various performance enhancements for {ref}`example-petsc-navier-stokes`. 33- Refactored {ref}`example-petsc-navier-stokes` to improve code reuse. 34- Added Shock Tube, Channel, and Flat Plate boundary layer problems to {ref}`example-petsc-navier-stokes`. 35- Added ability to use QFunctions for strong STG inflow in {ref}`example-petsc-navier-stokes`. 36 37### Maintainability 38 39- Refactored `/gpu/cuda/shared` and `/gpu/cuda/gen` as well as `/gpu/hip/shared` and `/gpu/hip/gen` backend to improve maintainablity and reduce duplicated code. 40- Enabled support for `p > 8` for `/gpu/*/shared` backends. 41 42(v0-10-1)= 43 44## v0.10.1 (Apr 11, 2022) 45 46### Interface changes 47 48- Added {c:func}`CeedQFunctionSetUserFlopsEstimate` and {c:func}`CeedOperatorGetFlopsEstimate` to facilitate estimating FLOPs in operator application. 49 50### New features 51 52- Switched MAGMA backends to use runtime compilation for tensor basis kernels (and element restriction kernels, in non-deterministic `/gpu/*/magma` backends). 53This reduces time to compile the library and increases the range of parameters for which the MAGMA tensor basis kernels will work. 54 55### Bugfix 56 57- Install JiT source files in install directory to fix GPU functionality for installed libCEED. 58 59(v0-10)= 60 61## v0.10 (Mar 21, 2022) 62 63### Interface changes 64 65- Update {c:func}`CeedQFunctionGetFields` and {c:func}`CeedOperatorGetFields` to include number of fields. 66- Promote to the public API: QFunction and Operator field objects, `CeedQFunctionField` and `CeedOperatorField`, and associated getters, {c:func}`CeedQFunctionGetFields`; {c:func}`CeedQFunctionFieldGetName`; {c:func}`CeedQFunctionFieldGetSize`; {c:func}`CeedQFunctionFieldGetEvalMode`; {c:func}`CeedOperatorGetFields`; {c:func}`CeedOperatorFieldGetElemRestriction`; {c:func}`CeedOperatorFieldGetBasis`; and {c:func}`CeedOperatorFieldGetVector`. 67- Clarify and document conditions where `CeedQFunction` and `CeedOperator` become immutable and no further fields or suboperators can be added. 68- Add {c:func}`CeedOperatorLinearAssembleQFunctionBuildOrUpdate` to reduce object creation overhead in assembly of CeedOperator preconditioning ingredients. 69- Promote {c:func}`CeedOperatorCheckReady`to the public API to facilitate interactive interfaces. 70- Warning added when compiling OCCA backend to alert users that this backend is experimental. 71- `ceed-backend.h`, `ceed-hash.h`, and `ceed-khash.h` removed. Users should use `ceed/backend.h`, `ceed/hash.h`, and `ceed/khash.h`. 72- Added {c:func}`CeedQFunctionGetKernelName`; refactored {c:func}`CeedQFunctionGetSourcePath` to exclude function kernel name. 73- Clarify documentation for {c:func}`CeedVectorTakeArray`; this function will error if {c:func}`CeedVectorSetArray` with `copy_mode == CEED_USE_POINTER` was not previously called for the corresponding `CeedMemType`. 74- Added {c:func}`CeedVectorGetArrayWrite` that allows access to uninitalized arrays; require initalized data for {c:func}`CeedVectorGetArray`. 75- Added {c:func}`CeedQFunctionContextRegisterDouble` and {c:func}`CeedQFunctionContextRegisterInt32` with {c:func}`CeedQFunctionContextSetDouble` and {c:func}`CeedQFunctionContextSetInt32` to facilitate easy updating of {c:struct}`CeedQFunctionContext` data by user defined field names. 76- Added {c:func}`CeedQFunctionContextGetFieldDescriptions` to retreive user defined descriptions of fields that are registered with `CeedQFunctionContextRegister*`. 77- Renamed `CeedElemTopology` entries for clearer namespacing between libCEED enums. 78- Added type `CeedSize` equivalent to `ptrdiff_t` for array sizes in {c:func}`CeedVectorCreate`, {c:func}`CeedVectorGetLength`, `CeedElemRestrictionCreate*`, {c:func}`CeedElemRestrictionGetLVectorSize`, and {c:func}`CeedOperatorLinearAssembleSymbolic`. This is a breaking change. 79- Added {c:func}`CeedOperatorSetQFunctionUpdated` to facilitate QFunction data re-use between operators sharing the same quadrature space, such as in a multigrid hierarchy. 80- Added {c:func}`CeedOperatorGetActiveVectorLengths` to get shape of CeedOperator. 81 82### New features 83 84- `CeedScalar` can now be set as `float` or `double` at compile time. 85- Added JiT utilities in `ceed/jit-tools.h` to reduce duplicated code in GPU backends. 86- Added support for JiT of QFunctions with `#include "relative/path/local-file.h"` statements for additional local files. Note that files included with `""` are searched relative to the current file first, then by compiler paths (as with `<>` includes). To use this feature, one should adhere to relative paths only, not compiler flags like `-I`, which the JiT will not be aware of. 87- Remove need to guard library headers in QFunction source for code generation backends. 88- `CeedDebugEnv()` macro created to provide debugging outputs when Ceed context is not present. 89- Added {c:func}`CeedStringAllocCopy` to reduce repeated code for copying strings internally. 90- Added {c:func}`CeedPathConcatenate` to facilitate loading kernel source files with a path relative to the current file. 91- Added support for non-tensor H(div) elements, to include CPU backend implementations and {c:func}`CeedBasisCreateHdiv` convenience constructor. 92- Added {c:func}`CeedQFunctionSetContextWritable` and read-only access to `CeedQFunctionContext` data as an optional feature to improve GPU performance. By default, calling the `CeedQFunctionUser` during {c:func}`CeedQFunctionApply` is assumed to write into the `CeedQFunctionContext` data, consistent with the previous behavior. Note that if a user asserts that their `CeedQFunctionUser` does not write into the `CeedQFunctionContext` data, they are responsible for the validity of this assertion. 93- Added support for element matrix assembly in GPU backends. 94 95### Maintainability 96 97- Refactored preconditioner support internally to facilitate future development and improve GPU completeness/test coverage. 98- `Include-what-you-use` makefile target added as `make iwyu`. 99- Create backend constant `CEED_FIELD_MAX` to reduce magic numbers in codebase. 100- Put GPU JiTed kernel source code into separate files. 101- Dropped legacy version support in PETSc based examples to better utilize PETSc DMPlex and Mat updates to support libCEED; current minimum PETSc version for the examples is v3.17. 102 103(v0-9)= 104 105## v0.9 (Jul 6, 2021) 106 107### Interface changes 108 109- Minor modification in error handling macro to silence pedantic warnings when compiling with Clang, but no functional impact. 110 111### New features 112 113- Add {c:func}`CeedVectorAXPY` and {c:func}`CeedVectorPointwiseMult` as a convenience for stand-alone testing and internal use. 114- Add `CEED_QFUNCTION_HELPER` macro to properly annotate QFunction helper functions for code generation backends. 115- Add `CeedPragmaOptimizeOff` macro for code that is sensitive to floating point errors from fast math optimizations. 116- Rust support: split `libceed-sys` crate out of `libceed` and [publish both on crates.io](https://crates.io/crates/libceed). 117 118### Performance improvements 119 120### Examples 121 122- Solid mechanics mini-app updated to explore the performance impacts of various formulations in the initial and current configurations. 123- Fluid mechanics example adds GPU support and improves modularity. 124 125### Deprecated backends 126 127- The `/cpu/self/tmpl` and `/cpu/self/tmpl/sub` backends have been removed. These backends were intially added to test the backend inheritance mechanism, but this mechanism is now widely used and tested in multiple backends. 128 129(v0-8)= 130 131## v0.8 (Mar 31, 2021) 132 133### Interface changes 134 135- Error handling improved to include enumerated error codes for C interface return values. 136- Installed headers that will follow semantic versioning were moved to {code}`include/ceed` directory. These headers have been renamed from {code}`ceed-*.h` to {code}`ceed/*.h`. Placeholder headers with the old naming schema are currently provided, but these headers will be removed in the libCEED v0.9 release. 137 138### New features 139 140- Julia and Rust interfaces added, providing a nearly 1-1 correspondence with the C interface, plus some convenience features. 141- Static libraries can be built with `make STATIC=1` and the pkg-config file is installed accordingly. 142- Add {c:func}`CeedOperatorLinearAssembleSymbolic` and {c:func}`CeedOperatorLinearAssemble` to support full assembly of libCEED operators. 143 144### Performance improvements 145 146- New HIP MAGMA backends for hipMAGMA library users: `/gpu/hip/magma` and `/gpu/hip/magma/det`. 147- New HIP backends for improved tensor basis performance: `/gpu/hip/shared` and `/gpu/hip/gen`. 148 149### Examples 150 151- {ref}`example-petsc-elasticity` example updated with traction boundary conditions and improved Dirichlet boundary conditions. 152- {ref}`example-petsc-elasticity` example updated with Neo-Hookean hyperelasticity in current configuration as well as improved Neo-Hookean hyperelasticity exploring storage vs computation tradeoffs. 153- {ref}`example-petsc-navier-stokes` example updated with isentropic traveling vortex test case, an analytical solution to the Euler equations that is useful for testing boundary conditions, discretization stability, and order of accuracy. 154- {ref}`example-petsc-navier-stokes` example updated with support for performing convergence study and plotting order of convergence by polynomial degree. 155 156(v0-7)= 157 158## v0.7 (Sep 29, 2020) 159 160### Interface changes 161 162- Replace limited {code}`CeedInterlaceMode` with more flexible component stride {code}`compstride` in {code}`CeedElemRestriction` constructors. 163 As a result, the {code}`indices` parameter has been replaced with {code}`offsets` and the {code}`nnodes` parameter has been replaced with {code}`lsize`. 164 These changes improve support for mixed finite element methods. 165- Replace various uses of {code}`Ceed*Get*Status` with {code}`Ceed*Is*` in the backend API to match common nomenclature. 166- Replace {code}`CeedOperatorAssembleLinearDiagonal` with {c:func}`CeedOperatorLinearAssembleDiagonal` for clarity. 167- Linear Operators can be assembled as point-block diagonal matrices with {c:func}`CeedOperatorLinearAssemblePointBlockDiagonal`, provided in row-major form in a {code}`ncomp` by {code}`ncomp` block per node. 168- Diagonal assemble interface changed to accept a {ref}`CeedVector` instead of a pointer to a {ref}`CeedVector` to reduce memory movement when interfacing with calling code. 169- Added {c:func}`CeedOperatorLinearAssembleAddDiagonal` and {c:func}`CeedOperatorLinearAssembleAddPointBlockDiagonal` for improved future integration with codes such as MFEM that compose the action of {ref}`CeedOperator`s external to libCEED. 170- Added {c:func}`CeedVectorTakeAray` to sync and remove libCEED read/write access to an allocated array and pass ownership of the array to the caller. 171 This function is recommended over {c:func}`CeedVectorSyncArray` when the {code}`CeedVector` has an array owned by the caller that was set by {c:func}`CeedVectorSetArray`. 172- Added {code}`CeedQFunctionContext` object to manage user QFunction context data and reduce copies between device and host memory. 173- Added {c:func}`CeedOperatorMultigridLevelCreate`, {c:func}`CeedOperatorMultigridLevelCreateTensorH1`, and {c:func}`CeedOperatorMultigridLevelCreateH1` to facilitate creation of multigrid prolongation, restriction, and coarse grid operators using a common quadrature space. 174 175### New features 176 177- New HIP backend: `/gpu/hip/ref`. 178- CeedQFunction support for user `CUfunction`s in some backends 179 180### Performance improvements 181 182- OCCA backend rebuilt to facilitate future performance enhancements. 183- Petsc BPs suite improved to reduce noise due to multiple calls to {code}`mpiexec`. 184 185### Examples 186 187- {ref}`example-petsc-elasticity` example updated with strain energy computation and more flexible boundary conditions. 188 189### Deprecated backends 190 191- The `/gpu/cuda/reg` backend has been removed, with its core features moved into `/gpu/cuda/ref` and `/gpu/cuda/shared`. 192 193(v0-6)= 194 195## v0.6 (Mar 29, 2020) 196 197libCEED v0.6 contains numerous new features and examples, as well as expanded 198documentation in [this new website](https://libceed.org). 199 200### New features 201 202- New Python interface using [CFFI](https://cffi.readthedocs.io/) provides a nearly 203 1-1 correspondence with the C interface, plus some convenience features. For instance, 204 data stored in the {cpp:type}`CeedVector` structure are available without copy as 205 {py:class}`numpy.ndarray`. Short tutorials are provided in 206 [Binder](https://mybinder.org/v2/gh/CEED/libCEED/main?urlpath=lab/tree/examples/tutorials/). 207- Linear QFunctions can be assembled as block-diagonal matrices (per quadrature point, 208 {c:func}`CeedOperatorAssembleLinearQFunction`) or to evaluate the diagonal 209 ({c:func}`CeedOperatorAssembleLinearDiagonal`). These operations are useful for 210 preconditioning ingredients and are used in the libCEED's multigrid examples. 211- The inverse of separable operators can be obtained using 212 {c:func}`CeedOperatorCreateFDMElementInverse` and applied with 213 {c:func}`CeedOperatorApply`. This is a useful preconditioning ingredient, 214 especially for Laplacians and related operators. 215- New functions: {c:func}`CeedVectorNorm`, {c:func}`CeedOperatorApplyAdd`, 216 {c:func}`CeedQFunctionView`, {c:func}`CeedOperatorView`. 217- Make public accessors for various attributes to facilitate writing composable code. 218- New backend: `/cpu/self/memcheck/serial`. 219- QFunctions using variable-length array (VLA) pointer constructs can be used with CUDA 220 backends. (Single source is coming soon for OCCA backends.) 221- Fix some missing edge cases in CUDA backend. 222 223### Performance Improvements 224 225- MAGMA backend performance optimization and non-tensor bases. 226- No-copy optimization in {c:func}`CeedOperatorApply`. 227 228### Interface changes 229 230- Replace {code}`CeedElemRestrictionCreateIdentity` and 231 {code}`CeedElemRestrictionCreateBlocked` with more flexible 232 {c:func}`CeedElemRestrictionCreateStrided` and 233 {c:func}`CeedElemRestrictionCreateBlockedStrided`. 234- Add arguments to {c:func}`CeedQFunctionCreateIdentity`. 235- Replace ambiguous uses of {cpp:enum}`CeedTransposeMode` for L-vector identification 236 with {cpp:enum}`CeedInterlaceMode`. This is now an attribute of the 237 {cpp:type}`CeedElemRestriction` (see {c:func}`CeedElemRestrictionCreate`) and no 238 longer passed as `lmode` arguments to {c:func}`CeedOperatorSetField` and 239 {c:func}`CeedElemRestrictionApply`. 240 241### Examples 242 243libCEED-0.6 contains greatly expanded examples with {ref}`new documentation <Examples>`. 244Notable additions include: 245 246- Standalone {ref}`ex2-surface` ({file}`examples/ceed/ex2-surface`): compute the area of 247 a domain in 1, 2, and 3 dimensions by applying a Laplacian. 248 249- PETSc {ref}`example-petsc-area` ({file}`examples/petsc/area.c`): computes surface area 250 of domains (like the cube and sphere) by direct integration on a surface mesh; 251 demonstrates geometric dimension different from topological dimension. 252 253- PETSc {ref}`example-petsc-bps`: 254 255 - {file}`examples/petsc/bpsraw.c` (formerly `bps.c`): transparent CUDA support. 256 - {file}`examples/petsc/bps.c` (formerly `bpsdmplex.c`): performance improvements 257 and transparent CUDA support. 258 - {ref}`example-petsc-bps-sphere` ({file}`examples/petsc/bpssphere.c`): 259 generalizations of all CEED BPs to the surface of the sphere; demonstrates geometric 260 dimension different from topological dimension. 261 262- {ref}`example-petsc-multigrid` ({file}`examples/petsc/multigrid.c`): new p-multigrid 263 solver with algebraic multigrid coarse solve. 264 265- {ref}`example-petsc-navier-stokes` ({file}`examples/fluids/navierstokes.c`; formerly 266 `examples/navier-stokes`): unstructured grid support (using PETSc's `DMPlex`), 267 implicit time integration, SU/SUPG stabilization, free-slip boundary conditions, and 268 quasi-2D computational domain support. 269 270- {ref}`example-petsc-elasticity` ({file}`examples/solids/elasticity.c`): new solver for 271 linear elasticity, small-strain hyperelasticity, and globalized finite-strain 272 hyperelasticity using p-multigrid with algebraic multigrid coarse solve. 273 274(v0-5)= 275 276## v0.5 (Sep 18, 2019) 277 278For this release, several improvements were made. Two new CUDA backends were added to 279the family of backends, of which, the new `cuda-gen` backend achieves state-of-the-art 280performance using single-source {ref}`CeedQFunction`. From this release, users 281can define Q-Functions in a single source code independently of the targeted backend 282with the aid of a new macro `CEED QFUNCTION` to support JIT (Just-In-Time) and CPU 283compilation of the user provided {ref}`CeedQFunction` code. To allow a unified 284declaration, the {ref}`CeedQFunction` API has undergone a slight change: 285the `QFunctionField` parameter `ncomp` has been changed to `size`. This change 286requires setting the previous value of `ncomp` to `ncomp*dim` when adding a 287`QFunctionField` with eval mode `CEED EVAL GRAD`. 288 289Additionally, new CPU backends 290were included in this release, such as the `/cpu/self/opt/*` backends (which are 291written in pure C and use partial **E-vectors** to improve performance) and the 292`/cpu/self/ref/memcheck` backend (which relies upon the 293[Valgrind](http://valgrind.org/) Memcheck tool to help verify that user 294{ref}`CeedQFunction` have no undefined values). 295This release also included various performance improvements, bug fixes, new examples, 296and improved tests. Among these improvements, vectorized instructions for 297{ref}`CeedQFunction` code compiled for CPU were enhanced by using `CeedPragmaSIMD` 298instead of `CeedPragmaOMP`, implementation of a {ref}`CeedQFunction` gallery and 299identity Q-Functions were introduced, and the PETSc benchmark problems were expanded 300to include unstructured meshes handling were. For this expansion, the prior version of 301the PETSc BPs, which only included data associated with structured geometries, were 302renamed `bpsraw`, and the new version of the BPs, which can handle data associated 303with any unstructured geometry, were called `bps`. Additionally, other benchmark 304problems, namely BP2 and BP4 (the vector-valued versions of BP1 and BP3, respectively), 305and BP5 and BP6 (the collocated versions---for which the quadrature points are the same 306as the Gauss Lobatto nodes---of BP3 and BP4 respectively) were added to the PETSc 307examples. Furthermoew, another standalone libCEED example, called `ex2`, which 308computes the surface area of a given mesh was added to this release. 309 310Backends available in this release: 311 312| CEED resource (`-ceed`) | Backend | 313|--------------------------|-----------------------------------------------------| 314| `/cpu/self/ref/serial` | Serial reference implementation | 315| `/cpu/self/ref/blocked` | Blocked reference implementation | 316| `/cpu/self/ref/memcheck` | Memcheck backend, undefined value checks | 317| `/cpu/self/opt/serial` | Serial optimized C implementation | 318| `/cpu/self/opt/blocked` | Blocked optimized C implementation | 319| `/cpu/self/avx/serial` | Serial AVX implementation | 320| `/cpu/self/avx/blocked` | Blocked AVX implementation | 321| `/cpu/self/xsmm/serial` | Serial LIBXSMM implementation | 322| `/cpu/self/xsmm/blocked` | Blocked LIBXSMM implementation | 323| `/cpu/occa` | Serial OCCA kernels | 324| `/gpu/occa` | CUDA OCCA kernels | 325| `/omp/occa` | OpenMP OCCA kernels | 326| `/ocl/occa` | OpenCL OCCA kernels | 327| `/gpu/cuda/ref` | Reference pure CUDA kernels | 328| `/gpu/cuda/reg` | Pure CUDA kernels using one thread per element | 329| `/gpu/cuda/shared` | Optimized pure CUDA kernels using shared memory | 330| `/gpu/cuda/gen` | Optimized pure CUDA kernels using code generation | 331| `/gpu/magma` | CUDA MAGMA kernels | 332 333Examples available in this release: 334 335:::{list-table} 336:header-rows: 1 337:widths: auto 338* - User code 339 - Example 340* - `ceed` 341 - * ex1 (volume) 342 * ex2 (surface) 343* - `mfem` 344 - * BP1 (scalar mass operator) 345 * BP3 (scalar Laplace operator) 346* - `petsc` 347 - * BP1 (scalar mass operator) 348 * BP2 (vector mass operator) 349 * BP3 (scalar Laplace operator) 350 * BP4 (vector Laplace operator) 351 * BP5 (collocated scalar Laplace operator) 352 * BP6 (collocated vector Laplace operator) 353 * Navier-Stokes 354* - `nek5000` 355 - * BP1 (scalar mass operator) 356 * BP3 (scalar Laplace operator) 357::: 358 359(v0-4)= 360 361## v0.4 (Apr 1, 2019) 362 363libCEED v0.4 was made again publicly available in the second full CEED software 364distribution, release CEED 2.0. This release contained notable features, such as 365four new CPU backends, two new GPU backends, CPU backend optimizations, initial 366support for operator composition, performance benchmarking, and a Navier-Stokes demo. 367The new CPU backends in this release came in two families. The `/cpu/self/*/serial` 368backends process one element at a time and are intended for meshes with a smaller number 369of high order elements. The `/cpu/self/*/blocked` backends process blocked batches of 370eight interlaced elements and are intended for meshes with higher numbers of elements. 371The `/cpu/self/avx/*` backends rely upon AVX instructions to provide vectorized CPU 372performance. The `/cpu/self/xsmm/*` backends rely upon the 373[LIBXSMM](http://github.com/hfp/libxsmm) package to provide vectorized CPU 374performance. The `/gpu/cuda/*` backends provide GPU performance strictly using CUDA. 375The `/gpu/cuda/ref` backend is a reference CUDA backend, providing reasonable 376performance for most problem configurations. The `/gpu/cuda/reg` backend uses a simple 377parallelization approach, where each thread treats a finite element. Using just in time 378compilation, provided by nvrtc (NVidia Runtime Compiler), and runtime parameters, this 379backend unroll loops and map memory address to registers. The `/gpu/cuda/reg` backend 380achieve good peak performance for 1D, 2D, and low order 3D problems, but performance 381deteriorates very quickly when threads run out of registers. 382 383A new explicit time-stepping Navier-Stokes solver was added to the family of libCEED 384examples in the `examples/petsc` directory (see {ref}`example-petsc-navier-stokes`). 385This example solves the time-dependent Navier-Stokes equations of compressible gas 386dynamics in a static Eulerian three-dimensional frame, using structured high-order 387finite/spectral element spatial discretizations and explicit high-order time-stepping 388(available in PETSc). Moreover, the Navier-Stokes example was developed using PETSc, 389so that the pointwise physics (defined at quadrature points) is separated from the 390parallelization and meshing concerns. 391 392Backends available in this release: 393 394| CEED resource (`-ceed`) | Backend | 395|--------------------------|-----------------------------------------------------| 396| `/cpu/self/ref/serial` | Serial reference implementation | 397| `/cpu/self/ref/blocked` | Blocked reference implementation | 398| `/cpu/self/tmpl` | Backend template, defaults to `/cpu/self/blocked` | 399| `/cpu/self/avx/serial` | Serial AVX implementation | 400| `/cpu/self/avx/blocked` | Blocked AVX implementation | 401| `/cpu/self/xsmm/serial` | Serial LIBXSMM implementation | 402| `/cpu/self/xsmm/blocked` | Blocked LIBXSMM implementation | 403| `/cpu/occa` | Serial OCCA kernels | 404| `/gpu/occa` | CUDA OCCA kernels | 405| `/omp/occa` | OpenMP OCCA kernels | 406| `/ocl/occa` | OpenCL OCCA kernels | 407| `/gpu/cuda/ref` | Reference pure CUDA kernels | 408| `/gpu/cuda/reg` | Pure CUDA kernels using one thread per element | 409| `/gpu/magma` | CUDA MAGMA kernels | 410 411Examples available in this release: 412 413:::{list-table} 414:header-rows: 1 415:widths: auto 416* - User code 417 - Example 418* - `ceed` 419 - * ex1 (volume) 420* - `mfem` 421 - * BP1 (scalar mass operator) 422 * BP3 (scalar Laplace operator) 423* - `petsc` 424 - * BP1 (scalar mass operator) 425 * BP3 (scalar Laplace operator) 426 * Navier-Stokes 427* - `nek5000` 428 - * BP1 (scalar mass operator) 429 * BP3 (scalar Laplace operator) 430::: 431 432(v0-3)= 433 434## v0.3 (Sep 30, 2018) 435 436Notable features in this release include active/passive field interface, support for 437non-tensor bases, backend optimization, and improved Fortran interface. This release 438also focused on providing improved continuous integration, and many new tests with code 439coverage reports of about 90%. This release also provided a significant change to the 440public interface: a {ref}`CeedQFunction` can take any number of named input and output 441arguments while {ref}`CeedOperator` connects them to the actual data, which may be 442supplied explicitly to `CeedOperatorApply()` (active) or separately via 443`CeedOperatorSetField()` (passive). This interface change enables reusable libraries 444of CeedQFunctions and composition of block solvers constructed using 445{ref}`CeedOperator`. A concept of blocked restriction was added to this release and 446used in an optimized CPU backend. Although this is typically not visible to the user, 447it enables effective use of arbitrary-length SIMD while maintaining cache locality. 448This CPU backend also implements an algebraic factorization of tensor product gradients 449to perform fewer operations than standard application of interpolation and 450differentiation from nodes to quadrature points. This algebraic formulation 451automatically supports non-polynomial and non-interpolatory bases, thus is more general 452than the more common derivation in terms of Lagrange polynomials on the quadrature points. 453 454Backends available in this release: 455 456| CEED resource (`-ceed`) | Backend | 457|-------------------------|-----------------------------------------------------| 458| `/cpu/self/blocked` | Blocked reference implementation | 459| `/cpu/self/ref` | Serial reference implementation | 460| `/cpu/self/tmpl` | Backend template, defaults to `/cpu/self/blocked` | 461| `/cpu/occa` | Serial OCCA kernels | 462| `/gpu/occa` | CUDA OCCA kernels | 463| `/omp/occa` | OpenMP OCCA kernels | 464| `/ocl/occa` | OpenCL OCCA kernels | 465| `/gpu/magma` | CUDA MAGMA kernels | 466 467Examples available in this release: 468 469:::{list-table} 470:header-rows: 1 471:widths: auto 472* - User code 473 - Example 474* - `ceed` 475 - * ex1 (volume) 476* - `mfem` 477 - * BP1 (scalar mass operator) 478 * BP3 (scalar Laplace operator) 479* - `petsc` 480 - * BP1 (scalar mass operator) 481 * BP3 (scalar Laplace operator) 482* - `nek5000` 483 - * BP1 (scalar mass operator) 484 * BP3 (scalar Laplace operator) 485::: 486 487(v0-21)= 488 489## v0.21 (Sep 30, 2018) 490 491A MAGMA backend (which relies upon the 492[MAGMA](https://bitbucket.org/icl/magma) package) was integrated in libCEED for this 493release. This initial integration set up the framework of using MAGMA and provided the 494libCEED functionality through MAGMA kernels as one of libCEED’s computational backends. 495As any other backend, the MAGMA backend provides extended basic data structures for 496{ref}`CeedVector`, {ref}`CeedElemRestriction`, and {ref}`CeedOperator`, and implements 497the fundamental CEED building blocks to work with the new data structures. 498In general, the MAGMA-specific data structures keep the libCEED pointers to CPU data 499but also add corresponding device (e.g., GPU) pointers to the data. Coherency is handled 500internally, and thus seamlessly to the user, through the functions/methods that are 501provided to support them. 502 503Backends available in this release: 504 505| CEED resource (`-ceed`) | Backend | 506|-------------------------|---------------------------------| 507| `/cpu/self` | Serial reference implementation | 508| `/cpu/occa` | Serial OCCA kernels | 509| `/gpu/occa` | CUDA OCCA kernels | 510| `/omp/occa` | OpenMP OCCA kernels | 511| `/ocl/occa` | OpenCL OCCA kernels | 512| `/gpu/magma` | CUDA MAGMA kernels | 513 514Examples available in this release: 515 516:::{list-table} 517:header-rows: 1 518:widths: auto 519* - User code 520 - Example 521* - `ceed` 522 - * ex1 (volume) 523* - `mfem` 524 - * BP1 (scalar mass operator) 525 * BP3 (scalar Laplace operator) 526* - `petsc` 527 - * BP1 (scalar mass operator) 528* - `nek5000` 529 - * BP1 (scalar mass operator) 530::: 531 532(v0-2)= 533 534## v0.2 (Mar 30, 2018) 535 536libCEED was made publicly available the first full CEED software distribution, release 537CEED 1.0. The distribution was made available using the Spack package manager to provide 538a common, easy-to-use build environment, where the user can build the CEED distribution 539with all dependencies. This release included a new Fortran interface for the library. 540This release also contained major improvements in the OCCA backend (including a new 541`/ocl/occa` backend) and new examples. The standalone libCEED example was modified to 542compute the volume volume of a given mesh (in 1D, 2D, or 3D) and placed in an 543`examples/ceed` subfolder. A new `mfem` example to perform BP3 (with the application 544of the Laplace operator) was also added to this release. 545 546Backends available in this release: 547 548| CEED resource (`-ceed`) | Backend | 549|-------------------------|---------------------------------| 550| `/cpu/self` | Serial reference implementation | 551| `/cpu/occa` | Serial OCCA kernels | 552| `/gpu/occa` | CUDA OCCA kernels | 553| `/omp/occa` | OpenMP OCCA kernels | 554| `/ocl/occa` | OpenCL OCCA kernels | 555 556Examples available in this release: 557 558:::{list-table} 559:header-rows: 1 560:widths: auto 561* - User code 562 - Example 563* - `ceed` 564 - * ex1 (volume) 565* - `mfem` 566 - * BP1 (scalar mass operator) 567 * BP3 (scalar Laplace operator) 568* - `petsc` 569 - * BP1 (scalar mass operator) 570* - `nek5000` 571 - * BP1 (scalar mass operator) 572::: 573 574(v0-1)= 575 576## v0.1 (Jan 3, 2018) 577 578Initial low-level API of the CEED project. The low-level API provides a set of Finite 579Elements kernels and components for writing new low-level kernels. Examples include: 580vector and sparse linear algebra, element matrix assembly over a batch of elements, 581partial assembly and action for efficient high-order operators like mass, diffusion, 582advection, etc. The main goal of the low-level API is to establish the basis for the 583high-level API. Also, identifying such low-level kernels and providing a reference 584implementation for them serves as the basis for specialized backend implementations. 585This release contained several backends: `/cpu/self`, and backends which rely upon the 586[OCCA](http://github.com/libocca/occa) package, such as `/cpu/occa`, 587`/gpu/occa`, and `/omp/occa`. 588It also included several examples, in the `examples` folder: 589A standalone code that shows the usage of libCEED (with no external 590dependencies) to apply the Laplace operator, `ex1`; an `mfem` example to perform BP1 591(with the application of the mass operator); and a `petsc` example to perform BP1 592(with the application of the mass operator). 593 594Backends available in this release: 595 596| CEED resource (`-ceed`) | Backend | 597|-------------------------|---------------------------------| 598| `/cpu/self` | Serial reference implementation | 599| `/cpu/occa` | Serial OCCA kernels | 600| `/gpu/occa` | CUDA OCCA kernels | 601| `/omp/occa` | OpenMP OCCA kernels | 602 603Examples available in this release: 604 605| User code | Example | 606|-----------------------|-----------------------------------| 607| `ceed` | ex1 (scalar Laplace operator) | 608| `mfem` | BP1 (scalar mass operator) | 609| `petsc` | BP1 (scalar mass operator) | 610``` 611