Delete out-of-date comments and do better overlap

Merge branch 'rmills/bindtocpu-not-pintocpu' into 'master'

Changed XXXPinToCPU() to XXXBindToCPU() to prevent confusion.

See merge request petsc/petsc!2477

Changed XXXPinToCPU() to XXXBindToCPU() to prevent confusion.

The reason for this change is that we already use the terminology
"pinned" to refer to memory that is non-pageable, in the context of
Pe

Changed XXXPinToCPU() to XXXBindToCPU() to prevent confusion.

The reason for this change is that we already use the terminology
"pinned" to refer to memory that is non-pageable, in the context of
PetscSF as well as allocating host memory when GPUs are being employed.

show more ...

Merge branch 'jczhang/feature-cuda-error-string' into 'master'

Map a cuda error code to its name and description

See merge request petsc/petsc!2228

Map a cuda error code to its name and description

Merge branch 'karlrupp/fix-cuda-streams' into 'master'

GPU: Fixed incorrect use of CUDA streams, SNES ex19 and ex56 now working with CUDA

See merge request petsc/petsc!2091

CUDA: Fixed incorrect use of separate streams.

This solves synchronization problems that have arisen due to the incorrect use of multiple CUDA streams for vector and matrix operations (without using

CUDA: Fixed incorrect use of separate streams.

This solves synchronization problems that have arisen due to the incorrect use of multiple CUDA streams for vector and matrix operations (without using proper synchronization mechanisms).
In particular, SNES ex19 and ex56 now run reliably (no failure after 20+ reruns).
Instead, the default stream (NULL pointer) is now used for all CUDA operations.
I don't have performance comparisons at hand for the performance implications in this commit, but expect any changes to be small.
Correctness first :-)

show more ...

Merge branch 'stefanozampini/gpu-bddc' into 'master'

Improvements towards BDDC on GPUs

See merge request petsc/petsc!2067

Fix compilation error for nvcc in optimized code with AVX-512 (march=native on my GPU workstation)

for some reason, the host compiler fails with this error message
/home/zampins/Devel/petsc/include/

Fix compilation error for nvcc in optimized code with AVX-512 (march=native on my GPU workstation)

for some reason, the host compiler fails with this error message
/home/zampins/Devel/petsc/include/../src/mat/impls/aij/seq/aij.h(535): error: identifier "_mm512_reduce_add_pd" is undefined

This optimized C kernel is not used in the GPU classes, so it is safe to skip its declaration

show more ...

Merge branch 'dalcinl/feature-math' [PR #1904]

* dalcinl/feature-math:
Math & PetscComplex: Various enhancements
- Define PetscXXXScalar to PetscXXXReal for real scalar type
- Add PetscCbrtReal(), P

Merge branch 'dalcinl/feature-math' [PR #1904]

* dalcinl/feature-math:
Math & PetscComplex: Various enhancements
- Define PetscXXXScalar to PetscXXXReal for real scalar type
- Add PetscCbrtReal(), PetscHypotReal(), and PetscAtan2Real()
- Add PetscArgComplex() and PetscArgScalar()
- Add PetscAtan{Real|Complex|Scalar}()
- Add PetscA{sin|cos|tan}h{Real|Complex|Scalar}()
- Docs: Petsc{Real|Imaginary}Part() return PetscReal
- Define __fp16 constants to use "F" suffix (ie. single precision)
- Fix PETSC_[SQRT_]MACHINE_EPSILON values for __fp16

PetscComplex: Remove PETSC_USE_CXX_COMPLEX_FLOAT_WORKAROUND

- Move the C++ complex fixes to its own header file
- Define PETSC_SKIP_CXX_COMPLEX_FIX to skip the C++ complex fixes

show more ...

Merge branch 'karlrupp/fix-cuda-MatSeqAIJCUSPARSEGenerateTransposeForMult' [PR #1948]

* karlrupp/fix-cuda-MatSeqAIJCUSPARSEGenerateTransposeForMult:
CUDA: Fixed issues in MatSeqAIJCUSPARSEGenerateTr

Merge branch 'karlrupp/fix-cuda-MatSeqAIJCUSPARSEGenerateTransposeForMult' [PR #1948]

* karlrupp/fix-cuda-MatSeqAIJCUSPARSEGenerateTransposeForMult:
CUDA: Fixed issues in MatSeqAIJCUSPARSEGenerateTransposeForMult and MatMultTransposeAdd_SeqAIJCUSPARSE

show more ...

CUDA: Fixed issues in MatSeqAIJCUSPARSEGenerateTransposeForMult and MatMultTransposeAdd_SeqAIJCUSPARSE

This is a cherry-pick of commits dde4751, 435e334, 1d884b8, 4e32a5a
Thanks-to: Mark Adams <ma23

CUDA: Fixed issues in MatSeqAIJCUSPARSEGenerateTransposeForMult and MatMultTransposeAdd_SeqAIJCUSPARSE

This is a cherry-pick of commits dde4751, 435e334, 1d884b8, 4e32a5a
Thanks-to: Mark Adams <ma2325@columbia.edu>

show more ...

CUDA: Skipping CXX complex fix.

Should fix warnings obtained with newer math functions.

This fix should be obsolete once the wrapper for GPU functionality is in place.

Merged in barry/cuda-multigrid-test (pull request #1763)

Various improvements for GPUs (mostly for performance and CUDA)

Various improvements for GPUs (mostly for performance and CUDA)

1) Add VecPinToCPU() for CUDA vector and matrices
2) Move initialization of cuBLAS to PetscInitialize() since it takes 1/2 second and

Various improvements for GPUs (mostly for performance and CUDA)

1) Add VecPinToCPU() for CUDA vector and matrices
2) Move initialization of cuBLAS to PetscInitialize() since it takes 1/2 second and distorts timing with -log_view
3) Add logging for DMCreateMatrix (for large meshes this is very large)
4) Add VecGet/RestoreArrayWrite() to prevent unneeded copies from GPU (only implemented so far for CUDA);
added a small number of usages in the source so that snes tutorials ex19 does not do unneeded communication from the GPU
5) Automatically convert MAIJ matrices to AIJ for CUDA since they are not yet supported natively in PETSc's CUDA matrix implementation
6) Pinned objects should still use the CUDA/ViennaCL versions of Destroy to clean up the GPU stuff

Commit-type: feature

show more ...

Merged in barry/update-collective-on (pull request #1744)

Update the use of Collective on in the manual pages to reflect the new style

Update the use of Collective on in the manual pages to reflect the new style

Commit-type: style-fix, documentation
Thanks-to: Patrick Sanan <patrick.sanan@gmail.com>

Merge branch 'master' into wence/feature-patch-all-at-once

Merge branch 'jczhang/fix-vecscatter-cuda/maint' into maint [PR #1388]

* jczhang/fix-vecscatter-cuda/maint:
CUDA vecscatter needs to take care of the ScatterMode argument

Merge branch 'jczhang/fix-vecscatter-cuda/maint' [PR #1388]

* jczhang/fix-vecscatter-cuda/maint:
CUDA vecscatter needs to take care of the ScatterMode argument

Merge branch 'jczhang/restore-error-check' [PR #1392]

* jczhang/restore-error-check:
Restore an error checking line in MatMultTranspose_MPIAIJCUSPARSE

Restore the error checking code

Add a ScatterMode arg in cuda vecscat to select to/from context

The old code VecScatterInitializeForGPU() initializes the pointer (PetscCUDAIndices*)&inctx->spptr) based
on an input ScatterMode befo

Add a ScatterMode arg in cuda vecscat to select to/from context

The old code VecScatterInitializeForGPU() initializes the pointer (PetscCUDAIndices*)&inctx->spptr) based
on an input ScatterMode before VecScatterBegin() is called.

If a vecscatter context is firstly used for a SCATTER_FORWARD, and secondly used
for a SCATTER_REVERSE, there will be an error. Since in the second VecScatter,
it uses out-of-date (PetscCUDAIndices*)&inctx->spptr)

The solution is "do not prematurely consider ScatterMode when building (PetscCUDAIndices*)&inctx->spptr). Instead,
select correct to/from until VecScatterBegin() is called"

show more ...

Merge branch 'master' into jpusztay/feature-swarm-symplectic-example

Merge branch 'master' into jpustay/feature-swarm-example