First attempt at sanitising chunking.

The rationale is to keep the old behaviour if possible, so we just add a if-
else after the chunk sizes are first calculated and change those sizes that
need to

First attempt at sanitising chunking.

The rationale is to keep the old behaviour if possible, so we just add a if-
else after the chunk sizes are first calculated and change those sizes that
need to be changed. [There is also a zeroing of the 6-element arrays in the
beginning of the function because we multiply them together to find the various
sizes and at least Cray seems to have random numbers in the stack when
entering the function, causing the sizes to be crazy.]

We first check if the average vector size exceeds 4 GiB and if it does, simply
use as few chunks as possible, splitting along slowest varying axis first and
if that is not enough, splitting the next axis, too etc. The patch *can* split
the fastest varying dimension, too, but that seems overkill: it would mean
that da_grid_x*dof > 4 GiB, which seems unlikely. It cannot split dof.

If the average Vec size is <= 4 GiB, but total Vec size is > 4 GiB, there is a
simple logic: first see if <local_z_size>*<global_y_size>*<global_x_size>*dof
is smaller than "target size". Target size comes from your formula earlier in
this thread. If that is so, use that, if not, then just use the local Vec size
as the chunk size. This could be improved by having an intermediate size
solution, too, but this solution gives *my* tests decent performance across
several global sizes, so it does not seem to be too bad. Now, the performance
we get is just ~3 GiB/s, using 36 OSTs on a lustre. Using one OST we get ~200
MiB/s, so the scaling is not that bad: we get half of 36*200 MiB/s, so we think
the performance is limited by the OSTs, not MPI-IO or HDF5 or chunking. Tests
with bigger systems would of course be needed to determine whether this
solution works for truly huge files, too: the biggest we tried are just over 50
GiB.

show more ...

Merge branch 'master' into barry/wirth-fusion-materials

Conflicts:
src/ts/examples/tutorials/advection-diffusion-reaction/ex10.c

Merge branch 'barry/dmvecmattypes' into barry/saws

Needed to work with version of PETSc that did not have constant calls to VecSetFromOptions() etc

Conflicts:
src/ksp/ksp/interface/ams/kspams.c
s

Merge branch 'barry/dmvecmattypes' into barry/saws

Needed to work with version of PETSc that did not have constant calls to VecSetFromOptions() etc

Conflicts:
src/ksp/ksp/interface/ams/kspams.c
src/snes/impls/composite/snescomposite.c
src/snes/impls/gs/snesgs.c
src/snes/impls/nasm/nasm.c
src/snes/impls/ngmres/snesngmres.c

show more ...

Merge branch 'master' into knepley/feature-dt-fem

* master: (211 commits)
Mat ex170: Comments
VTK: Small fix to error message (.vts to .vtu)
VTK: Small fix to error message
Fixed bib entries

Merge branch 'master' into knepley/feature-dt-fem

* master: (211 commits)
Mat ex170: Comments
VTK: Small fix to error message (.vts to .vtu)
VTK: Small fix to error message
Fixed bib entries
Bib: Updates
AO: fix erroneous processing of -ao_view and factor into AOViewFromOptions
doc: fix named argument in {Vec,Mat,DM}ViewFromOptions
Sys: add PetscDataTypeFromString() and test code
Mat: Should say that it has a nullspace in MatView()
parms: update tarball with fix for namespace conflict with metis
fix citation 'Golub_Varga_1961'
parmetis: update tarball to parmetis-4.0.2-p5 which fixes an install issue with cygwin
Sys Logging: revert parent traversal
fixed hdf5.py so that if self.libraries.compression is None the code still runs correctly
DMDA: fix bad cast of DM_DA to PetscObject
MatClique: follow DistMultiVec API changes
MatClique: remove unused variables
config cmakeboot: add C++ flags any time compiler is available
config OpenMP: check for C++ flag any time the compiler is available
replaced all left-over uses of a single PetscMalloc() to allocated multiple arrays: replaced with PetscMallocN() The only ones left are when the second array is set into the first array and one ugly usage in the MUMPS interface that cannot be easily fixed
...

Conflicts:
config/builder.py
src/dm/impls/plex/plexgeometry.c

show more ...

Merge branch 'master' into knepley/feature-plex-refine-3d

* master: (273 commits)
Mat ex170: Comments
VTK: Small fix to error message (.vts to .vtu)
VTK: Small fix to error message
Fixed bib

Merge branch 'master' into knepley/feature-plex-refine-3d

* master: (273 commits)
Mat ex170: Comments
VTK: Small fix to error message (.vts to .vtu)
VTK: Small fix to error message
Fixed bib entries
Bib: Updates
AO: fix erroneous processing of -ao_view and factor into AOViewFromOptions
doc: fix named argument in {Vec,Mat,DM}ViewFromOptions
Sys: add PetscDataTypeFromString() and test code
Mat: Should say that it has a nullspace in MatView()
parms: update tarball with fix for namespace conflict with metis
fix citation 'Golub_Varga_1961'
parmetis: update tarball to parmetis-4.0.2-p5 which fixes an install issue with cygwin
Sys Logging: revert parent traversal
fixed hdf5.py so that if self.libraries.compression is None the code still runs correctly
DMDA: fix bad cast of DM_DA to PetscObject
MatClique: follow DistMultiVec API changes
MatClique: remove unused variables
config cmakeboot: add C++ flags any time compiler is available
config OpenMP: check for C++ flag any time the compiler is available
replaced all left-over uses of a single PetscMalloc() to allocated multiple arrays: replaced with PetscMallocN() The only ones left are when the second array is set into the first array and one ugly usage in the MUMPS interface that cannot be easily fixed
...

Conflicts:
include/petscdmplex.h

show more ...

Merge branch 'master' into knepley/feature-dmda-section

* master: (287 commits)
Mat ex170: Comments
VTK: Small fix to error message (.vts to .vtu)
VTK: Small fix to error message
Fixed bib e

Merge branch 'master' into knepley/feature-dmda-section

* master: (287 commits)
Mat ex170: Comments
VTK: Small fix to error message (.vts to .vtu)
VTK: Small fix to error message
Fixed bib entries
Bib: Updates
AO: fix erroneous processing of -ao_view and factor into AOViewFromOptions
doc: fix named argument in {Vec,Mat,DM}ViewFromOptions
Sys: add PetscDataTypeFromString() and test code
Mat: Should say that it has a nullspace in MatView()
parms: update tarball with fix for namespace conflict with metis
fix citation 'Golub_Varga_1961'
parmetis: update tarball to parmetis-4.0.2-p5 which fixes an install issue with cygwin
Sys Logging: revert parent traversal
fixed hdf5.py so that if self.libraries.compression is None the code still runs correctly
DMDA: fix bad cast of DM_DA to PetscObject
MatClique: follow DistMultiVec API changes
MatClique: remove unused variables
config cmakeboot: add C++ flags any time compiler is available
config OpenMP: check for C++ flag any time the compiler is available
replaced all left-over uses of a single PetscMalloc() to allocated multiple arrays: replaced with PetscMallocN() The only ones left are when the second array is set into the first array and one ugly usage in the MUMPS interface that cannot be easily fixed
...

show more ...

Merge branch 'master' of bitbucket.org:petsc/petsc

Merge branch 'jed/fix-viewer-matlab-info' into maint

* jed/fix-viewer-matlab-info:
Viewer: fix Binary Matlab viewer info to use '#' instead of '%'

Merge branch 'jed/fix-viewer-matlab-info'

* jed/fix-viewer-matlab-info:
Viewer: fix Binary Matlab viewer info to use '#' instead of '%'

Viewer: fix Binary Matlab viewer info to use '#' instead of '%'

The % character for comments in info files was removed in

7c43ebc8e6c "only use # for comments in PETSc options files"

but continu

Viewer: fix Binary Matlab viewer info to use '#' instead of '%'

The % character for comments in info files was removed in

7c43ebc8e6c "only use # for comments in PETSc options files"

but continued to be used for the PETSC_VIEWER_BINARY_MATLAB format
characters. This prevented loading these files from PETSc. This commit
switches to using '#' for those comment characters and updates
PetscReadBinaryMatlab to accept both '%$$ ' and '#$$ '.

Noticed-by: Richard Katz <richard.katz@earth.ox.ac.uk>

show more ...

Merge branch 'master' into jed/ts-eimex

Sync to include Git conversion, PETSC_EXTERN, and minor API changes.

Conflicts:
src/ts/interface/tsregall.c

VecView_MPI_HDF5_DA: remove unused variable

Automerge.

Hg-commit: 81500f7e84c8ec0c4dcae9cb31d1cb32892d545d

fixed marking of PETSC_EXTERN/INTERN for a bunch of private functions

Hg-commit: a1e34d83003ce63bcd85d0f32ea6cc694c6fd4eb

Automerge.

Hg-commit: b6659d546870fb013f3da5bcd5066d1dc0dc329c

Merge branch 'master' of gitifyhg::ssh://hg@bitbucket.org/BarryFSmith/petsc-dev-simp

Symbol visibility and namespacing.

C++ builds always set extern "C" and can be called from plain C. Most
users w

Merge branch 'master' of gitifyhg::ssh://hg@bitbucket.org/BarryFSmith/petsc-dev-simp

Symbol visibility and namespacing.

C++ builds always set extern "C" and can be called from plain C. Most
users will only want --with-clanguage=C++ for std::complex.


Hg-commit: f848d02318cae92d7b32037c7ee88f92dbe46347

show more ...

began removing unneeded EXTERN_C_BEGIN/END and converting needed ones to PETSC_EXTERN_C

Hg-commit: e0c157606a3283d5a4ba8902a0da5cad0e2aa531

add missing prototype
allow setting coordinates of contour plot smaller than entire domain

Hg-commit: 3cd8fd1ba588913ef042c7c4d699c111b3ea4867

Remove petscviewer.h from petscsys.h

Much of sys is independent of viewer. Viewer is used in so many other
places that it is now included by petscvec.h.

Hg-commit: 1d745f01618aa51d3043fb1f87b0546f6

Remove petscviewer.h from petscsys.h

Much of sys is independent of viewer. Viewer is used in so many other
places that it is now included by petscvec.h.

Hg-commit: 1d745f01618aa51d3043fb1f87b0546f60e6e730

show more ...

daimpl.h was never fixed? Well it is now

Hg-commit: a4f3f6d7e4f165ac5276405a089a596a28adc0b3

added petscdrawtypes.h removed the typedef _xx* xx definitions from petscdraw.h and removed petscdraw.h from petscsys.h

Hg-commit: 01be7b4c0ac1e993b540535c319de45af820a255

removed PetscViewerBinaryMatlabOpen() and friends, incorporated functionality into binary PETSc viewer

Hg-commit: 1676211ea1509edcdd7a044ca5b8b543f443b8ab

added PetscObjectComm() and used it to replace (((PetscObject)obj)->comm)

Hg-commit: 3da37c458124ad48ae939f4e9823e4430ee0b8be

fix NULL usage when integer 0 was intended

Hg-commit: fc3430d57afb79227acdf580915f2ffbe770bb38

removed PETSC_NULL from C and Fortran (except declaration in C for backward compatibility). Kept PETSC_NULL_xxx for Fortran
Fixed a few bugs where PETSC_NULL had been used incorrectly.

Hg-commit: 05

removed PETSC_NULL from C and Fortran (except declaration in C for backward compatibility). Kept PETSC_NULL_xxx for Fortran
Fixed a few bugs where PETSC_NULL had been used incorrectly.

Hg-commit: 054705a517d7f4388a8a084415d7478cbe95dff4

show more ...