Sys: Introduce -on_error_malloc_dump and PETSC_ERR_MEM_LEAK

To avoid a malloc dump that swamps the error output of another error,
this makes calling `PetscMallocDump()` in the error handlers opt-in,

Sys: Introduce -on_error_malloc_dump and PETSC_ERR_MEM_LEAK

To avoid a malloc dump that swamps the error output of another error,
this makes calling `PetscMallocDump()` in the error handlers opt-in,
with a different flag (`-on_error_malloc_dump`) from the flag for
calling `PetscMallocDump()` in `PetscFinalize()` (just `-malloc_dump`).

The addition of `PETSC_ERR_MEM_LEAK` allows for better error messages:
`~PoolAllocator()` can detect a memory leak even if the system
is not out of memory, so the error message will not say that for
`PETSC_ERR_MEM_LEAK`.

show more ...

Docs: fix typos and missing back ticks

Merge branch 'barry/2023-07-18/manual-linter-fixes-ksp' into 'main'

Manual fixes needed for linter for the KSP directory

See merge request petsc/petsc!6735

Merge branch 'release'

Increase patchlevel to 3.19.4

Add docs on using MPI environmental variables to control compilers selected by mpicc etc

Manual fixes needed for linter for the KSP directory

Add convenience functions VecCreateFromOptions() and MatCreateFromOptions() to reduce the use of specialize creation routines in the examples and thus make it easier for people to use GPU based solve

Add convenience functions VecCreateFromOptions() and MatCreateFromOptions() to reduce the use of specialize creation routines in the examples and thus make it easier for people to use GPU based solvers

show more ...

Profiling: Add composed functions for default log handler functions

This makes them extensible to other implementations.
Where appropriate, the global log handler will now dispatch
these functions t

Profiling: Add composed functions for default log handler functions

This makes them extensible to other implementations.
Where appropriate, the global log handler will now dispatch
these functions to all running log handlers.

It is not always appropriate to dispatch to all running log handlers:
when a single return value is expected or when a filename
is given, it will continue to dispatch only to the default
log handler.

show more ...

Profiling: Rename PETSC_LOG_HANDLER_XXX -> PETSCLOGHANDLERXXX

Doc: Document flamegraph and perfstubs profiling usage

Profiling: Switch logging over to the new PetscLogHandler infrastructure

- Move data structures only needed by the default logger now (Action,
Object) to logdefault.c

- Create petsclogdeprecated.

Profiling: Switch logging over to the new PetscLogHandler infrastructure

- Move data structures only needed by the default logger now (Action,
Object) to logdefault.c

- Create petsclogdeprecated.h and petsc/private/logimpldeprecated.h
for publicly visible symbols that are now deprecated. Yes,
lots of symbols in petsc/private/logimpl.h were not only visible
but had docstrings.

- The macros PetscLogEventBegin(), PetscLogEventEnd(),
PetscLogEventSync(), PetscLogObjectCreate(), and
PetscLogObjectDestroy() now looks through the log handlers
in PetscLogHandlers, sees if they have they appropriate
callbacks (Exposed in PetscLogHandlerHot), and calls them.
Up to 4 log handlers can run simultaneously.

- All queries about the registered things now go through
petsc_log_state.

- Output functions dispatch to the appropriate PetscLogHandler

- There is no need for xmllogevent.c, xmlviewer.c, eventlog.c,
classlog.c, and stagelog.c anymore. Some orphaned functions
(PetscLogSetThreshold(), PetscLogEventSetDof(),
PetscLogEventSetError(), PetscASend(), PetscARecv(), PetscAReduce())

- PetscFEOpenCLLogResidual() did some intrusive things related
to the old logging infrastruture: the private header
for the default log handler is now imported into feopencl.c
so that it can do equivalent things with the new infrastructure.

- PetscLogPauseCurrentEvent_Internal() is replaced with
PetscLogEventsPause()

Deprecated symbols include:

- All functions related to PetscStageLog and the resizable array structs
that it contained.

- The legacy EventBegin()/EventEnd() callbacks.

- PetscLogAllBegin(), which actually reported less information than
PetscLogDefaultBegin() because it was not kept up-to-date with
regards to GPU profiling.

show more ...

Profiling: Add PetscLogState and PetscLogHandler to global logging

Without removing the existing global objects (such as petsc_stageLog),
This commit creates a global PetscLogState (petsc_log_state)

Profiling: Add PetscLogState and PetscLogHandler to global logging

Without removing the existing global objects (such as petsc_stageLog),
This commit creates a global PetscLogState (petsc_log_state) and
connects it to PetscLogInitialize() and PetscLogFinalize().

The functions PetscLogHandlerStart() and PetscLogHandlerStop() are added
to connect a general PetscLogHandler() to the global petsc_log_state,

A small array of slots for log handlers, PetscLogHandlers, is
added. It will be used in the macros like `PetscLogEventBegin()`
and `PetscLogEventEnd()`, so we make a wrapper type
`PetscLogHandlerHot` that exposes just as much of the internals
as are necessary to implement the macros.

Because they do not conflict with existing functions,
the following functions that depend on the new infrastructure
are added:

- PetscLogStageGetName(), PetscLogEventGetName(), PetscLogClassGetId(),
and PetscLogClassIdGetName() for completeness and to bring
of the functionality from petsc4pys' custom header into the
library proper.

- PetscLogEventsPause() and PetscLogEventsResume(), which
will replace PetscLogPauseCurrentEvent_Internal() and
PetscLogEventResume_Internal()

- PetscLogPerfstubsBegin() and PetscLogLegacyCallbacksBegin()

show more ...

Profiling: Add legacy callbacks implementation of PetscLogHandler

The existing logging infrastructures has four exposed callback pointers:

- PetscLogPLB / PetscLogPLE for PetscLogEventBegin() / Pet

Profiling: Add legacy callbacks implementation of PetscLogHandler

The existing logging infrastructures has four exposed callback pointers:

- PetscLogPLB / PetscLogPLE for PetscLogEventBegin() / PetscLogEventEnd()
These can also be set with PetscLogSet

- PetscLogPHC / PetscLogPHD for PetscLogObjectCreate() /
PetscLogObjectDestroy()

This log handler implementation provides a path for users who have been
setting their own callbacks with these functions to upgrade to the new
PetscLogHandler interface.

show more ...

Profiling: Add tau perfstubs implementation of PetscLogHandler

In the existing logging, perfstubs runs automatically with -log_view
if PETSc is configured --with-tau-perfstubs. This new PetscLogHan

Profiling: Add tau perfstubs implementation of PetscLogHandler

In the existing logging, perfstubs runs automatically with -log_view
if PETSc is configured --with-tau-perfstubs. This new PetscLogHandler
implementation will be used with a new -log_perfstubs command line
option that will only run this handler and not the default handler.

show more ...

Profiling: add MPE implementation of PetscLogHandler

This is meant to handle -log_mpe.

Profiling: Add trace implementation of PetscLogHandler

This is a minimal PetscLogHandler implementation for -log_trace.

Profiling: Add nested PetscLogHandler implementation

This log handler has behavior like the behavior of
PetscLogNestedBegin().

Unlike the implementation of PetscLogHandler_Default, I did not try to

Profiling: Add nested PetscLogHandler implementation

This log handler has behavior like the behavior of
PetscLogNestedBegin().

Unlike the implementation of PetscLogHandler_Default, I did not try to
do a faithful translation of the existing nested logging behavior, for
two reasons.

- The existing implementation keeps sorted lists of nested paths ("Event
A;Event B;"), and for each of those paths keeps a sorted list of the
top-level events ("Event C", "Event D") that map that path to other
paths (path + event => other path). It's a complicated data structure,
and the benefit of the sorting (O(log n) lookup) is probably not worth
the more expensive insertion cost each time a new path + event combo
appears. Instead I implemented the path + event => other path
lookup as a hash map, which simplified the implementation
significantly.

- The existing implementation assigns an event to each path, modifying
the event registry in place. This messes things up if you want
to both -log_view and -log_view ::ascii_flamegraph output, the
original motivation for refactoring the profiling infrastructure.
In the new implementation, the nested handler has its own
PetscLogState that it modifies, and an inner copy of
PetscLogHandler_Default the acts on that nested state.

- The PetscLogGlobalNames data structure also simplifies the
construction of a linearized tree of events in the final
viewing stage, although the low-level xml construction
routines are copied almost verbatim from the previous
implementation.

show more ...

Profiling: Implement the default PetscLogHandler

The existing default log handling that uses the PetscStageLog
is translated in the PetscLogHandler_Default implementation
of PetscLogHandler.

- Most

Profiling: Implement the default PetscLogHandler

The existing default log handling that uses the PetscStageLog
is translated in the PetscLogHandler_Default implementation
of PetscLogHandler.

- Most data that is global in the existing log handling
is moved into the struct, with the exception of the
raw counters.

- The registry data structures of PetscStageLog (eventLog,
classLog) are not reproduced because they are now in
the PetscLogState interface.

- PETSC_LOG_RESIZABLE_ARRAY is used to implement
resizable arrays instead of PetscStageInfo,
PetscEventPerfLog, and PetscClassPerfLog

- The tau perfstubs and MPE functionality is not
reproduced because they will become new
handlers on their own.

- The PetscLogGlobalNames provided by the
PetscLogRegistry interface make the logging
robust to differences in which events / stages are
registered on which process and in which order.

Otherwise the functions are relatively faithful translations
of the existing PetscStageLog functions into the new interface.
When reviewing this commit it might help to have a diff
between the PetscStageLog and PetscLogHandler_Default functions
to see what changes between them. I am working on a way to attach
that to the MR associated with this change.

show more ...

Profiling: Add PetscLogHandler interface

PetscStageLog tries to handle multiple ways of interpreting profiling
data:

* -log_view (default),
* -log_view ::ascii_xml and -log_view ::ascii_flamegraph

Profiling: Add PetscLogHandler interface

PetscStageLog tries to handle multiple ways of interpreting profiling
data:

* -log_view (default),
* -log_view ::ascii_xml and -log_view ::ascii_flamegraph (nested),
* -log_trace,
* -log_mpe,
* tau perfstubs (implicitly),
* nvidia

They can interact with petsc_logState in incompatible ways.

PetscLogHandler gives each handler its own interface so that
it can have its own data structures. They key operations
that a log handler can implement are:

- PetscLogHandlerEventBegin() / PetscLogHandlerEventEnd()
- PetscLogHandlerEventSync()
- PetscLogHandlerStagePush() / PetscLogHandlerStagePop()
- PetscLogHandlerObjectCreate() / PetscLogHandlerObjectDestroy()

The remaining operations are PetscObject standard functions,
with the except of `PetscLogHandlerSetState()` / `PetscLogHandlerGetState()`,
which control the `PetscLogState` that the handler can use to
get information about registered things and about the stage stack.

This change only introduces the type, it does not deploy it
to replace anything in the current profiling interface.
When it does, the user-defined log handling that was implemented
by setting callbacks in in PetscLogSet() will be replaced with
user-defined PetscLogHandler instances.

show more ...

Profiling: Add PetscLogState interface to describe events and stages

The design that is being implemented is a separation between the
logging state that is common to multiple log handlers and the lo

Profiling: Add PetscLogState interface to describe events and stages

The design that is being implemented is a separation between the
logging state that is common to multiple log handlers and the log
handlers themselves. This new interface PetscLogState will
become the replacement for PetscStageLog, but this change
only introduces it without deploying it.

Internally, PetscLogState is:

- A registry (PetscLogRegistry) of more-or-less immutable information
about stages, events, and classes that have been registered. The
registry is not exposed, but the registry entries (PetscLogEventInfo,
PetscLogStageInfo, PetscLogClassInfo), that I feel comfortable exposing
them (pass-by-copy) through PetscLogStateXXXGetInfo() functions.

- A stack of stages that have been pushed and popped

- A PetscBT describing the active/inactive state of logging stages
and events. I have kept the existing semantics of
PetscLogStateSetActive / PetscLogEventActivate /
PetscLogEventActivateClass, which is: if the stage is active
and the (event,stage) pair is active, then log handling proceeds.
This logic is encoded in PetscLogStateStageEventIsActive

- For symmetry I have added PetscLogClass that matches PetscLogStage
and PetscLogEvent

- The profiling interface is littered with ad hoc resizable array types
(PetscEventRegLog, PetscEventPerfLog, PetscClassPerfLog,
PetscStageLog). The intention is for these to go away. This commit
introduces a PetscHashMap-style macro constructor for resizable
arrays (PETSC_LOG_RESIZABLE_ARRAY). It is only used in implementing
these arrays: they do not appear in the public interface. Storage
could be switched to a hashmap or other backing storage at some point
in the future.

show more ...

Profiling: make visibility of logging types align with manpages

- Create file petsclogtypes.h for exposed logging data types. Move types
that will remain through upcoming changes (PetscLogEvent,

Profiling: make visibility of logging types align with manpages

- Create file petsclogtypes.h for exposed logging data types. Move types
that will remain through upcoming changes (PetscLogEvent,
PetscLogStage, PetscIntStack, PetscLogEventPerfInfo).

- Add docstring for PetscLogEventPerfInfo: it is the one important
type carrying profiling information that will remain through upcoming
changes.

- Change the submansec of petsclog.h from Sys to Profiling, better
reflecting the contents of the file.

- As the PetscIntStack methods have had manual pages (and externally
visible symbols) for years, move their declarations from
petsc/private/logimpl.h to petsclog.h. PetscIntStack is a low-level
data-structure similar to PetscBT, and this change puts PetscIntStack on
roughly equal footing with PetscBT.

show more ...

Testing: Add 'temporaries:' field to the test suite

It's nice when the output that needs to be tested goes to stdout, and we
can do that with our own viewers, but some output that needs to be
tested

Testing: Add 'temporaries:' field to the test suite

It's nice when the output that needs to be tested goes to stdout, and we
can do that with our own viewers, but some output that needs to be
tested can only be written to temporary files. An example is MPE
logging, which produces a .clog2 file, let's say it's called ex30.clog2.

We can access that file through the test suite:

/*TEST

test:
filter: strings ex30.clog2 | grep "Petsc"

TEST*/

But the problem is that ex30.clog2 will stay in the test build
directory. Suppose the test breaks and the program does not write
the ex30.clog2 file: then a stale version of that file will be read.

This change allows us to write the test as:

/*TEST

test:
temporaries: ex30.clog2
filter: strings ex30.clog2 | grep "Petsc"

TEST*/

which indices that (the localized version of) `rm -f ex30.clog2` will
run before the test.

show more ...

Merge branch 'barry/2023-06-16/pcmpi-simplify-options-database-api' into 'main'

Change the options database API for PCMPI to eliminate the -mpi_ options...

See merge request petsc/petsc!6611

Change the options database API for PCMPI to eliminate the -mpi_ options prefix; thus one can run -mpi_linear_solver_server -ksp_type cg -pc_type gamg to get the parallel version of these preconditio

Change the options database API for PCMPI to eliminate the -mpi_ options prefix; thus one can run -mpi_linear_solver_server -ksp_type cg -pc_type gamg to get the parallel version of these preconditioners.

Commit-type: usability

show more ...