xref: /petsc/doc/developers/style.md (revision 4bcd95a3096906c174cd1a87ff30988eadf5695d)
1*4bcd95a3SBarry Smith(style)=
2*4bcd95a3SBarry Smith
3*4bcd95a3SBarry Smith# PETSc Style and Usage Guide
4*4bcd95a3SBarry Smith
5*4bcd95a3SBarry SmithThe PETSc team uses certain conventions to make the source code
6*4bcd95a3SBarry Smithconsistent and easier to maintain. We will interchangeably use the
7*4bcd95a3SBarry Smithterminology *subclass*, *implementation*, or *type* [^footnote-1] to refer to a
8*4bcd95a3SBarry Smithconcrete realization of an abstract base class. For example,
9*4bcd95a3SBarry Smith`KSPGMRES` is a type for the base class `KSP`.
10*4bcd95a3SBarry Smith
11*4bcd95a3SBarry Smith## Names
12*4bcd95a3SBarry Smith
13*4bcd95a3SBarry SmithConsistency of names for variables, functions, and so on is extremely
14*4bcd95a3SBarry Smithimportant. We use several conventions
15*4bcd95a3SBarry Smith
16*4bcd95a3SBarry Smith01. All function names and enum types consist of acronyms or words, each
17*4bcd95a3SBarry Smith    of which is capitalized, for example, `KSPSolve()` and
18*4bcd95a3SBarry Smith    `MatGetOrdering()`.
19*4bcd95a3SBarry Smith
20*4bcd95a3SBarry Smith02. All enum elements and macro variables are named with all capital
21*4bcd95a3SBarry Smith    letters. When they consist of several complete words, there is an
22*4bcd95a3SBarry Smith    underscore between each word. For example, `MAT_FINAL_ASSEMBLY`.
23*4bcd95a3SBarry Smith
24*4bcd95a3SBarry Smith03. Functions that are private to PETSc (not callable by the application
25*4bcd95a3SBarry Smith    code) either
26*4bcd95a3SBarry Smith
27*4bcd95a3SBarry Smith    - have an appended `_Private` (for example, `StashValues_Private`)
28*4bcd95a3SBarry Smith      or
29*4bcd95a3SBarry Smith    - have an appended `_Subtype` (for example, `MatMultSeq_AIJ`).
30*4bcd95a3SBarry Smith
31*4bcd95a3SBarry Smith    In addition, functions that are not intended for use outside of a
32*4bcd95a3SBarry Smith    particular file are declared `static`. Also, see the item
33*4bcd95a3SBarry Smith    on symbol visibility in {ref}`usage_of_petsc_functions_and_macros`.
34*4bcd95a3SBarry Smith
35*4bcd95a3SBarry Smith04. Function names in structures (for example, `_matops`) are the same
36*4bcd95a3SBarry Smith    as the base application function name without the object prefix and
37*4bcd95a3SBarry Smith    in lowercase. For example, `MatMultTranspose()` has a
38*4bcd95a3SBarry Smith    structure name of `multtranspose`.
39*4bcd95a3SBarry Smith
40*4bcd95a3SBarry Smith05. Names of implementations of class functions should begin with the
41*4bcd95a3SBarry Smith    function name, an underscore, and the name of the implementation, for
42*4bcd95a3SBarry Smith    example, `KSPSolve_GMRES()`.
43*4bcd95a3SBarry Smith
44*4bcd95a3SBarry Smith06. Each application-usable function begins with the name of the class
45*4bcd95a3SBarry Smith    object, followed by any subclass name, for example,
46*4bcd95a3SBarry Smith    `ISInvertPermutation()`, `MatMult()`, or
47*4bcd95a3SBarry Smith    `KSPGMRESSetRestart()`.
48*4bcd95a3SBarry Smith
49*4bcd95a3SBarry Smith07. Functions that PETSc provides as defaults for user-providable
50*4bcd95a3SBarry Smith    functions end with `Default` (for example, `PetscSignalHandlerDefault()`).
51*4bcd95a3SBarry Smith
52*4bcd95a3SBarry Smith08. Options database keys are lower case, have an underscore between
53*4bcd95a3SBarry Smith    words, and match the function name associated with the option without
54*4bcd95a3SBarry Smith    the word “set” or “get”, for example, `-ksp_gmres_restart`.
55*4bcd95a3SBarry Smith
56*4bcd95a3SBarry Smith09. Specific `XXXType` values (for example, `MATSEQAIJ`) do not have
57*4bcd95a3SBarry Smith    an underscore in them unless they refer to another package that uses
58*4bcd95a3SBarry Smith    an underscore, for example, `MATSOLVERSUPERLU_DIST`.
59*4bcd95a3SBarry Smith
60*4bcd95a3SBarry Smith10. Typedefs for functions should end in `Fn` as in, for example, `SNESFunctionFn`.
61*4bcd95a3SBarry Smith
62*4bcd95a3SBarry Smith(stylepetsccount)=
63*4bcd95a3SBarry Smith
64*4bcd95a3SBarry Smith## PETSc and standard datatypes
65*4bcd95a3SBarry Smith
66*4bcd95a3SBarry Smith1. `PetscInt` is generally used for array indices, and array lengths. It
67*4bcd95a3SBarry Smith   is a signed 32-bit or 64-bit `int` depending on the `./configure` option
68*4bcd95a3SBarry Smith   `--with-64-bit-indices`. There is the possibility of integer overflow with the
69*4bcd95a3SBarry Smith   32-bit version.
70*4bcd95a3SBarry Smith
71*4bcd95a3SBarry Smith2. `PetscCount` is `ptrdiff_t`, (on 64-bit systems it is 64-bits) and should be used for array sizes (number of entries)
72*4bcd95a3SBarry Smith   and indices that may become large. `PetscIntCast()` should always be used when converting
73*4bcd95a3SBarry Smith   to `PetscInt` from `PetscCount`. Since, in most configurations an array of `PetscCount` requires twice the memory
74*4bcd95a3SBarry Smith   of an array of `PetscInt` most index arrays (such as `ISGetIndices()` use `PetscInt`,
75*4bcd95a3SBarry Smith   when these arrays get too large then `--with-64-bit-indices` must be used to
76*4bcd95a3SBarry Smith   `./configure` PETSc. In most cases it is appropriate to use `PetscCount` in lieu of `PetscInt64`.
77*4bcd95a3SBarry Smith
78*4bcd95a3SBarry Smith3. `size_t` is used for variables that contain the amount of memory, generally in bytes.
79*4bcd95a3SBarry Smith   It should **not** be used for the number of
80*4bcd95a3SBarry Smith   entries in an array, or to index into an array, that should be `PetscCount`, or `PetscInt`.
81*4bcd95a3SBarry Smith   Though `size_t` is unsigned and hence can have values larger then those that can be stored
82*4bcd95a3SBarry Smith   in a `PetscCount` those sizes will never be reached in practice so it is ok to cast with `(PetscCount)`
83*4bcd95a3SBarry Smith   from a `size_t` variable to a `PetscCount` variable, but **not** a `PetscInt`.
84*4bcd95a3SBarry Smith   One should not blindly cast from a `PetscCount` or a `PetscInt`
85*4bcd95a3SBarry Smith   to `size_t` since, when the value is negative, it will produce garbage.
86*4bcd95a3SBarry Smith
87*4bcd95a3SBarry Smith4. **Never** blindly put in a cast from a potentially longer (in number of bits) to a shorter integer such as
88*4bcd95a3SBarry Smith
89*4bcd95a3SBarry Smith   ```
90*4bcd95a3SBarry Smith   PetscInt64 a
91*4bcd95a3SBarry Smith   PetscInt b
92*4bcd95a3SBarry Smith   b = (PetscInt)a
93*4bcd95a3SBarry Smith   ```
94*4bcd95a3SBarry Smith
95*4bcd95a3SBarry Smith   simply to prevent a compiler warning. Use the appropriate PETSc cast function unless you
96*4bcd95a3SBarry Smith   absolutely know the value will fit in the lesser number of bits.
97*4bcd95a3SBarry Smith
98*4bcd95a3SBarry Smith5. MPI 4.0 supports the use of `MPI_Count` (large count) for many MPI functions that previously used `int` in a new API where the MPI function
99*4bcd95a3SBarry Smith   names end in `_c`. Since not all installed MPI implementations have such support, use `MPIU_XXX()` routines
100*4bcd95a3SBarry Smith   that use `PetscCount` for count arguments and use the large count MPI versions when possible.
101*4bcd95a3SBarry Smith   When not possible they first check the size of the input count arguments and error if they
102*4bcd95a3SBarry Smith   will not fit in the MPI required `int`, if they fit then the standard MPI functions are automatically called.
103*4bcd95a3SBarry Smith
104*4bcd95a3SBarry Smith   ```
105*4bcd95a3SBarry Smith   sizeof(MPI_Count) >= sizeof(PetscCount) >= sizeof(PetscInt)
106*4bcd95a3SBarry Smith   sizeof(PetscInt64) >= sizeof(PetscCount) >= sizeof(PetscInt)
107*4bcd95a3SBarry Smith   sizeof(MPI_Count) may be strictly greater than sizeof(PetscInt64)
108*4bcd95a3SBarry Smith   ```
109*4bcd95a3SBarry Smith
110*4bcd95a3SBarry Smith## Coding Conventions and Style
111*4bcd95a3SBarry Smith
112*4bcd95a3SBarry SmithWithin the PETSc source code, we adhere to the following guidelines so
113*4bcd95a3SBarry Smiththat the code is uniform and easily maintained.
114*4bcd95a3SBarry Smith
115*4bcd95a3SBarry Smith### C Formatting
116*4bcd95a3SBarry Smith
117*4bcd95a3SBarry SmithThe `.clang-format` file in the PETSc root directory controls the white space and basic layout. You can run the formatter in the entire repository with `make clangformat`. All merge requests must be properly formatted; this is automatically checked for merge requests with `make checkclangformat`.
118*4bcd95a3SBarry Smith
119*4bcd95a3SBarry SmithEven with the use of `clang-format` there are still many decisions about code formatting that must be constantly made. A subset of these is automatically checked for merge requests with `make checkbadSource`.
120*4bcd95a3SBarry Smith
121*4bcd95a3SBarry Smith01. The prototypes for functions should not include the names of the
122*4bcd95a3SBarry Smith    variables
123*4bcd95a3SBarry Smith
124*4bcd95a3SBarry Smith    ```
125*4bcd95a3SBarry Smith    PETSC_EXTERN PetscErrorCode MyFunction(PetscInt); // Correct
126*4bcd95a3SBarry Smith    PETSC_EXTERN PetscErrorCode MyFunction(PetscInt myvalue); // Incorrect
127*4bcd95a3SBarry Smith    ```
128*4bcd95a3SBarry Smith
129*4bcd95a3SBarry Smith02. All local variables of a particular type (for example, `PetscInt`) should be listed
130*4bcd95a3SBarry Smith    on the same line if possible; otherwise, they should be listed on adjacent lines. Note
131*4bcd95a3SBarry Smith    that pointers of different arity (levels of indirection) are considered to be different types. `clang-format` automatically
132*4bcd95a3SBarry Smith    handles the indenting shown below.
133*4bcd95a3SBarry Smith
134*4bcd95a3SBarry Smith    ```
135*4bcd95a3SBarry Smith    // Correct
136*4bcd95a3SBarry Smith    PetscInt   a, b, c;
137*4bcd95a3SBarry Smith    PetscInt  *d, *e;
138*4bcd95a3SBarry Smith    PetscInt **f;
139*4bcd95a3SBarry Smith
140*4bcd95a3SBarry Smith    // Incorrect
141*4bcd95a3SBarry Smith    PetscInt a, b, c, *d, *e, **f;
142*4bcd95a3SBarry Smith    ```
143*4bcd95a3SBarry Smith
144*4bcd95a3SBarry Smith03. Local variables should be initialized in their declaration when possible
145*4bcd95a3SBarry Smith
146*4bcd95a3SBarry Smith    ```
147*4bcd95a3SBarry Smith    // Correct
148*4bcd95a3SBarry Smith    PetscInt a = 11;
149*4bcd95a3SBarry Smith
150*4bcd95a3SBarry Smith    PetscFunctionBegin;
151*4bcd95a3SBarry Smith    // use a
152*4bcd95a3SBarry Smith
153*4bcd95a3SBarry Smith    // Incorrect
154*4bcd95a3SBarry Smith    PetscInt a;
155*4bcd95a3SBarry Smith
156*4bcd95a3SBarry Smith    PetscFunctionBegin;
157*4bcd95a3SBarry Smith    a = 11;
158*4bcd95a3SBarry Smith    // use a
159*4bcd95a3SBarry Smith    ```
160*4bcd95a3SBarry Smith
161*4bcd95a3SBarry Smith04. All PETSc subroutine code blocks *must* start with a single blank line between the local variable
162*4bcd95a3SBarry Smith    declarations followed by `PetscFunctionBegin`.
163*4bcd95a3SBarry Smith
164*4bcd95a3SBarry Smith    ```
165*4bcd95a3SBarry Smith    // Correct
166*4bcd95a3SBarry Smith    PetscInt x;
167*4bcd95a3SBarry Smith
168*4bcd95a3SBarry Smith    PetscFunctionBegin;
169*4bcd95a3SBarry Smith
170*4bcd95a3SBarry Smith    // Incorrect
171*4bcd95a3SBarry Smith    PetscInt x;
172*4bcd95a3SBarry Smith    PetscFunctionBegin;
173*4bcd95a3SBarry Smith
174*4bcd95a3SBarry Smith    // Incorrect
175*4bcd95a3SBarry Smith    PetscInt x;
176*4bcd95a3SBarry Smith    y = 11;
177*4bcd95a3SBarry Smith    ```
178*4bcd95a3SBarry Smith
179*4bcd95a3SBarry Smith05. Functions in PETSc examples, including `main()` should have `PetscFunctionBeginUser` as the first line after the local variable declarations.
180*4bcd95a3SBarry Smith
181*4bcd95a3SBarry Smith06. PETSc functions that begin `PetscFunctionBegin` must always return via `PetscFunctionReturn()`, or `PetscFunctionReturnVoid()`, not `return`. If the function returns a `PetscErrorCode`, then it must always return with `PetscFunctionReturn(PETSC_SUCCESS)`.
182*4bcd95a3SBarry Smith
183*4bcd95a3SBarry Smith07. Functions that do use return should use `return xx;` rather than `return(xx);`
184*4bcd95a3SBarry Smith
185*4bcd95a3SBarry Smith08. All PETSc function calls must have their return value checked for errors using the
186*4bcd95a3SBarry Smith    `PetscCall()` macro. This should be wrapped around the function in question.
187*4bcd95a3SBarry Smith
188*4bcd95a3SBarry Smith    ```
189*4bcd95a3SBarry Smith    PetscCall(MyFunction(...)); // Correct
190*4bcd95a3SBarry Smith    PetscErrorCode ierr = MyFunction(...);PetscCall(ierr); // Incorrect
191*4bcd95a3SBarry Smith    ```
192*4bcd95a3SBarry Smith
193*4bcd95a3SBarry Smith    The only exceptions to this rule are begin-end style macros which embed local variables
194*4bcd95a3SBarry Smith    or loops as part of their expansion
195*4bcd95a3SBarry Smith    (e.g. `PetscOptionsBegin()`/`PetscOptionsEnd()`). These handle errors internally
196*4bcd95a3SBarry Smith    and do not need error checking.
197*4bcd95a3SBarry Smith
198*4bcd95a3SBarry Smith    ```
199*4bcd95a3SBarry Smith    // Correct
200*4bcd95a3SBarry Smith    PetscOptionsBegin(...);
201*4bcd95a3SBarry Smith    PetscOptionsEnd();
202*4bcd95a3SBarry Smith    ```
203*4bcd95a3SBarry Smith
204*4bcd95a3SBarry Smith    As a rule, always try to wrap the function first; if this fails to compile, you do
205*4bcd95a3SBarry Smith    not need to add the error checking.
206*4bcd95a3SBarry Smith
207*4bcd95a3SBarry Smith    Calls to external package functions are generally made with `PetscCallExternal()` or its variants that are specialized for particular packages, for example `PetscCallBLAS()`
208*4bcd95a3SBarry Smith
209*4bcd95a3SBarry Smith09. Single operation `if` and `else` commands should not be wrapped in braces. They should be done as follows,
210*4bcd95a3SBarry Smith
211*4bcd95a3SBarry Smith    ```
212*4bcd95a3SBarry Smith    if ( ) XXXX;
213*4bcd95a3SBarry Smith    else YYY;
214*4bcd95a3SBarry Smith    ```
215*4bcd95a3SBarry Smith
216*4bcd95a3SBarry Smith    not
217*4bcd95a3SBarry Smith
218*4bcd95a3SBarry Smith    ```
219*4bcd95a3SBarry Smith    if ( ) {XXXX;}
220*4bcd95a3SBarry Smith    else {YYY;}
221*4bcd95a3SBarry Smith    ```
222*4bcd95a3SBarry Smith
223*4bcd95a3SBarry Smith10. Do not leave sections of commented-out code or dead source code protected with `ifdef foo` in the source files.
224*4bcd95a3SBarry Smith
225*4bcd95a3SBarry Smith11. Use classic block comments (`/* There must be a space before the first word in the comment and a space at the end */`,
226*4bcd95a3SBarry Smith    (`/*Do not do this*/`) for multi-line comments, and `// Comment` for single-line comments in source files.
227*4bcd95a3SBarry Smith
228*4bcd95a3SBarry Smith12. Do not put a `*` at the beginning or end of each line of a multi-line comment.
229*4bcd95a3SBarry Smith
230*4bcd95a3SBarry Smith13. Do not use `/* ---- ... ----- */` or similar constructs to separate parts of source code files.
231*4bcd95a3SBarry Smith
232*4bcd95a3SBarry Smith14. Use appropriate grammar and spelling in the comments.
233*4bcd95a3SBarry Smith
234*4bcd95a3SBarry Smith15. All variables must be declared at the beginning of the code block (C89
235*4bcd95a3SBarry Smith    style), never mixed in with code. However, when variables are only used in a limited
236*4bcd95a3SBarry Smith    scope, it is encouraged to declare them in that scope. For example:
237*4bcd95a3SBarry Smith
238*4bcd95a3SBarry Smith    ```
239*4bcd95a3SBarry Smith    if (cond) {
240*4bcd95a3SBarry Smith      PetscScalar *tmp;
241*4bcd95a3SBarry Smith
242*4bcd95a3SBarry Smith      PetscCall(PetscMalloc1(10, &tmp));
243*4bcd95a3SBarry Smith      // use tmp
244*4bcd95a3SBarry Smith      PetscCall(PetscFree(tmp));
245*4bcd95a3SBarry Smith    }
246*4bcd95a3SBarry Smith    ```
247*4bcd95a3SBarry Smith
248*4bcd95a3SBarry Smith    The only exception to this is variables used exclusively within a `for` loop, which must
249*4bcd95a3SBarry Smith    be declared inside the loop initializer:
250*4bcd95a3SBarry Smith
251*4bcd95a3SBarry Smith    ```
252*4bcd95a3SBarry Smith    // Correct
253*4bcd95a3SBarry Smith    for (PetscInt i = 0; i < n; ++i) {
254*4bcd95a3SBarry Smith      // loop body
255*4bcd95a3SBarry Smith    }
256*4bcd95a3SBarry Smith    ```
257*4bcd95a3SBarry Smith
258*4bcd95a3SBarry Smith    ```
259*4bcd95a3SBarry Smith    // Correct, variable used outside of loop
260*4bcd95a3SBarry Smith    PetscInt i;
261*4bcd95a3SBarry Smith    ```
262*4bcd95a3SBarry Smith
263*4bcd95a3SBarry Smith    ```
264*4bcd95a3SBarry Smith    for (i = 0; i < n; ++i) {
265*4bcd95a3SBarry Smith      // loop body
266*4bcd95a3SBarry Smith    }
267*4bcd95a3SBarry Smith    j = i;
268*4bcd95a3SBarry Smith    ```
269*4bcd95a3SBarry Smith
270*4bcd95a3SBarry Smith    ```
271*4bcd95a3SBarry Smith    // Incorrect
272*4bcd95a3SBarry Smith    PetscInt i;
273*4bcd95a3SBarry Smith    ...
274*4bcd95a3SBarry Smith    for (i = 0; i < n; ++i) {
275*4bcd95a3SBarry Smith      // loop body
276*4bcd95a3SBarry Smith    }
277*4bcd95a3SBarry Smith    ```
278*4bcd95a3SBarry Smith
279*4bcd95a3SBarry Smith16. Developers can use // to split very long lines when it improves code readability. For example
280*4bcd95a3SBarry Smith
281*4bcd95a3SBarry Smith    ```
282*4bcd95a3SBarry Smith    f[j][i].omega = xdot[j][i].omega + uxx + uyy //
283*4bcd95a3SBarry Smith                  + (vxp * (u - x[j][i - 1].omega) + vxm * (x[j][i + 1].omega - u)) * hy //
284*4bcd95a3SBarry Smith                  + (vyp * (u - x[j - 1][i].omega) + vym * (x[j + 1][i].omega - u)) * hx //
285*4bcd95a3SBarry Smith                  - .5 * grashof * (x[j][i + 1].temp - x[j][i - 1].temp) * hy;
286*4bcd95a3SBarry Smith    ```
287*4bcd95a3SBarry Smith
288*4bcd95a3SBarry Smith17. The use of `// clang-format off` is allowed in the source code but should only be used when necessary. It should not
289*4bcd95a3SBarry Smith    be used when trailing // to split lines works.
290*4bcd95a3SBarry Smith
291*4bcd95a3SBarry Smith    ```
292*4bcd95a3SBarry Smith    // clang-format off
293*4bcd95a3SBarry Smith    f ...
294*4bcd95a3SBarry Smith    // clang-format on
295*4bcd95a3SBarry Smith    ```
296*4bcd95a3SBarry Smith
297*4bcd95a3SBarry Smith18. `size` and `rank` should be used exclusively for the results of `MPI_Comm_size()` and `MPI_Comm_rank()` and other variable names for these values should be avoided unless necessary.
298*4bcd95a3SBarry Smith
299*4bcd95a3SBarry Smith### C Usage
300*4bcd95a3SBarry Smith
301*4bcd95a3SBarry Smith01. Do not use language features that are not in the intersection of C99, C++11, and MSVC
302*4bcd95a3SBarry Smith    v1900+ (Visual Studio 2015). Examples of such banned features include variable-length arrays.
303*4bcd95a3SBarry Smith    Note that variable-length arrays (including VLA-pointers) are not supported in C++ and
304*4bcd95a3SBarry Smith    were made optional in C11. You may use designated initializers via the
305*4bcd95a3SBarry Smith    `PetscDesignatedInitializer()` macro.
306*4bcd95a3SBarry Smith
307*4bcd95a3SBarry Smith02. Array and pointer arguments where the array values are not changed
308*4bcd95a3SBarry Smith    should be labeled as `const` arguments.
309*4bcd95a3SBarry Smith
310*4bcd95a3SBarry Smith03. Scalar values passed to functions should *never* be labeled as
311*4bcd95a3SBarry Smith    `const`.
312*4bcd95a3SBarry Smith
313*4bcd95a3SBarry Smith04. Subroutines that would normally have a `void **` argument to return
314*4bcd95a3SBarry Smith    a pointer to some data should be prototyped as `void *`.
315*4bcd95a3SBarry Smith    This prevents the caller from having to put a `(void **)` cast in
316*4bcd95a3SBarry Smith    each function call. See, for example, `DMDAVecGetArray()`.
317*4bcd95a3SBarry Smith
318*4bcd95a3SBarry Smith05. Do not use the `register` directive.
319*4bcd95a3SBarry Smith
320*4bcd95a3SBarry Smith06. Use `if (v == NULL)` or `if (flg == PETSC_TRUE)`, instead of using `if (!v)` or `if (flg)` or `if (!flg)`.
321*4bcd95a3SBarry Smith
322*4bcd95a3SBarry Smith07. Avoid `#ifdef` or `#ifndef` when possible. Rather, use `#if defined` or `#if
323*4bcd95a3SBarry Smith    !defined`. Better, use `PetscDefined()` (see below). The only exception to this
324*4bcd95a3SBarry Smith    rule is for header guards, where the `#ifndef` form is preferred (see below).
325*4bcd95a3SBarry Smith
326*4bcd95a3SBarry Smith08. Header guard macros should be done using `#pragma once`. This must be the very first
327*4bcd95a3SBarry Smith    non-comment line of the file. There must be no leading or trailing empty (non-comment)
328*4bcd95a3SBarry Smith    lines in the header. For example, do
329*4bcd95a3SBarry Smith
330*4bcd95a3SBarry Smith    ```
331*4bcd95a3SBarry Smith    /*
332*4bcd95a3SBarry Smith      It's OK to have
333*4bcd95a3SBarry Smith
334*4bcd95a3SBarry Smith      comments
335*4bcd95a3SBarry Smith    */
336*4bcd95a3SBarry Smith    // before the guard
337*4bcd95a3SBarry Smith    #pragma once
338*4bcd95a3SBarry Smith
339*4bcd95a3SBarry Smith    // OK, other headers included after the guard
340*4bcd95a3SBarry Smith    #include <petscdm.h>
341*4bcd95a3SBarry Smith    #include <petscdevice.h>
342*4bcd95a3SBarry Smith
343*4bcd95a3SBarry Smith    // OK, other preprocessor symbols defined after the guard
344*4bcd95a3SBarry Smith    #define FOO_BAR_BAZ
345*4bcd95a3SBarry Smith
346*4bcd95a3SBarry Smith    // OK, regular symbols defined after the guard
347*4bcd95a3SBarry Smith    typedef struct _p_PetscFoo *PetscFoo;
348*4bcd95a3SBarry Smith    ...
349*4bcd95a3SBarry Smith    ```
350*4bcd95a3SBarry Smith
351*4bcd95a3SBarry Smith    Do not do
352*4bcd95a3SBarry Smith
353*4bcd95a3SBarry Smith    ```
354*4bcd95a3SBarry Smith    // ERROR, empty lines at the beginning of the header
355*4bcd95a3SBarry Smith
356*4bcd95a3SBarry Smith
357*4bcd95a3SBarry Smith
358*4bcd95a3SBarry Smith    // ERROR, included other headers before the guard
359*4bcd95a3SBarry Smith    #include <petscdm.h>
360*4bcd95a3SBarry Smith    #include <petscdevice.h>
361*4bcd95a3SBarry Smith
362*4bcd95a3SBarry Smith    // ERROR, defined other preprocessor symbols before the guard
363*4bcd95a3SBarry Smith    #define FOO_BAR_BAZ
364*4bcd95a3SBarry Smith
365*4bcd95a3SBarry Smith    // ERROR, defined regular symbols before the guard
366*4bcd95a3SBarry Smith    typedef struct _p_PetscFoo *PetscFoo;
367*4bcd95a3SBarry Smith
368*4bcd95a3SBarry Smith    #pragma once
369*4bcd95a3SBarry Smith    ```
370*4bcd95a3SBarry Smith
371*4bcd95a3SBarry Smith09. Never use system random number generators such as `rand()` in PETSc
372*4bcd95a3SBarry Smith    code or examples because these can produce different results on
373*4bcd95a3SBarry Smith    different systems, thus making portability testing difficult. Instead,
374*4bcd95a3SBarry Smith    use `PetscRandom` which produces the same results regardless
375*4bcd95a3SBarry Smith    of the system used.
376*4bcd95a3SBarry Smith
377*4bcd95a3SBarry Smith10. Variadic macros may be used in PETSc, but must work with MSVC v1900+ (Visual Studio
378*4bcd95a3SBarry Smith    2015). Most compilers have conforming implementations of the C99/C++11 rules for
379*4bcd95a3SBarry Smith    `__VA_ARGS__`, but MSVC's implementation is not conforming and may need workarounds.
380*4bcd95a3SBarry Smith    See `PetscDefined()` for an example of how to work around MSVC's limitations to write
381*4bcd95a3SBarry Smith    a macro that is usable in both.
382*4bcd95a3SBarry Smith
383*4bcd95a3SBarry Smith(usage_of_petsc_functions_and_macros)=
384*4bcd95a3SBarry Smith
385*4bcd95a3SBarry Smith### Usage of PETSc Functions and Macros
386*4bcd95a3SBarry Smith
387*4bcd95a3SBarry Smith01. Lengthy conditional preprocessor blocks should mark any `#else` or `#endif`
388*4bcd95a3SBarry Smith    directives with a comment containing (or explaining) either the boolean condition or
389*4bcd95a3SBarry Smith    the macro's name if the first directive tests whether one is defined. One
390*4bcd95a3SBarry Smith    should be able to read any part of the macroblock and find or deduce the
391*4bcd95a3SBarry Smith    initial `#if`. That is:
392*4bcd95a3SBarry Smith
393*4bcd95a3SBarry Smith    ```
394*4bcd95a3SBarry Smith    #if defined(MY_MACRO)
395*4bcd95a3SBarry Smith    // many lines of code
396*4bcd95a3SBarry Smith    #else // MY_MACRO (use name of macro)
397*4bcd95a3SBarry Smith    // many more lines of code
398*4bcd95a3SBarry Smith    #endif // MY_MACRO
399*4bcd95a3SBarry Smith
400*4bcd95a3SBarry Smith    #if MY_MACRO > 10
401*4bcd95a3SBarry Smith    // code
402*4bcd95a3SBarry Smith    #else // MY_MACRO < 10
403*4bcd95a3SBarry Smith    // more code
404*4bcd95a3SBarry Smith    #endif // MY_MACRO > 10
405*4bcd95a3SBarry Smith    ```
406*4bcd95a3SBarry Smith
407*4bcd95a3SBarry Smith02. Public PETSc include files, `petsc*.h`, should not reference
408*4bcd95a3SBarry Smith    private PETSc `petsc/private/*impl.h` include files.
409*4bcd95a3SBarry Smith
410*4bcd95a3SBarry Smith03. Public and private PETSc include files cannot reference include files
411*4bcd95a3SBarry Smith    located in the PETSc source tree.
412*4bcd95a3SBarry Smith
413*4bcd95a3SBarry Smith04. All public functions must sanity-check their arguments using the appropriate
414*4bcd95a3SBarry Smith    `PetscValidXXX()` macros. These must appear between `PetscFunctionBegin` and
415*4bcd95a3SBarry Smith    `PetscFunctionReturn()` For example
416*4bcd95a3SBarry Smith
417*4bcd95a3SBarry Smith    ```
418*4bcd95a3SBarry Smith    PetscErrorCode PetscPublicFunction(Vec v, PetscScalar *array, PetscInt collectiveInt)
419*4bcd95a3SBarry Smith    {
420*4bcd95a3SBarry Smith      PetscFunctionBegin;
421*4bcd95a3SBarry Smith      PetscValidHeaderSpecific(v, VEC_CLASSID, 1);
422*4bcd95a3SBarry Smith      PetscAssertPointer(array, 2);
423*4bcd95a3SBarry Smith      PetscValidLogicalCollectiveInt(v, collectiveInt, 3);
424*4bcd95a3SBarry Smith      ...
425*4bcd95a3SBarry Smith      PetscFunctionReturn(PETSC_SUCCESS);
426*4bcd95a3SBarry Smith    }
427*4bcd95a3SBarry Smith    ```
428*4bcd95a3SBarry Smith
429*4bcd95a3SBarry Smith    See `include/petsc/private/petscimpl.h` and search for "PetscValid" to see all
430*4bcd95a3SBarry Smith    available checker macros.
431*4bcd95a3SBarry Smith
432*4bcd95a3SBarry Smith05. When possible, use `PetscDefined()` instead of preprocessor conditionals.
433*4bcd95a3SBarry Smith    For example, use:
434*4bcd95a3SBarry Smith
435*4bcd95a3SBarry Smith    ```
436*4bcd95a3SBarry Smith    if (PetscDefined(USE_DEBUG)) { ... }
437*4bcd95a3SBarry Smith    ```
438*4bcd95a3SBarry Smith
439*4bcd95a3SBarry Smith    instead of:
440*4bcd95a3SBarry Smith
441*4bcd95a3SBarry Smith    ```
442*4bcd95a3SBarry Smith    #if defined(PETSC_USE_DEBUG)
443*4bcd95a3SBarry Smith      ...
444*4bcd95a3SBarry Smith    #endif
445*4bcd95a3SBarry Smith    ```
446*4bcd95a3SBarry Smith
447*4bcd95a3SBarry Smith    The former usage allows syntax and type-checking in all configurations of
448*4bcd95a3SBarry Smith    PETSc, whereas the latter needs to be compiled with and without debugging
449*4bcd95a3SBarry Smith    to confirm that it compiles.
450*4bcd95a3SBarry Smith
451*4bcd95a3SBarry Smith06. *Never* put a function call in a `return` statement; do not write
452*4bcd95a3SBarry Smith
453*4bcd95a3SBarry Smith    ```
454*4bcd95a3SBarry Smith    PetscFunctionReturn( somefunction(...) ); /* Incorrect */
455*4bcd95a3SBarry Smith    ```
456*4bcd95a3SBarry Smith
457*4bcd95a3SBarry Smith07. Do *not* put a blank line immediately after `PetscFunctionBegin;`
458*4bcd95a3SBarry Smith    or a blank line immediately before `PetscFunctionReturn(PETSC_SUCCESS);`.
459*4bcd95a3SBarry Smith
460*4bcd95a3SBarry Smith08. Do not include `assert.h` in PETSc source code. Do not use
461*4bcd95a3SBarry Smith    `assert()`, it doesn’t play well in the parallel MPI world.
462*4bcd95a3SBarry Smith    You may use `PetscAssert()` where appropriate. See `PetscCall()` documentation
463*4bcd95a3SBarry Smith    for guidance of when to use `PetscCheck()` vs. `PetscAssert()`.
464*4bcd95a3SBarry Smith
465*4bcd95a3SBarry Smith09. Make error messages short but informative. The user should be able to reasonably
466*4bcd95a3SBarry Smith    diagnose the greater problem from your error message.
467*4bcd95a3SBarry Smith
468*4bcd95a3SBarry Smith10. Except in code that may be called before PETSc is fully initialized,
469*4bcd95a3SBarry Smith    always use `PetscMallocN()` (for example, `PetscMalloc1()`),
470*4bcd95a3SBarry Smith    `PetscCallocN()`, `PetscNew()`, and `PetscFree()`, not
471*4bcd95a3SBarry Smith    `malloc()` and `free()`.
472*4bcd95a3SBarry Smith
473*4bcd95a3SBarry Smith11. MPI routines and macros that are not part of the 2.1 standard
474*4bcd95a3SBarry Smith    should not be used in PETSc without appropriate `configure`
475*4bcd95a3SBarry Smith    checks and `#if PetscDefined()` checks. Code should also be provided
476*4bcd95a3SBarry Smith    that works if the MPI feature is not available; for example,
477*4bcd95a3SBarry Smith
478*4bcd95a3SBarry Smith    ```
479*4bcd95a3SBarry Smith    #if PetscDefined(HAVE_MPI_REDUCE_LOCAL)
480*4bcd95a3SBarry Smith      PetscCallMPI(MPI_Reduce_local(inbuf, inoutbuf, count, MPIU_INT, MPI_SUM));
481*4bcd95a3SBarry Smith    #else
482*4bcd95a3SBarry Smith      PetscCallMPI(MPI_Reduce(inbuf, inoutbuf, count, MPIU_INT, MPI_SUM, 0, PETSC_COMM_SELF);
483*4bcd95a3SBarry Smith    #endif
484*4bcd95a3SBarry Smith    ```
485*4bcd95a3SBarry Smith
486*4bcd95a3SBarry Smith12. Do not introduce PETSc routines that provide essentially the same
487*4bcd95a3SBarry Smith    functionality as an available MPI routine. For example, do not write
488*4bcd95a3SBarry Smith    a routine `PetscGlobalSum()` that takes a scalar value and performs
489*4bcd95a3SBarry Smith    an `MPI_Allreduce()` on it. Instead, use the MPI routine
490*4bcd95a3SBarry Smith    `MPI_Allreduce()` directly in the code.
491*4bcd95a3SBarry Smith
492*4bcd95a3SBarry Smith13. Never use a local variable counter such as `PetscInt flops = 0;` to
493*4bcd95a3SBarry Smith    accumulate flops and then call `PetscLogFlops();` *always* just
494*4bcd95a3SBarry Smith    call `PetscLogFlops()` directly when needed.
495*4bcd95a3SBarry Smith
496*4bcd95a3SBarry Smith14. Library symbols meant to be directly usable by the user should be declared
497*4bcd95a3SBarry Smith    `PETSC_EXTERN` in their respective public header file. Symbols intended for internal use should instead be declared `PETSC_INTERN`. Note that doing so is
498*4bcd95a3SBarry Smith    unnecessary in the case of symbols local to a single translation unit; these should
499*4bcd95a3SBarry Smith    be declared `static`. PETSc can be configured to build a separate shared
500*4bcd95a3SBarry Smith    library for each top-level class (`Mat`, `Vec`, `KSP`, and so on), and that plugin
501*4bcd95a3SBarry Smith    implementations of these classes can be included as separate shared libraries; thus,
502*4bcd95a3SBarry Smith    otherwise private symbols may need to be marked `PETSC_SINGLE_LIBRARY_INTERN`. For
503*4bcd95a3SBarry Smith    example
504*4bcd95a3SBarry Smith
505*4bcd95a3SBarry Smith    - `MatStashCreate_Private()` is marked `PETSC_INTERN` as it is used
506*4bcd95a3SBarry Smith      across compilation units, but only within the `Mat` package;
507*4bcd95a3SBarry Smith    - all functions, such as `KSPCreate()`, included in the public
508*4bcd95a3SBarry Smith      headers (`include/petsc*.h`) should be marked `PETSC_EXTERN`;
509*4bcd95a3SBarry Smith    - `VecLoad_Default()` is marked
510*4bcd95a3SBarry Smith      `PETSC_SINGLE_LIBRARY_INTERN` as it may be used across library boundaries, but is
511*4bcd95a3SBarry Smith      not intended to be visible to users;
512*4bcd95a3SBarry Smith
513*4bcd95a3SBarry Smith15. Before removing or renaming an API function, type, or enumerator,
514*4bcd95a3SBarry Smith    `PETSC_DEPRECATED_XXX()` should be used in the relevant header file
515*4bcd95a3SBarry Smith    to indicate the new usage and the PETSc version number where the
516*4bcd95a3SBarry Smith    deprecation will first appear. The old function or type, with the
517*4bcd95a3SBarry Smith    deprecation warning, should remain for at least one major release. We do not remove support for the
518*4bcd95a3SBarry Smith    deprecated functionality unless there is a specific reason to remove it; it is not removed simply because
519*4bcd95a3SBarry Smith    it has been deprecated for "a long time."
520*4bcd95a3SBarry Smith
521*4bcd95a3SBarry Smith    The function or type’s manual page should be updated (see {ref}`manual_page_format`).
522*4bcd95a3SBarry Smith    For example,
523*4bcd95a3SBarry Smith
524*4bcd95a3SBarry Smith    ```
525*4bcd95a3SBarry Smith    typedef NewType OldType PETSC_DEPRECATED_TYPEDEF("Use NewType (since version 3.9)");
526*4bcd95a3SBarry Smith
527*4bcd95a3SBarry Smith    PETSC_DEPRECATED_FUNCTION("Use NewFunction() (since version 3.9)") PetscErrorCode OldFunction();
528*4bcd95a3SBarry Smith
529*4bcd95a3SBarry Smith    #define OLD_ENUMERATOR_DEPRECATED  OLD_ENUMERATOR PETSC_DEPRECATED_ENUM("Use NEW_ENUMERATOR (since version 3.9)")
530*4bcd95a3SBarry Smith    typedef enum {
531*4bcd95a3SBarry Smith      OLD_ENUMERATOR_DEPRECATED = 3,
532*4bcd95a3SBarry Smith      NEW_ENUMERATOR = 3
533*4bcd95a3SBarry Smith    } MyEnum;
534*4bcd95a3SBarry Smith    ```
535*4bcd95a3SBarry Smith
536*4bcd95a3SBarry Smith    Note that after compiler preprocessing, the enum above would be transformed into something like
537*4bcd95a3SBarry Smith
538*4bcd95a3SBarry Smith    ```
539*4bcd95a3SBarry Smith    typedef enum {
540*4bcd95a3SBarry Smith      OLD_ENUMERATOR __attribute__((deprecated)) = 3,
541*4bcd95a3SBarry Smith      NEW_ENUMERATOR = 3
542*4bcd95a3SBarry Smith    } MyEnum;
543*4bcd95a3SBarry Smith    ```
544*4bcd95a3SBarry Smith
545*4bcd95a3SBarry Smith16. Before removing or renaming an options database key,
546*4bcd95a3SBarry Smith    `PetscOptionsDeprecated()` should be used for at least one major
547*4bcd95a3SBarry Smith    release. We do not remove support for the
548*4bcd95a3SBarry Smith    deprecated functionality unless there is a specific reason to remove it; it is not removed simply because
549*4bcd95a3SBarry Smith    it has been deprecated for "a long time."
550*4bcd95a3SBarry Smith
551*4bcd95a3SBarry Smith17. The format strings in PETSc ASCII output routines, such as
552*4bcd95a3SBarry Smith    `PetscPrintf()`, take a `%" PetscInt_FMT "` for all PETSc variables of type `PetscInt`,
553*4bcd95a3SBarry Smith    not a `%d`.
554*4bcd95a3SBarry Smith
555*4bcd95a3SBarry Smith18. All arguments of type `PetscReal` to PETSc ASCII output routines,
556*4bcd95a3SBarry Smith    such as `PetscPrintf`, must be cast to `double`, for example,
557*4bcd95a3SBarry Smith
558*4bcd95a3SBarry Smith    ```
559*4bcd95a3SBarry Smith    PetscPrintf(PETSC_COMM_WORLD, "Norm %g\n", (double)norm);
560*4bcd95a3SBarry Smith    ```
561*4bcd95a3SBarry Smith
562*4bcd95a3SBarry Smith## Formatted Comments
563*4bcd95a3SBarry Smith
564*4bcd95a3SBarry SmithPETSc uses formatted comments and the Sowing packages {cite}`gropp1993sowing` {cite}`gropp1993sowing2`
565*4bcd95a3SBarry Smithto generate documentation (manual pages) and the Fortran interfaces. Documentation
566*4bcd95a3SBarry Smithfor Sowing and the formatting may be found at
567*4bcd95a3SBarry Smith<http://wgropp.cs.illinois.edu/projects/software/sowing/>; in particular,
568*4bcd95a3SBarry Smithsee the documentation for `doctext`. Currently, doctext produces Markdown files ending in `.md`, which
569*4bcd95a3SBarry SmithSphinx later processes.
570*4bcd95a3SBarry Smith
571*4bcd95a3SBarry Smith- `/*@`
572*4bcd95a3SBarry Smith  a formatted comment of a function that will be used for documentation and a Fortran interface.
573*4bcd95a3SBarry Smith- `/*@C`
574*4bcd95a3SBarry Smith  a formatted comment of a function that will be used only for documentation, not to generate a Fortran interface. Certain constructs and usages do not yet support automatically generating a Fortran interface. In general, such labeled C functions should have a custom Fortran interface provided.
575*4bcd95a3SBarry Smith- `/*E`
576*4bcd95a3SBarry Smith  a formatted comment of an enum used for documentation only. Note that each of these needs to be listed in
577*4bcd95a3SBarry Smith  `lib/petsc/conf/bfort-petsc.txt`
578*4bcd95a3SBarry Smith   as a native and defined in the corresponding
579*4bcd95a3SBarry Smith  `include/petsc/finclude/petscxxx.h`
580*4bcd95a3SBarry Smith   Fortran include file and the values set as parameters in the file
581*4bcd95a3SBarry Smith  `src/SECTION/f90-mod/petscSUBSECTION.h`
582*4bcd95a3SBarry Smith  , for example,
583*4bcd95a3SBarry Smith  `src/vec/f90-mod/petscis.h`
584*4bcd95a3SBarry Smith  .
585*4bcd95a3SBarry Smith- `/*S`
586*4bcd95a3SBarry Smith  a formatted comment for a data type such as
587*4bcd95a3SBarry Smith  `KSP`
588*4bcd95a3SBarry Smith  . Each of these needs to be listed in
589*4bcd95a3SBarry Smith  `lib/petsc/conf/bfort-petsc.txt`
590*4bcd95a3SBarry Smith   as a
591*4bcd95a3SBarry Smith  `nativeptr`
592*4bcd95a3SBarry Smith  .
593*4bcd95a3SBarry Smith- `/*J`
594*4bcd95a3SBarry Smith  a formatted comment for a string type such as
595*4bcd95a3SBarry Smith  `KSPType`
596*4bcd95a3SBarry Smith  .
597*4bcd95a3SBarry Smith- `/*MC`
598*4bcd95a3SBarry Smith  a formatted comment of a CPP macro or enum value for documentation.
599*4bcd95a3SBarry Smith
600*4bcd95a3SBarry SmithThe Fortran interface files supplied manually by the developer go into the two
601*4bcd95a3SBarry Smithdirectories `ftn-custom` and `f90-custom`, while those generated by
602*4bcd95a3SBarry SmithSowing go into `ftn-auto` directories in the \$PETSC_ARCH/src\`\` directory tree.
603*4bcd95a3SBarry Smith
604*4bcd95a3SBarry SmithEach include file that contains formatted comments needs to have a line of the form
605*4bcd95a3SBarry Smith
606*4bcd95a3SBarry Smith> ```
607*4bcd95a3SBarry Smith> /* SUBMANSEC = submansec (for example Sys) */
608*4bcd95a3SBarry Smith> ```
609*4bcd95a3SBarry Smith
610*4bcd95a3SBarry Smithpreceded by and followed by a blank line. For source code, this information is found in the makefile in that source code's directory in the format
611*4bcd95a3SBarry Smith
612*4bcd95a3SBarry Smith> ```
613*4bcd95a3SBarry Smith> MANSEC   = DM
614*4bcd95a3SBarry Smith> SUBMANSEC= DMPlex
615*4bcd95a3SBarry Smith> ```
616*4bcd95a3SBarry Smith
617*4bcd95a3SBarry Smith(manual_page_format)=
618*4bcd95a3SBarry Smith
619*4bcd95a3SBarry Smith### Manual Page Format
620*4bcd95a3SBarry Smith
621*4bcd95a3SBarry SmithEach function, typedef, class, macro, enum, and so on in the public API
622*4bcd95a3SBarry Smithshould include the following data, correctly formatted (see codes
623*4bcd95a3SBarry Smithsection) to generate complete manual pages and (possibly) Fortran interfaces with
624*4bcd95a3SBarry SmithSowing. All entries below should be separated by blank lines. Except
625*4bcd95a3SBarry Smithwhere noted, add a newline after the section headings.
626*4bcd95a3SBarry Smith
627*4bcd95a3SBarry Smith01. The item’s name, followed by a dash and brief (one-sentence)
628*4bcd95a3SBarry Smith    description.
629*4bcd95a3SBarry Smith
630*4bcd95a3SBarry Smith02. If documenting a function implemented with a preprocessor macro
631*4bcd95a3SBarry Smith    (e.g., `PetscOptionsBegin()`), an explicit `Synopsis:` section
632*4bcd95a3SBarry Smith    noting the required header and the function signature.
633*4bcd95a3SBarry Smith
634*4bcd95a3SBarry Smith03. If documenting a function, a description of the function’s
635*4bcd95a3SBarry Smith    “collectivity”.
636*4bcd95a3SBarry Smith
637*4bcd95a3SBarry Smith    - `Not Collective` if the function need not be called on multiple (or possibly all) MPI
638*4bcd95a3SBarry Smith      processes
639*4bcd95a3SBarry Smith    - `Collective` if the function is a collective operation.
640*4bcd95a3SBarry Smith    - `Logically Collective; yyy must contain common value]`
641*4bcd95a3SBarry Smith      if the function is collective but does not require any actual
642*4bcd95a3SBarry Smith      synchronization (e.g., setting class parameters uniformly). Any
643*4bcd95a3SBarry Smith      argument yyy, which must have the same value on all ranks of the
644*4bcd95a3SBarry Smith      MPI communicator should be noted here.
645*4bcd95a3SBarry Smith
646*4bcd95a3SBarry Smith04. If the function is not supported in Fortran, then after the collective information, on the same line,
647*4bcd95a3SBarry Smith    one should provide `; No Fortran support`.
648*4bcd95a3SBarry Smith
649*4bcd95a3SBarry Smith05. If documenting a function with input parameters, a list of input
650*4bcd95a3SBarry Smith    parameter descriptions in an `Input Parameter(s):` section.
651*4bcd95a3SBarry Smith
652*4bcd95a3SBarry Smith06. If documenting a function with output parameters, a list of output
653*4bcd95a3SBarry Smith    parameter descriptions in an `Output Parameter(s):` section.
654*4bcd95a3SBarry Smith
655*4bcd95a3SBarry Smith07. If any input or output parameters are function pointers, they should be documented in the style
656*4bcd95a3SBarry Smith
657*4bcd95a3SBarry Smith    ```console
658*4bcd95a3SBarry Smith    Calling sequence of `func()`:
659*4bcd95a3SBarry Smith    . arg - the integer argument description
660*4bcd95a3SBarry Smith    ```
661*4bcd95a3SBarry Smith
662*4bcd95a3SBarry Smith08. If documenting a function that interacts with the options database, a
663*4bcd95a3SBarry Smith    list of options database keys in an `Options Database Key(s):`
664*4bcd95a3SBarry Smith    section.
665*4bcd95a3SBarry Smith
666*4bcd95a3SBarry Smith09. `Level:` (no newline) followed by `beginner`,
667*4bcd95a3SBarry Smith    `intermediate`, `advanced`, `developer`, or `deprecated`. This
668*4bcd95a3SBarry Smith    should be listed before the various `Note(s):` sections.
669*4bcd95a3SBarry Smith
670*4bcd95a3SBarry Smith10. (Optional) a `Note(s):` section containing in-depth discussion,
671*4bcd95a3SBarry Smith    technical caveats, special cases, and so on. If it is ambiguous
672*4bcd95a3SBarry Smith    whether returned pointers/objects need to be freed/destroyed by the
673*4bcd95a3SBarry Smith    user or not, this information should be mentioned here.
674*4bcd95a3SBarry Smith
675*4bcd95a3SBarry Smith11. (If applicable) a `Fortran Note(s):` section detailing any relevant
676*4bcd95a3SBarry Smith    differences in calling or using the item from Fortran.
677*4bcd95a3SBarry Smith
678*4bcd95a3SBarry Smith12. (If applicable) a `Developer Note(s):` section detailing any relevant
679*4bcd95a3SBarry Smith    information about the code for developers, for example, why a
680*4bcd95a3SBarry Smith    particular algorithm was implemented.
681*4bcd95a3SBarry Smith
682*4bcd95a3SBarry Smith13. (If applicable) references should be indicated inline with \{cite}\`Bibtex-key\` where
683*4bcd95a3SBarry Smith    Bibtex-key is in the file `doc/petsc.bib`, as in the manual page for `PCFIELDSPLIT`.
684*4bcd95a3SBarry Smith
685*4bcd95a3SBarry Smith14. `.seealso:` (no newline, no spaces to the left of this text), followed by a list of related manual
686*4bcd95a3SBarry Smith    pages. These manual pages should usually also point back to this
687*4bcd95a3SBarry Smith    manual page in their `seealso:` sections. This is the final entry in the
688*4bcd95a3SBarry Smith    comment. There should be no blank line after the `.seealso:` items.
689*4bcd95a3SBarry Smith
690*4bcd95a3SBarry Smith15. All PETSc functions that appear in a manual page (except the one in the header at the top) should end with a `()` and be enclosed
691*4bcd95a3SBarry Smith    in single back tick marks. All PETSc enum types and macros etc, should also be enclosed in single back tick marks.
692*4bcd95a3SBarry Smith    This includes each item listed in the `.seealso:` lines.
693*4bcd95a3SBarry Smith
694*4bcd95a3SBarry Smith[^footnote-1]: Type also refers to the string name of the subclass.
695*4bcd95a3SBarry Smith
696*4bcd95a3SBarry Smith### Spelling and Capitalization
697*4bcd95a3SBarry Smith
698*4bcd95a3SBarry Smith1. Proper nouns, including Unix, Linux, X Windows, and Microsoft Windows, should be fully written and capitalized. This includes all operating systems.
699*4bcd95a3SBarry Smith   The Apple computer operating system is written as macOS.
700*4bcd95a3SBarry Smith2. Company names and product names should be capitalized.
701*4bcd95a3SBarry Smith3. Company names and terms that are traditionally all capitalized, for example, MATLAB, NVIDIA, and CUDA, should be all capitalized.
702*4bcd95a3SBarry Smith4. ARM is a family of processor designs, while Arm is the company that licenses them.
703*4bcd95a3SBarry Smith5. Unix should not be all capitalized.
704*4bcd95a3SBarry Smith6. Microsoft Windows should always be written out with two words. That is, it should not be shortened to Windows or MS Win etc.
705*4bcd95a3SBarry Smith7. CMake should be capitalized as shown.
706*4bcd95a3SBarry Smith8. BLAS and LAPACK are written in full capitalization.
707*4bcd95a3SBarry Smith9. Open MPI is written as two words.
708*4bcd95a3SBarry Smith
709*4bcd95a3SBarry Smith## References
710*4bcd95a3SBarry Smith
711*4bcd95a3SBarry Smith```{bibliography} /petsc.bib
712*4bcd95a3SBarry Smith:filter: docname in docnames
713*4bcd95a3SBarry Smith```
714