Merge branch 'jed/gmakegen-with-file' into 'master'

gmakegen*.py: use with statement for file handling

See merge request petsc/petsc!2654

gmakegen*.py: use with statement for file handling

Better handling of error conditions and potentially lingering file
handles.

Merge branch 'jed/promote-examples-tests-tutorials' [petsc/petsc!2610]

* jed/promote-examples-tests-tutorials:
Promote examples/{tests,tutorials}/ to {tests,tutorials}/

Promote examples/{tests,tutorials}/ to {tests,tutorials}/

This shortens paths and improves consistency between test target names
and paths to the source and output files. Most of the work was
compl

Promote examples/{tests,tutorials}/ to {tests,tutorials}/

This shortens paths and improves consistency between test target names
and paths to the source and output files. Most of the work was
completed by this script, followed by mild cleanup of nonconforming
cases.

for makefile in `git ls-files 'src/*makefile'`; do
if rg -q 'DIRS.*\bexamples\b' $makefile; then
base=$(dirname $makefile)
dirs=$(cd $base/examples && ls -d tests tutorials 2>/dev/null | xargs echo)
perl -pi -e "s#^(DIRS.*)\bexamples\b#\1${dirs}#" $makefile
git rm $base/examples/makefile
for t in $dirs; do
git mv $base/examples/$t $base/
perl -pi -e "s#^(LOCDIR[[:space:]]*=).*#\1 $base/$t/#" $base/$t
done
fi
done

git grep -l -E -z 'examples/(tutorials|tests)' | xargs -0 perl -pi -e 's#examples/(tutorials|tests)#\1#g'
git checkout @ \
src/docs/website/documentation/changes/ \
src/benchmarks/results/

show more ...

Merge remote-tracking branch 'origin/maint'

Merge branch 'scott/fix-subtests-uni' into 'maint'

Scott/fix subtests uni

See merge request petsc/petsc!2602

Fix for parallel subtests with serial builds

Merge remote-tracking branch 'origin/maint'

Merge branch 'scott/test-fix-resub' into 'maint'

Avoid re.sub interpreting special chars

See merge request petsc/petsc!2403

Avoid re.sub interpreting special chars

Reported by: @joseroman

Merge remote-tracking branch 'origin/maint'

Merge branch 'scott/test-diff-filter' into 'maint'

Move to new filter/filter_output method

See merge request petsc/petsc!2329

Move to new filter/filter_output method

Change to using filters as arguments to petscdiff. By moving the
manipulation of the input and output into petscdiff, this fixes problems
in using piped comm

Move to new filter/filter_output method

Change to using filters as arguments to petscdiff. By moving the
manipulation of the input and output into petscdiff, this fixes problems
in using piped commands (based on original, ancient test system), which
is problematic in capturing the right error codes. This also required
fixes for some tests: the "filter: Error: ..." is used to test stderr of
tests.

This fixes problems with REPLACE=1 as reported by @knepley and debugged by @sbalay.

show more ...

Merge branch 'scott/test-fail' into 'master'

See merge request petsc/petsc!2296

This MR got merged to master (instead of maint) - so manually merging to maint now

Merge commit '6f5e9bd57b38f022b0f

Merge branch 'scott/test-fail' into 'master'

See merge request petsc/petsc!2296

This MR got merged to master (instead of maint) - so manually merging to maint now

Merge commit '6f5e9bd57b38f022b0f369f1410d1b0511a5a792' into maint

show more ...

Merge branch 'scott/test-fail' into 'master'

Make it easier to test the last failed tests

See merge request petsc/petsc!2296

Enable rich querying of tests

Tool for querying the tests.

Which tests to query? Two options:
1. Query only the tests that are run for a given configuration.
2. Query all of the test files

Enable rich querying of tests

Tool for querying the tests.

Which tests to query? Two options:
1. Query only the tests that are run for a given configuration.
2. Query all of the test files in the source directory
For #1:
Use dataDict as written out by gmakegentest.py in $PETSC_ARCH/$TESTBASE
For #2:
Walk the entire tree parsing the files as we go along using testparse.
The tree walker is simpler than what is in gmakegentest.py because it
can avoid all of the logic regarding configuration.
See documentation at the top of query_tests.py for further details.
The dataDict is further transformed to allow fast searching.

The query_test.py script by default outputs the info into a form that is
able to be used by gmakefile.test to enable this functionality:
make -f ${makefile} test query='requires' queryval='*MPI_PROCESS_SHARED_MEMORY*'

More rich querying can be done from within ipython. This is documented
in the developer's documentation.

show more ...

Make it easier to test the last failed tests

Re-running the last failed tests were a pain because of how loops and
subtests were labelled with a suffix that started with "_" as a
delimiter. Because

Make it easier to test the last failed tests

Re-running the last failed tests were a pain because of how loops and
subtests were labelled with a suffix that started with "_" as a
delimiter. Because the underscore is commonly used by developers, this
created a non-unique mapping from test name to target name. The first
fix is to add in a delimiter for these cases ('+') so that unique
mapping from testname to target name can be created.

This can result in long string of targets to run, so report_tests.py
creates a silly little script to do nothing but echo the tests that
failed. A new search-like argument 'test-fail=1' can be invoked to
re-run all of the failed tests. The new syntax is:
make -f gmakefile test test-fail=1
To see which tests would be run:
make -f gmakefile print-test test-fail=1

show more ...

Merge branch 'scott/fix_check_output' into 'master'

Fix for check_output

See merge request petsc/petsc!2088

Fix for check_output

Reported by Vaclav Hapla in Issue 376

Merge branch 'scott/test-improve' into 'master'

Various improvements to test harness

See merge request petsc/petsc!2024

Various improvements

1. Test logging now separates tap output and just the error output.
What is streamed is now controlled with the -o flag to the shell
scripts which now has two options curr

Various improvements

1. Test logging now separates tap output and just the error output.
What is streamed is now controlled with the -o flag to the shell
scripts which now has two options currently: 'interactive' and
'err_only'. The V=0 flag and OUTPUT=1 flags to gmakefile.test
turn on the err_only option
The OUTPUT=1 is default for gitlab

2. gmakegentest.py has a (-c,--check-output) option which does nothing
but check if the output files specified by tests are in the output
directories, and return a non-zero error code if they are not there
check_output target is in outer makefile and add `make check_output` to
stage zero tests to prevent running stage 2 and 3 with missing output
files

3. report_tests.py was not giving the correct command to invoke just the
files that failed. This is because it was coded assuming a
particular naming scheme for the tests: an assumption that was both
incorrect, and led to an ill-posed problem for fixing: If a test
label has underscores, it was not easy to fix. The solution was to
change to telling users to globsearch. In fixing, globsearch had to
be generalized to allow for multiple patterns: e.g.,
make -f gmakefile test globsearch='dm_impls_plex_tutorials-ex5_* ts_tutorials-ex11_*'

4. Bug fix for subtests with loop variables
(Reported Jakub Kruzik <jakub.kruzik@vsb.cz> on 6/26/19)

5. This does a minor fix in this utility script which prints out datafiles.

Commit-type: testing-fix, feature

Out check_output rule to outer makefile

Commit-type: feature
Reported-by: gitlab-ci

show more ...

Merge branch 'master' into this one.

Merge branch 'jed/gitlab-ci'

genRunScript() should close the script files it creates before setting the ch.mod()

Commit-type: bug-fix

Merge branch 'jose/harness-requires-debug' [PR #1854]

* jose/harness-requires-debug:
Test harness: allow 'debug' as a requirement