Merge branch 'scott/test-fail' into 'master'

See merge request petsc/petsc!2296

This MR got merged to master (instead of maint) - so manually merging to maint now

Merge commit '6f5e9bd57b38f022b0f

Merge branch 'scott/test-fail' into 'master'

See merge request petsc/petsc!2296

This MR got merged to master (instead of maint) - so manually merging to maint now

Merge commit '6f5e9bd57b38f022b0f369f1410d1b0511a5a792' into maint

show more ...

Merge branch 'scott/test-fail' into 'master'

Make it easier to test the last failed tests

See merge request petsc/petsc!2296

Enable rich querying of tests

Tool for querying the tests.

Which tests to query? Two options:
1. Query only the tests that are run for a given configuration.
2. Query all of the test files

Enable rich querying of tests

Tool for querying the tests.

Which tests to query? Two options:
1. Query only the tests that are run for a given configuration.
2. Query all of the test files in the source directory
For #1:
Use dataDict as written out by gmakegentest.py in $PETSC_ARCH/$TESTBASE
For #2:
Walk the entire tree parsing the files as we go along using testparse.
The tree walker is simpler than what is in gmakegentest.py because it
can avoid all of the logic regarding configuration.
See documentation at the top of query_tests.py for further details.
The dataDict is further transformed to allow fast searching.

The query_test.py script by default outputs the info into a form that is
able to be used by gmakefile.test to enable this functionality:
make -f ${makefile} test query='requires' queryval='*MPI_PROCESS_SHARED_MEMORY*'

More rich querying can be done from within ipython. This is documented
in the developer's documentation.

show more ...

Remove dependency of test on check-test-errors

The CI system invokes check-test-errors to get the correct error code,
but it's not that nice for users.

Make it easier to test the last failed tests

Re-running the last failed tests were a pain because of how loops and
subtests were labelled with a suffix that started with "_" as a
delimiter. Because

Make it easier to test the last failed tests

Re-running the last failed tests were a pain because of how loops and
subtests were labelled with a suffix that started with "_" as a
delimiter. Because the underscore is commonly used by developers, this
created a non-unique mapping from test name to target name. The first
fix is to add in a delimiter for these cases ('+') so that unique
mapping from testname to target name can be created.

This can result in long string of targets to run, so report_tests.py
creates a silly little script to do nothing but echo the tests that
failed. A new search-like argument 'test-fail=1' can be invoked to
re-run all of the failed tests. The new syntax is:
make -f gmakefile test test-fail=1
To see which tests would be run:
make -f gmakefile print-test test-fail=1

show more ...

Merge remote-tracking branch 'origin/maint'

Merge branch 'balay/fix-CXXPPFLAGS-usage' into 'maint'

build: fix usage of preprocessor flags across languages

See merge request petsc/petsc!2159

build: fix usage of preprocessor flags across languages
- consistently set flags CPP_FLAGS, FPP_FLAGS, CXXPP_FLAGS, PCPP_FLAGS
- add PPPFLAGS, PFLAGS to map user-make-flags CFLAGS/CXXFLAGS or CPPFL

build: fix usage of preprocessor flags across languages
- consistently set flags CPP_FLAGS, FPP_FLAGS, CXXPP_FLAGS, PCPP_FLAGS
- add PPPFLAGS, PFLAGS to map user-make-flags CFLAGS/CXXFLAGS or CPPFLAGS/CXXPPFLAGS correctly to the PCC compile targets
- use PCPP_FLAGS with PCC targets and CXXPP_FLAGS with CXX targets

Notes
- CFLAGS/CPPFLAGS [API] for makefile does not override the the same variables specified to configure.
[CFLAGS/CPPFAGS to configure are saved in CC_FLAGS, CPP_FLAGS]
- For some reason CPPFLAGS gets added to both CC_FLAGS [via self.setCompilers.getCompilerFlags()] and CPP_FLAGS. Its not clear if one can be remoted
- We do not have a clean C compile target like we have for Cxx - due to --with-clanguage [hence PCC etc intertwined with PETSC_COMPILE..]

show more ...

Merge branch 'maint'

Merge remote-tracking branch 'origin/balay/fix-check-test-errors/maint' into maint

Balay/fix check test errors/maint

See merge request petsc/petsc!2199

revert change from 6e5deea7 as this breaks tests suite

Merge remote-tracking branch 'origin/maint'

Merge branch 'scott/misc-testharness-fixes' into 'maint'

Scott/misc testharness fixes

See merge request petsc/petsc!2138

Adding show-test functionality

Per the discussion on the petsc-dev mailing list, this shows the
tests that failed and how to run them from the CLI.

Simplify name of log files

Merge branch 'scott/test-improve' into 'master'

Various improvements to test harness

See merge request petsc/petsc!2024

Various improvements

1. Test logging now separates tap output and just the error output.
What is streamed is now controlled with the -o flag to the shell
scripts which now has two options curr

Various improvements

1. Test logging now separates tap output and just the error output.
What is streamed is now controlled with the -o flag to the shell
scripts which now has two options currently: 'interactive' and
'err_only'. The V=0 flag and OUTPUT=1 flags to gmakefile.test
turn on the err_only option
The OUTPUT=1 is default for gitlab

2. gmakegentest.py has a (-c,--check-output) option which does nothing
but check if the output files specified by tests are in the output
directories, and return a non-zero error code if they are not there
check_output target is in outer makefile and add `make check_output` to
stage zero tests to prevent running stage 2 and 3 with missing output
files

3. report_tests.py was not giving the correct command to invoke just the
files that failed. This is because it was coded assuming a
particular naming scheme for the tests: an assumption that was both
incorrect, and led to an ill-posed problem for fixing: If a test
label has underscores, it was not easy to fix. The solution was to
change to telling users to globsearch. In fixing, globsearch had to
be generalized to allow for multiple patterns: e.g.,
make -f gmakefile test globsearch='dm_impls_plex_tutorials-ex5_* ts_tutorials-ex11_*'

4. Bug fix for subtests with loop variables
(Reported Jakub Kruzik <jakub.kruzik@vsb.cz> on 6/26/19)

5. This does a minor fix in this utility script which prints out datafiles.

Commit-type: testing-fix, feature

Out check_output rule to outer makefile

Commit-type: feature
Reported-by: gitlab-ci

show more ...

Merge branch 'master' into this one.

Merge branch 'jed/gitlab-ci'

tests: create check-test-errors allgtests-tap [which ignores timeout errors] for gitlab-ci.

Also pass PETSC_ARCH/PETSC_DIR to gmakefile for allgtest, allgtest-tap, cleantest

fix allgtests-tap - as

tests: create check-test-errors allgtests-tap [which ignores timeout errors] for gitlab-ci.

Also pass PETSC_ARCH/PETSC_DIR to gmakefile for allgtest, allgtest-tap, cleantest

fix allgtests-tap - as it was not stopping on error

do not flag failure on test timeout

show more ...

Merge branch 'jed/gmakefile-include-by-goals' [PR #1905]

* jed/gmakefile-include-by-goals:
gmakefile: skip including generated files when MAKECMDGOALS is clean*

gmakefile: skip including generated files when MAKECMDGOALS is clean*

Suggested-by: Barry Smith <bsmith@mcs.anl.gov>

Merge branch 'barry/13-7-2019/fix-gmaketest-depend' [PR #1869]

* barry/13-7-2019/fix-gmaketest-depend:
Fix dependencies in gmakefile.test
Fix the error from gmakefile.test test about trying to remov

Merge branch 'barry/13-7-2019/fix-gmaketest-depend' [PR #1869]

* barry/13-7-2019/fix-gmaketest-depend:
Fix dependencies in gmakefile.test
Fix the error from gmakefile.test test about trying to remove a non-empty directory.
The problem was the target to rm -r the directory was running at the same time as tests tests where generatering new files in the directory.

show more ...

gmakefile.test: set prereqs for accurate cleaning and timing

Fix dependencies in gmakefile.test, fix proposed by Jed Brown

In 27d73d1f0a5c445a3a02971e31a2a1a02ed6d224 I broke gmakefile.test in attempting to eliminate the problem

Fix the error from gmakefile.

Fix dependencies in gmakefile.test, fix proposed by Jed Brown

In 27d73d1f0a5c445a3a02971e31a2a1a02ed6d224 I broke gmakefile.test in attempting to eliminate the problem

Fix the error from gmakefile.test test about trying to remove a non-empty directory
The problem was the target to rm -r the directory was running at the same time as
tests tests where generatering new files in the directory

My "fix" broke the dependency on test on the libraries (libraries no longer got rebuilt as needed)
and also likely caused the error:
/usr/bin/python: can't open file '/Users/petsc/petsc.master-2/petsc-install/config/report_tests.py': [Errno 2] No such file or directory
make[3]: [report_tests] Error 2 (ignored)

Commit-type: bug-fix
Reported-by: Matthew Knepley <knepley@gmail.com>
Thanks-to: Jed Brown <jed@jedbrown.org>

show more ...