| 50b47da0 | 19-Jul-2018 |
Adam Denchfield <adenchfi@hawk.iit.edu> |
diagonal Broyden matrix type framework added
Started implementing diagonal scaling using a common interface with Alp's code. Currently have forward B diagonal scaling, which means I need to derive S
diagonal Broyden matrix type framework added
Started implementing diagonal scaling using a common interface with Alp's code. Currently have forward B diagonal scaling, which means I need to derive SSML_BFGS in the forward case. Will apply this to case SSML_BFGS_Mod. Will also implement the inverse diagonal sometime.
Have basic DiagBrdn functionality working, though it isn't producing optimal ssml_bfgs output since the two are not super compatible - inverse (H) update with a forward (B) diagonal scaling?
Tried changing MatMult to Multiply instead of Divide, to see what effect it has. It may have a large effect...
Made it a developer option to change between Mult and Divide for the DiagBrdn MatMult. That way the BNCG code can have the correct behavior.
Added, hopefully successfully, my version of the inverse Hessian diagonal scaling to DiagBrdn. It can currently be enabled with command line options, but that will likely change.
Basic working code for using Alp's Mat objects. However, his and I's code still produce different results. To be investigated...
Removed the unnecessary mult parameter.
Trying out more reset strategies.
Committing before making experimental changes to SSML_BFGS_MOD where I directly update on the gradient.
Committing before making a change to SSML_BFGS that involves correcting a sign error.
Corrected a sign error in the diagonally scaled SSML_BFGS.
Nevermind, the sign was correct.
Implemented diagonal scaling for SSML_Broyden method.
Implemented diagonally scaled DFP.
Added trial versions of diagonally scaled Hestenes-Stiefel and Dai-Yuan methods.
Added trial version of diagonally scaled Fletcher-Reeves method.
Removed the testing version of SSML_BFGS; now there is only one final version.
Also added diagonally scaled Polar-Ribiere, trial version. About to try simplifying the diagonal update scheme in DiagBrdn to reduce numerical noise.
Tried adding features to the forward and backward updates to the diagonal scalinng that would reduce numerical noise.
Added a minor change to increase numerical precision of both forward and backward diagonal scaling updates.
Added restart conditions which would affect the Hestenes-Stiefel and Polar-Ribiere class of methods.
Removed the restart condition from the Hestenes-Stiefel method, as it appears to make it much worse, and going through the math shows it isn't necessarily the best restart strategy.
Removed the restarts based on gradient orthogonality, since those appear to be attempting to prevent non-descent directions, which is already checked for.
Fixed Fletcher-Reeves' diagonal preconditioning to match that written by HZ's survey.
Trying a new approach in Dai-Yuan inspired by the HZ survey.
The Preconditioning of FR in HZ's survey is wrong for our diagonal preconditioner. Just makes it worse. Reverted Dai0Yuan to a form where only the gradient is diagonally scaled.
Made global the check for whether or not the current direction is already good, enabling us to skip a direction update.
Also added diagonally scaled gradient descent.
Fixed a bug where the rescaling for scalar classic methods was constant.
Fixed a bug where the classic methods all seemed to be producing the same results. Had to initialize cg_continue.
Fixed issue in PR and PRP scalar methods.
Added some minor changes.
Cleaned up some redundancies in the BNCG code. Added some constants as command line options, user-options for the HZ and DK methods.
Cleaned a little more redundancy.
Some comments removed, a couple more redundant lines removed. Found an uninitialized variable in DaiYuan and fixed that; could be why DY was performing so badly.
Addressed a few of Alp's comments.
Trying to clean up memory leaks. First set lmvm->m = 1 in DiagBrdn.
Fixed memory leak problems.
Allowed the epsilon cutoff for curvature checks to be changed by the user. Defaults to machine precision.
Edited a couple things in DiagBrdn. Added the KD-like scaling to DiagBrdn for the inverse.
Merged with another branch that developed KD-like changes to diagonal scaling, and edited some comments.
Added a comment on the diagonal scaling as applied to FR, PR, PRP, DY methods. Beginning work on implementing them properly.
Got rid of the unused variables in DiagBrdn.c
Fixed an uninitialized variable in SSML_DFP
Fixed fortran interface bug.
Added a line about DiagBrdn in the PETSC Users Manual.
Trying to fix the DY diagonal scaling.
Currently at test2 of DY, does worse than initial test. About to start test 3 - where only g is replaced by its preconditioned counterpart.
Temporarily took out taking the reciprocal in the rescaling part of the code where we calculate ytDy, etc
Now onto test4 for the DY diagonally scaled method. Test3 worked well, in previous commit. By that I mean it almost got to the scalar performance.
show more ...
|
| 539b0084 | 07-Aug-2018 |
Karl Rupp <me@karlrupp.net> |
Merge branch 'denera/petsc-submat-virtual-shift' [PR #1062]
* denera/petsc-submat-virtual-shift:
MatShift support for MATSUBMAT.
Tao Newton-LS method does shifting on the Hessian matrix to guarante
Merge branch 'denera/petsc-submat-virtual-shift' [PR #1062]
* denera/petsc-submat-virtual-shift:
MatShift support for MATSUBMAT.
Tao Newton-LS method does shifting on the Hessian matrix to guarantee positive-definitiveness in degenerate cases.
This shifting breaks down when the user-provided Hessian is matrix-free (MATSHELL), because the active-set reduced Hessian comes out as MATSUBMAT type, which does not support the shift operation.
This PR clones the virtual shift from MATSHELL onto MATSUBMAT.
New ex99 test added under the Mat package (copied from ex97) to compare shifted MATSUBMAT operations against shifted MATAIJ.
Previously implemented safeguards in TAO BNK against MATSUBMAT shifts removed.
show more ...
|