1*55a74a43SLisandro Dalcin 2*55a74a43SLisandro Dalcin''' 3*55a74a43SLisandro DalcinEx2 from PETSc example files implemented for PETSc4py. 4*55a74a43SLisandro Dalcinhttps://petsc.org/release/src/ksp/ksp/tutorials/ex2.c.html 5*55a74a43SLisandro DalcinBy: Miguel Arriaga 6*55a74a43SLisandro Dalcin 7*55a74a43SLisandro Dalcin 8*55a74a43SLisandro DalcinSolves a linear system in parallel with KSP. 9*55a74a43SLisandro DalcinInput parameters include: 10*55a74a43SLisandro Dalcin -view_exact_sol : write exact solution vector to stdout 11*55a74a43SLisandro Dalcin -m <mesh_x> : number of mesh points in x-direction 12*55a74a43SLisandro Dalcin -n <mesh_n> : number of mesh points in y-direction 13*55a74a43SLisandro Dalcin 14*55a74a43SLisandro Dalcin 15*55a74a43SLisandro DalcinVec x,b,u; # approx solution, RHS, exact solution 16*55a74a43SLisandro DalcinMat A; # linear system matrix 17*55a74a43SLisandro DalcinKSP ksp; # linear solver context 18*55a74a43SLisandro DalcinPetscReal norm; # norm of solution error 19*55a74a43SLisandro Dalcin''' 20*55a74a43SLisandro Dalcinimport sys 21*55a74a43SLisandro Dalcinimport petsc4py 22*55a74a43SLisandro Dalcinpetsc4py.init(sys.argv) 23*55a74a43SLisandro Dalcinfrom petsc4py import PETSc 24*55a74a43SLisandro Dalcin 25*55a74a43SLisandro Dalcinimport numpy as np 26*55a74a43SLisandro Dalcin 27*55a74a43SLisandro Dalcincomm = PETSc.COMM_WORLD 28*55a74a43SLisandro Dalcinsize = comm.getSize() 29*55a74a43SLisandro Dalcinrank = comm.getRank() 30*55a74a43SLisandro Dalcin 31*55a74a43SLisandro DalcinOptDB = PETSc.Options() 32*55a74a43SLisandro Dalcinm = OptDB.getInt('m', 8) 33*55a74a43SLisandro Dalcinn = OptDB.getInt('n', 7) 34*55a74a43SLisandro Dalcin 35*55a74a43SLisandro Dalcin''' - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 36*55a74a43SLisandro Dalcin Compute the matrix and right-hand-side vector that define 37*55a74a43SLisandro Dalcin the linear system, Ax = b. 38*55a74a43SLisandro Dalcin - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ''' 39*55a74a43SLisandro Dalcin''' 40*55a74a43SLisandro Dalcin Create parallel matrix, specifying only its global dimensions. 41*55a74a43SLisandro Dalcin When using MatCreate(), the matrix format can be specified at 42*55a74a43SLisandro Dalcin runtime. Also, the parallel partitioning of the matrix is 43*55a74a43SLisandro Dalcin determined by PETSc at runtime. 44*55a74a43SLisandro Dalcin 45*55a74a43SLisandro Dalcin Performance tuning note: For problems of substantial size, 46*55a74a43SLisandro Dalcin preallocation of matrix memory is crucial for attaining good 47*55a74a43SLisandro Dalcin performance. See the matrix chapter of the users manual for details. 48*55a74a43SLisandro Dalcin''' 49*55a74a43SLisandro Dalcin 50*55a74a43SLisandro DalcinA = PETSc.Mat().create(comm=comm) 51*55a74a43SLisandro DalcinA.setSizes((m*n,m*n)) 52*55a74a43SLisandro DalcinA.setFromOptions() 53*55a74a43SLisandro DalcinA.setPreallocationNNZ((5,5)) 54*55a74a43SLisandro Dalcin 55*55a74a43SLisandro Dalcin''' 56*55a74a43SLisandro Dalcin Currently, all PETSc parallel matrix formats are partitioned by 57*55a74a43SLisandro Dalcin contiguous chunks of rows across the processors. Determine which 58*55a74a43SLisandro Dalcin rows of the matrix are locally owned. 59*55a74a43SLisandro Dalcin''' 60*55a74a43SLisandro DalcinIstart,Iend = A.getOwnershipRange() 61*55a74a43SLisandro Dalcin 62*55a74a43SLisandro Dalcin''' 63*55a74a43SLisandro Dalcin Set matrix elements for the 2-D, five-point stencil in parallel. 64*55a74a43SLisandro Dalcin - Each processor needs to insert only elements that it owns 65*55a74a43SLisandro Dalcin locally (but any non-local elements will be sent to the 66*55a74a43SLisandro Dalcin appropriate processor during matrix assembly). 67*55a74a43SLisandro Dalcin - Always specify global rows and columns of matrix entries. 68*55a74a43SLisandro Dalcin 69*55a74a43SLisandro Dalcin Note: this uses the less common natural ordering that orders first 70*55a74a43SLisandro Dalcin all the unknowns for x = h then for x = 2h etc; Hence you see J = Ii +- n 71*55a74a43SLisandro Dalcin instead of J = I +- m as you might expect. The more standard ordering 72*55a74a43SLisandro Dalcin would first do all variables for y = h, then y = 2h etc. 73*55a74a43SLisandro Dalcin''' 74*55a74a43SLisandro Dalcin 75*55a74a43SLisandro Dalcinfor Ii in range(Istart,Iend): 76*55a74a43SLisandro Dalcin v = -1.0 77*55a74a43SLisandro Dalcin i = int(Ii/n) 78*55a74a43SLisandro Dalcin j = int(Ii - i*n) 79*55a74a43SLisandro Dalcin 80*55a74a43SLisandro Dalcin if (i>0): 81*55a74a43SLisandro Dalcin J = Ii - n 82*55a74a43SLisandro Dalcin A.setValues(Ii,J,v,addv=True) 83*55a74a43SLisandro Dalcin if (i<m-1): 84*55a74a43SLisandro Dalcin J = Ii + n 85*55a74a43SLisandro Dalcin A.setValues(Ii,J,v,addv=True) 86*55a74a43SLisandro Dalcin if (j>0): 87*55a74a43SLisandro Dalcin J = Ii - 1 88*55a74a43SLisandro Dalcin A.setValues(Ii,J,v,addv=True) 89*55a74a43SLisandro Dalcin if (j<n-1): 90*55a74a43SLisandro Dalcin J = Ii + 1 91*55a74a43SLisandro Dalcin A.setValues(Ii,J,v,addv=True) 92*55a74a43SLisandro Dalcin 93*55a74a43SLisandro Dalcin v = 4.0 94*55a74a43SLisandro Dalcin A.setValues(Ii,Ii,v,addv=True) 95*55a74a43SLisandro Dalcin 96*55a74a43SLisandro Dalcin''' 97*55a74a43SLisandro Dalcin Assemble matrix, using the 2-step process: 98*55a74a43SLisandro Dalcin MatAssemblyBegin(), MatAssemblyEnd() 99*55a74a43SLisandro Dalcin Computations can be done while messages are in transition 100*55a74a43SLisandro Dalcin by placing code between these two statements. 101*55a74a43SLisandro Dalcin''' 102*55a74a43SLisandro Dalcin 103*55a74a43SLisandro DalcinA.assemblyBegin(A.AssemblyType.FINAL) 104*55a74a43SLisandro DalcinA.assemblyEnd(A.AssemblyType.FINAL) 105*55a74a43SLisandro Dalcin''' A is symmetric. Set symmetric flag to enable ICC/Cholesky preconditioner ''' 106*55a74a43SLisandro Dalcin 107*55a74a43SLisandro DalcinA.setOption(A.Option.SYMMETRIC,True) 108*55a74a43SLisandro Dalcin 109*55a74a43SLisandro Dalcin''' 110*55a74a43SLisandro Dalcin Create parallel vectors. 111*55a74a43SLisandro Dalcin - We form 1 vector from scratch and then duplicate as needed. 112*55a74a43SLisandro Dalcin - When using VecCreate(), VecSetSizes and VecSetFromOptions() 113*55a74a43SLisandro Dalcin in this example, we specify only the 114*55a74a43SLisandro Dalcin vector's global dimension; the parallel partitioning is determined 115*55a74a43SLisandro Dalcin at runtime. 116*55a74a43SLisandro Dalcin - When solving a linear system, the vectors and matrices MUST 117*55a74a43SLisandro Dalcin be partitioned accordingly. PETSc automatically generates 118*55a74a43SLisandro Dalcin appropriately partitioned matrices and vectors when MatCreate() 119*55a74a43SLisandro Dalcin and VecCreate() are used with the same communicator. 120*55a74a43SLisandro Dalcin - The user can alternatively specify the local vector and matrix 121*55a74a43SLisandro Dalcin dimensions when more sophisticated partitioning is needed 122*55a74a43SLisandro Dalcin (replacing the PETSC_DECIDE argument in the VecSetSizes() statement 123*55a74a43SLisandro Dalcin below). 124*55a74a43SLisandro Dalcin''' 125*55a74a43SLisandro Dalcin 126*55a74a43SLisandro Dalcinu = PETSc.Vec().create(comm=comm) 127*55a74a43SLisandro Dalcinu.setSizes(m*n) 128*55a74a43SLisandro Dalcinu.setFromOptions() 129*55a74a43SLisandro Dalcin 130*55a74a43SLisandro Dalcinb = u.duplicate() 131*55a74a43SLisandro Dalcinx = b.duplicate() 132*55a74a43SLisandro Dalcin 133*55a74a43SLisandro Dalcin''' 134*55a74a43SLisandro Dalcin Set exact solution; then compute right-hand-side vector. 135*55a74a43SLisandro Dalcin By default we use an exact solution of a vector with all 136*55a74a43SLisandro Dalcin elements of 1.0; 137*55a74a43SLisandro Dalcin''' 138*55a74a43SLisandro Dalcinu.set(1.0) 139*55a74a43SLisandro Dalcinb = A(u) 140*55a74a43SLisandro Dalcin 141*55a74a43SLisandro Dalcin''' 142*55a74a43SLisandro Dalcin View the exact solution vector if desired 143*55a74a43SLisandro Dalcin''' 144*55a74a43SLisandro Dalcinflg = OptDB.getBool('view_exact_sol', False) 145*55a74a43SLisandro Dalcinif flg: 146*55a74a43SLisandro Dalcin u.view() 147*55a74a43SLisandro Dalcin 148*55a74a43SLisandro Dalcin''' - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 149*55a74a43SLisandro Dalcin Create the linear solver and set various options 150*55a74a43SLisandro Dalcin - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ''' 151*55a74a43SLisandro Dalcinksp = PETSc.KSP().create(comm=comm) 152*55a74a43SLisandro Dalcin 153*55a74a43SLisandro Dalcin''' 154*55a74a43SLisandro Dalcin Set operators. Here the matrix that defines the linear system 155*55a74a43SLisandro Dalcin also serves as the preconditioning matrix. 156*55a74a43SLisandro Dalcin''' 157*55a74a43SLisandro Dalcinksp.setOperators(A,A) 158*55a74a43SLisandro Dalcin 159*55a74a43SLisandro Dalcin''' 160*55a74a43SLisandro Dalcin Set linear solver defaults for this problem (optional). 161*55a74a43SLisandro Dalcin - By extracting the KSP and PC contexts from the KSP context, 162*55a74a43SLisandro Dalcin we can then directly call any KSP and PC routines to set 163*55a74a43SLisandro Dalcin various options. 164*55a74a43SLisandro Dalcin - The following two statements are optional; all of these 165*55a74a43SLisandro Dalcin parameters could alternatively be specified at runtime via 166*55a74a43SLisandro Dalcin KSPSetFromOptions(). All of these defaults can be 167*55a74a43SLisandro Dalcin overridden at runtime, as indicated below. 168*55a74a43SLisandro Dalcin''' 169*55a74a43SLisandro Dalcinrtol=1.e-2/((m+1)*(n+1)) 170*55a74a43SLisandro Dalcinksp.setTolerances(rtol=rtol,atol=1.e-50) 171*55a74a43SLisandro Dalcin 172*55a74a43SLisandro Dalcin''' 173*55a74a43SLisandro DalcinSet runtime options, e.g., 174*55a74a43SLisandro Dalcin -ksp_type <type> -pc_type <type> -ksp_monitor -ksp_rtol <rtol> 175*55a74a43SLisandro DalcinThese options will override those specified above as long as 176*55a74a43SLisandro DalcinKSPSetFromOptions() is called _after_ any other customization 177*55a74a43SLisandro Dalcinroutines. 178*55a74a43SLisandro Dalcin''' 179*55a74a43SLisandro Dalcinksp.setFromOptions() 180*55a74a43SLisandro Dalcin 181*55a74a43SLisandro Dalcin''' - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 182*55a74a43SLisandro Dalcin Solve the linear system 183*55a74a43SLisandro Dalcin - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ''' 184*55a74a43SLisandro Dalcin 185*55a74a43SLisandro Dalcinksp.solve(b,x) 186*55a74a43SLisandro Dalcin 187*55a74a43SLisandro Dalcin''' - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 188*55a74a43SLisandro Dalcin Check the solution and clean up 189*55a74a43SLisandro Dalcin - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ''' 190*55a74a43SLisandro Dalcinx = x - u # x.axpy(-1.0,u) 191*55a74a43SLisandro Dalcinnorm = x.norm(PETSc.NormType.NORM_2) 192*55a74a43SLisandro Dalcinits = ksp.getIterationNumber() 193*55a74a43SLisandro Dalcin 194*55a74a43SLisandro Dalcin''' 195*55a74a43SLisandro Dalcin Print convergence information. PetscPrintf() produces a single 196*55a74a43SLisandro Dalcin print statement from all processes that share a communicator. 197*55a74a43SLisandro Dalcin An alternative is PetscFPrintf(), which prints to a file. 198*55a74a43SLisandro Dalcin''' 199*55a74a43SLisandro Dalcinif norm > rtol*10: 200*55a74a43SLisandro Dalcin PETSc.Sys.Print("Norm of error {}, Iterations {}".format(norm,its),comm=comm) 201*55a74a43SLisandro Dalcinelse: 202*55a74a43SLisandro Dalcin if size==1: 203*55a74a43SLisandro Dalcin PETSc.Sys.Print("- Serial OK",comm=comm) 204*55a74a43SLisandro Dalcin else: 205*55a74a43SLisandro Dalcin PETSc.Sys.Print("- Parallel OK",comm=comm) 206