1b94bfe40SJeremy L Thompson# GPU Development 2b94bfe40SJeremy L Thompson 3b94bfe40SJeremy L ThompsonRuntime selection of libCEED backends allows users to use CPU backends for easier debugging. 4b94bfe40SJeremy L ThompsonCode that produces correct results with CPU backends will produce correct results on GPU backends, provided that JiT and memory access assumptions of the libCEED API are respected. 5b94bfe40SJeremy L Thompson 6b94bfe40SJeremy L Thompson## JiT Compilation 7b94bfe40SJeremy L Thompson 8b94bfe40SJeremy L ThompsonThe filepath to the user source code is passed in {c:func}`CeedQFunctionCreateInterior` as the `source` argument. 9b94bfe40SJeremy L ThompsonThis filepath should typically be an absolute path to ensure the JiT compilation can locate the source file. 10*d538d163SJeremy L ThompsonThe filepath may also be a relative path with respect to a root directory set with {c:func}`CeedAddJitSourceRoot`. 11*d538d163SJeremy L ThompsonThe {c:macro}`CEED_QFUNCTION` macro automatically creates a string with the absolute path, for example a {c:type}`CeedQFunctionUser` called `user` would have this string stored in the variable `user_loc`. 12b94bfe40SJeremy L Thompson 13*d538d163SJeremy L ThompsonThe entire contents of this source file and all included files are used during JiT compilation for GPU backends. 14*d538d163SJeremy L ThompsonInclude statements for system headers that are required for CPU compilation but are not available in GPU compilation environments should be guarded with `#ifdef CEED_RUNNING_JIT_PASS`. 15*d538d163SJeremy L ThompsonAny function definitions in these system headers must still be available in the GPU compilation environments, such as the contents of `<math.h>`. 16b94bfe40SJeremy L ThompsonThese source file must only contain syntax constructs supported by C99 and all targeted backends (i.e. CUDA for `/gpu/cuda`, OpenCL/SYCL for `/gpu/sycl`, etc.). 17b94bfe40SJeremy L Thompson 18b94bfe40SJeremy L ThompsonAll source files must be at the provided filepath at runtime for JiT to function. 19b94bfe40SJeremy L Thompson 20b94bfe40SJeremy L Thompson## Memory Access 21b94bfe40SJeremy L Thompson 22b94bfe40SJeremy L ThompsonGPU backends require stricter adherence to memory access assumptions, but CPU backends may occasionally report correct results despite violations of memory access assumptions. 23860c52dbSJeremy L ThompsonBoth `CeedVector` and `CeedQFunctionContext` have read-only and read-write accessors, and `CeedVector` allow write-only access. 24*d538d163SJeremy L ThompsonRead-only access of `CeedVector` and `CeedQFunctionContext` memory spaces must be respected for to ensure proper GPU behavior. 25860c52dbSJeremy L ThompsonWrite-only access of `CeedVector` memory spaces asserts that all data in the `CeedVector` is invalid until overwritten. 26860c52dbSJeremy L Thompson 27*d538d163SJeremy L Thompson`CeedQFunction` assume that all input arrays are read-only and all output arrays are write-only and the {c:type}`CeedQFunctionUser` must adhere to these assumptions, only reading data in the input arrays and fully overwriting all entries in the output arrays. 28860c52dbSJeremy L ThompsonAdditionally, {c:type}`CeedQFunctionUser` have read-write access for `CeedQFunctionContext` data, unless {c:func}`CeedQFunctionSetContextWritable` was used to indicate that read-only access is sufficient. 29860c52dbSJeremy L Thompson 30b94bfe40SJeremy L ThompsonThe `/cpu/self/memcheck` backends explicitly verify read-only and write-only memory access assumptions. 31b94bfe40SJeremy L Thompson 32