Build1 publisher3 min readPublished
Per-row pointer arrays stall HPCCG's CUDA port before the first kernel is written
More than 80 percent of HPCCG's CPU time sits in sparse matrix-vector multiply. A dev.to walkthrough moving that kernel to CUDA finds the matrix has to be flattened into CSR before any device code runs.
The Engineer · Build desk
What happened
- HPCCG, a conjugate gradient mini-app from the Mantevo project, does almost all of its work in three kernels: sparse matrix-vector multiply, WAXPBY and DDOT.
- The profiled case is a 100^3 problem of one million rows, which the original single-process CPU build solved in 149 iterations to a final residual of about 8.0 x 10^-21.
- The staged plan ends by keeping the CG working set resident on the GPU and profiling again, with an independent Kokkos port run under both CPU and CUDA backends.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- cost The first work the port requires is a refactor of code that already passed: the CSR conversion rewrites the matrix representation the CPU build depends on, and it comes before a single kernel runs on the device.
- decision The CPU run's 149 iterations and 8.0 x 10^-21 residual become the correctness target for every later stage, so a change that alters the iteration count has to be treated as a defect and not a result.
- constraint SpMV's share was measured in one single-process run at one problem size, so the ordering of this port carries only to codes where the same kernel holds a comparable share of time.
- capability Because each porting stage is a separate published commit, a reader can check out the version before device residency and measure the delta on their own hardware.
HPCCG keeps each matrix row behind its own pointer: `ptr_to_vals_in_row[i]`, `ptr_to_inds_in_row[i]`, and a per-row nonzero count [9]. On the CPU that is convenient. On the device the top-level struct copies fine and the pointers inside it still address host memory. The author wrote that copying the top-level structure to device memory "would not make its host pointers magically become valid GPU pointers" [10]. The replacement is CSR: `row_offsets` of length nrow+1, `cols` and `vals` of length nnz, three contiguous arrays that can each be allocated and copied on their own [11]. The commit that opens the migration is `51fdbe5`, "Starting off CUDA migration -- sparsemv is hotspot" [12].
The published loop shows why moving one kernel at a time runs out. In an iteration after the first, `ddot(r, r)` produces rtrans, beta is rtrans/oldrtrans, a `waxpby` forms p, normr is the square root of rtrans and is tested by the `for` condition, `HPC_sparsemv` produces Ap, a second `ddot` produces alpha, and two more `waxpby` calls update x and r [14]. That is six kernel launches in a single iteration [3]. Two of them are global reductions, and both results are needed as ordinary doubles before the arguments to the next kernel exist. The 100^3 run took 149 iterations [5]. Across those iterations, 298 reduction results stand in the path of whatever computes beta and alpha [4].
SpMV was more than 80 percent of CPU runtime, according to the walkthrough [6], so zeroing it leaves the rest of the solver at under 20 percent and caps the whole-solver gain near 5x [2]. That is a computable ceiling on the first hypothesis. Host-device transfers subtract from it. For 5x to be the ceiling in someone else's code, their profile would need the same share of time in indexed loads doing little arithmetic per byte moved [7].
The plan continues past that point: migrate the remaining vector operations and reductions, keep the CG working set resident on the GPU, then profile again to see where the bottleneck moved [13]. The article's text stops after the CSR arrays and does not report GPU timings [18]. The author wrote that "the goal is to make the porting and performance-engineering process itself visible: profile, form a hypothesis, change one part of the application, measure again, and follow the bottleneck" [16].
The last stage sets the CUDA version aside and ports the original CPU implementation to Kokkos, then runs essentially the same Kokkos source under both the CPU and CUDA backends [15]. All of the code and the intermediate CUDA-porting commits are in the author's HPCCG repository, so the individual stages can be checked out and reproduced [17].
What to watch
- The post-residency profile: whether SpMV still dominates once the CG working set never leaves the device.
- Whether alpha, beta and the convergence test move on-device, removing the two per-iteration reduction round trips.
- How the Kokkos CUDA backend compares with the hand-written kernels on the same 100^3, 149-iteration problem.