[Discussion/Tip] Root cause found for the ">50k cells" segmentation fault (Upstream SGL C++ bug) & Potential workaround #25

First of all, thank you for developing and maintaining this excellent framework! It has been incredibly helpful for my research.

I am opening this issue to share a debugging discovery regarding the warning in DoLP_SGL: "The number of cells in the scRNA-seq data is too large (>50k), this may result in segmentation fault."

I recently encountered this exact segmentation fault (exit 139) on a dataset with about 22,000 cells. Out of curiosity, I traced the stack dump and found that the crash is not caused by LP_SGL at all, nor is it due to memory exhaustion. The root cause is actually an out-of-bounds memory access bug in the C++ backend of the upstream CRAN package SGL (v1.3).

What happens in the upstream SGL package?

In SGL's src/Coxfit.cpp (both Solver and logitSolver), an array G is dynamically allocated for each group (groupLen[i]). However, during the update step, the loop uses a global absolute index j to access it (G[j]).

When there are many cells and multiple clusters, the group offset (rangeGroupInd[i]) becomes very large. The G[j] pointer eventually strays into unmapped memory, which triggers the OS-level segfault. This perfectly explains why you consistently observed crashes when scaling up to >50k cells!

The Fix & 60k+ Cells Validation

I applied a one-character fix to my local SGL source by converting it to a relative index: G[j - rangeGroupInd[i]].

After recompiling SGL locally, I was able to successfully run the pipeline on over 60,000 cells without any crashes. Furthermore, since the array in question is write-only in the C++ code, I verified that this patch does not alter the mathematical results at all (identical(beta_orig, beta_fixed) == TRUE).

Suggestions

I have already reported this bug and the patch to the CRAN SGL maintainer. Since CRAN updates can take some time, I thought it would be helpful to share this finding here.

You might consider:

1. Updating the documentation: Adding a brief note that the segmentation fault is tied to an upstream SGL 1.3 C++ bug, which can occasionally trigger even on datasets smaller than 50k cells.
2. Local Patch Guide: Perhaps adding a small wiki page or script showing users how to patch and recompile SGL locally if they need to process very large datasets.

Thanks again for the fantastic work on this package. I hope this information is helpful for future development!