fix(kernels): remove global --use_fast_math; selective per-kernel intrinsics only (T3.1) - #142
fix(kernels): remove global --use_fast_math; selective per-kernel intrinsics only (T3.1)#142dndungu wants to merge 4 commits into
Conversation
…max only (T3.1) Drop --use_fast_math from NVCC_FLAGS: the flag swapped every expf/tanhf/logf/division for unsafe approximations library-wide; the non-saturating fast tanhf caused the GELU blowup behind the GPU f32 CrossAsset cliff (ztensor#125). Per the torch numerics convention (zerfoo plan-gpu-training-hardening T3.1) fast intrinsics are now re-enabled selectively in kernel source, only where the PyTorch-oracle gate proves equivalence within tolerance: - kernel_softmax keeps __expf AFTER max-subtraction (argument <= 0, result in (0,1], ~2^-21 relative error -- inside the Softmax tolerance atol 1e-6 / rtol 1e-4). - Everything else gets the accurate libdevice functions. - The tanh saturation clamp (ztensor#125) is RETAINED as defense-in-depth. Gate plumbing: oracle-gen grows -engine gpu (GenerateAllWith records compute.GPUEngine bundles through the same format -- the GPU-side oracle gate the scripts/oracle README reserved for T3.x), and compute/gpu_kernel_bench_test.go adds per-kernel GPU micro-benchmarks for the ops whose codegen changed, for the before/after perf delta. Verifies zerfoo UC-GH-5.
…, perf delta within noise
GB10 gate results (all green)Build: sm_121 Oracle (S3.1.1) — first live GPU-engine oracle run: 25 GPU-engine bundles ( Parity sanity (#140 pattern, rebuilt kernels): Perf delta (engine-level, 4M f32, mean of 3x100):
End-to-end: parity suite 0.720s (old) → 0.691s (new). All deltas within run-to-run noise (~±2%): the engine path is bandwidth-bound at these sizes, so no measurable regression. Full numbers + ops notes in the devlog entry in this PR. |
|
Post-merge-of-main re-gate (pod |
Implements zerfoo plan-gpu-training-hardening T3.1 + S3.1.1 (verifies UC-GH-5).
What
--use_fast_mathfromNVCC_FLAGS(internal/cuda/kernels/Makefile). The flag swapped everyexpf/tanhf/logf/division for unsafe approximations library-wide; the non-saturating fasttanhfcaused the GELU blowup behind the GPU f32 CrossAsset cliff (fix(cuda): saturate kernel_tanh argument (--use_fast_math tanhf overflow) #125).kernel_softmaxkeeps__expfafter max-subtraction (argument <= 0, result in (0,1], ~2^-21 rel error — inside the Softmax tolerance atol 1e-6 / rtol 1e-4). Everything else uses accurate libdevice functions.oracle-gen -engine gpu:GenerateAllWithrecordscompute.GPUEnginebundles through the same format — the GPU-side oracle gate the scripts/oracle README reserved for T3.x.compute/gpu_kernel_bench_test.go: per-kernel GPU micro-benchmarks (Exp/Log/Sqrt/Rsqrt/Sin/Cos/Tanh/Div/Softmax) for the before/after perf delta.Gates (GB10 via Spark, serialized — results to be appended to this PR + devlog)
libkernels.sorebuilt from this branchrun_oracle.py, NGC 26.02): all ops green within tolerancescripts/parity/run.sh, test(parity): Wolf-pattern training loop under poison -- GB10 + CI variants (S2.3.1) #140 pattern): both schedules + red-proof green against the rebuilt kernels