Skip to content

host: preserve IEEE denormal assumptions in compiler wrappers - #75

Open
mihawk-99 wants to merge 1 commit into
ps5-payload-dev:masterfrom
mihawk-99:fix-ieee-denormal-assumptions
Open

mihawk-99 wants to merge 1 commit into
ps5-payload-dev:masterfrom
mihawk-99:fix-ieee-denormal-assumptions

Conversation

@mihawk-99

Copy link
Copy Markdown
Contributor

A program that enables IEEE denormal handling at runtime can still misclassify subnormal values when compiled with the SDK wrappers. Clang's PS5 target defaults to preserve-sign denormal assumptions, so optimization can remove the subnormal branch of __builtin_fpclassify even after the program clears MXCSR's FTZ and DAZ bits.

This passes -fdenormal-fp-math=ieee from both prospero-clang and prospero-clang++. It preserves the classification needed by homebrew that enables IEEE handling, including the Vulkan CTS calculations that exposed the problem in my ports. The flag precedes the user's arguments, so an explicit -fdenormal-fp-math=preserve-sign still overrides it.

The change affects compiler assumptions only. It does not change startup MXCSR, and applications that need hardware denormal handling still have to configure it.

Reproducer and validation

samples/test_denormals classifies the smallest positive binary32 subnormal with FTZ and DAZ disabled, then restores the caller's MXCSR. The classifier is in a separate translation unit so the caller cannot constant-fold it.

make -C samples/test_denormals PS5_PAYLOAD_SDK=/path/to/patched-sdk

Built and linked the PS5 sample with Clang/LLD 23.1.1. Also linked its PS5-target C objects into an x86-64 Linux test executable to check the generated CPU code. With preserve-sign, it reported FP_NORMAL (4) and failed; with ieee, it reported FP_SUBNORMAL (8) and passed. Both objects used the SDK headers, so their classification constants agree.

Checked both C and C++ wrapper output: the default retains IEEE assumptions, and the explicit preserve-sign override restores the old model. Both wrapper scripts pass bash -n.

No new console run was performed. The host run verifies the generated classification code, not PS5 process startup behavior. This is a proposed default-policy change; projects intentionally relying on flush-to-zero compiler assumptions can retain them with the explicit flag.

Refs #70.

@john-tornblom

Copy link
Copy Markdown
Contributor

I think it would be better to add this cflag to the cflags when building mesa instead (a parameter to meson cmd arg?) since this changes the behavior of generated code and may very well break other projects that assume default ps5 flags. Will keep pr open for a while so I can ponder…

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants