CNA-BUG-259: Matrix::Decompose follows FNA's algorithm, not XNA 4.0's: a mirrored matrix decomposes to positive scales and a non-unit quaternion, and a sheared matrix reports success

CNA snapshot 009d40f5  ·  Known Issues › Current bugs  ·  source links pinned to 009d40f5

Matrix::Decompose never checks the determinant: a mirrored matrix returns positive scales, a non-unit quaternion and true where XNA 4.0 negates the largest scale, and a sheared matrix returns true where XNA returns false.

Identifier

CNA-BUG-259

Category

Bug

Subsystem

Math & geometry

Status

Open

Verified against

CNA 009d40f5 (009d40f5dd085c4e674d3479675fac84b12b3e0a)

Severity

Medium (a triage suggestion, not a project priority)

Evidence basis

Source-verified: read at TARGET, not executed

Tests touching this area

Yes: see Current tests

Affected contract

Microsoft::Xna::Framework::Matrix::Decompose(Vector3& scale, Quaternion& rotation, Vector3& translation)

Expected behaviour

XNA 4.0's Matrix.Decompose (read from the decompiled genuine Microsoft.Xna.Framework assembly and its IL; not executed) takes each scale as the length of the row, orders the three axes by scale, substitutes a canonical basis row for any axis shorter than 0.0001 and normalises the rows. When the determinant of that basis is negative it negates the largest axis's scale and row, so Matrix::CreateScale(-1, 1, 1) decomposes to scale (-1, 1, 1), the identity rotation and true. When the squared error of the determinant from 1 exceeds 0.0001, as for a sheared matrix, it returns false with an identity rotation. CNA's own rule is that XNA 4.0 wins where FNA and XNA disagree.

Actual behaviour at TARGET

Matrix::Decompose is the FNA (and MonoGame) algorithm: each scale takes its sign from the product of the row's first four elements, which is always +1 for an affine matrix because M14, M24 and M34 are zero; it divides each row by its scale and passes the result to Quaternion::CreateFromRotationMatrix. It never looks at the determinant and returns false only when a scale is within FLT_EPSILON of zero. CreateScale(-1, 1, 1) therefore yields scale (1, 1, 1), the non-unit quaternion (0, 0, 0, 0.7071) and true (a reflection has no rotation, so the trace-based conversion returns a quaternion of norm 0.7071), and a sheared matrix returns true with a quaternion that is not a unit rotation. MatrixTest.DecomposeReportsPositiveScaleForAMirroredAffineTransform pins the FNA result and attributes it to XNA's “per-row sign heuristic”, which the XNA IL does not contain.

Source locations

• modules/math/src/Matrix.cpp — Matrix::Decompose: sign-of-product scale, no determinant handling
• modules/math/src/Quaternion.cpp — CreateFromRotationMatrix: trace-based conversion that assumes a proper rotation
• modules/math/tests/Microsoft/Xna/Framework/MatrixTests.cpp — DecomposeReportsPositiveScaleForAMirroredAffineTransform pins the FNA behaviour
• modules/math/include/Microsoft/Xna/Framework/Matrix.hpp — Decompose documents true on success and false otherwise
• CLAUDE.md — the rule that XNA wins where FNA and XNA disagree

Evidence

Read at 009d40f5 against the decompiled XNA 4.0 Matrix.Decompose and CNA's code; no CNA binary was built or run. The orchestrator ported both algorithms to a float32 model and evaluated CreateScale(-2, 3, 4) times a Y rotation of 0.7 rad: the CNA model gives scale (2, 3, 4), a quaternion of norm 0.707 and true, the XNA model scale (2, 3, -4), a unit quaternion and true; for a sheared matrix (rows (1,0,0), (1,1,0), (0,0,1)) the CNA model returns true with a quaternion of norm 0.98 where XNA returns false. The CNA side also follows by hand from the code: diag(-1, 1, 1) has trace 1, so CreateFromRotationMatrix returns (0, 0, 0, 0.7071). Reported as a narrow restore by the D2 dismissal reviewer, who had dismissed a related epsilon claim, and re-verified independently.

Focused reproduction

No focused reproduction is known. Nothing has been invented here; the evidence above is what exists.

Current tests

MatrixTest.DecomposeReportsPositiveScaleForAMirroredAffineTransform asserts the FNA result (positive scales for a mirrored matrix) and states it as XNA's behaviour; DecomposeAxisAlignedQuarterTurnRecoversTheRotationRatherThanIdentity covers proper rotations only. No test checks the norm of the returned quaternion or a sheared input.

Regression test

Decompose CreateScale(-1, 1, 1) and expect scale (-1, 1, 1), a unit quaternion and true; decompose a matrix with a shear component and expect false with the identity rotation. The existing mirrored-matrix test has to be rewritten, because it asserts the FNA result.

Blast radius

Code that decomposes a transform containing a mirror (a negative scale on one axis), for example mirrored nodes in imported glTF scenes, whose bind-pose matrices GltfImportCore decomposes, and any caller that then uses the quaternion as a rotation. Proper rotations with positive scales, the common case, decompose correctly.

Workaround

Test the sign of the 3×3 Determinant() first (the math internals page and the test comment recommend this for handedness) and remove the mirror before decomposing, or build the transform from its parts instead of decomposing it.