CNA-BUG-252: Vector3, Vector4, Quaternion and Matrix GetHashCode add raw int bit patterns with signed arithmetic, which overflows (undefined behaviour) for ordinary values

CNA snapshot 009d40f5  ·  Known Issues › Current bugs  ·  source links pinned to 009d40f5

The four GetHashCode functions sum FloatHash results as int with plain +; three 1.0f components already exceed INT_MAX, so the sum is signed-overflow undefined behaviour, while Vector2::GetHashCode already sums as unsigned to avoid it.

Identifier

CNA-BUG-252

Category

Bug

Subsystem

Math & geometry

Status

Open

Verified against

CNA 009d40f5 (009d40f5dd085c4e674d3479675fac84b12b3e0a)

Severity

Low (a triage suggestion, not a project priority)

Evidence basis

Recorded by CNA: CNA's own recorded run, not repeated here

Tests touching this area

Yes: see Current tests

Affected contract

CNA::Vector3::GetHashCode, Vector4::GetHashCode, Quaternion::GetHashCode and Matrix::GetHashCode (Microsoft::Xna::Framework)

Expected behaviour

Signed integer overflow is undefined behaviour in C++, so a hash that sums the int bit patterns of several floats must wrap through unsigned arithmetic. Vector2::GetHashCode does exactly that, and its comment says why (“Unsigned wraparound avoids signed-overflow UB (UBSan, INPUT-BUILD-006); result unchanged”). .NET's unchecked int addition wraps, which is the behaviour XNA's hashes rely on.

Actual behaviour at TARGET

FloatHash returns the IEEE bit pattern of the float as an int. Vector3::GetHashCode in Vector3.cpp returns FloatHash(X) + FloatHash(Y) + FloatHash(Z) with int operands; 1.0f is 0x3F800000 = 1,065,353,216, so Vector3(1, 1, 1) sums to 3,196,059,648 and overflows a 32-bit int. Vector4 (four terms), Quaternion (four) and Matrix (sixteen) do the same. In practice the sum usually wraps, but it is undefined behaviour: a UBSan signed-integer-overflow run reports it, and an optimiser may assume it cannot happen. Vector2 was already repaired; these four were not.

Source locations

• modules/math/src/Vector3.cpp — FloatHash and Vector3::GetHashCode (signed int sum)
• modules/math/src/Vector4.cpp — Vector4::GetHashCode (signed int sum)
• modules/math/src/Quaternion.cpp — Quaternion::GetHashCode (signed int sum)
• modules/math/src/Matrix.cpp — Matrix::GetHashCode (signed int sum of sixteen terms)
• modules/math/src/Vector2.cpp — Vector2::GetHashCode: the unsigned-wraparound form and its UBSan comment

Evidence

Checked by reading at 009d40f5; nothing was built or executed for this entry (no UBSan run). The arithmetic is exact: three 1.0f bit patterns are 3,196,059,648, above INT_MAX (2,147,483,647). Found by the independent adversarial review of the math entries and re-verified by the orchestrator. CNA-BUG-074 covers a different defect in the same functions (negative zero hashing differently from positive zero); the two fixes can share a change.

Focused reproduction

// Illustrative; not compiled or run for this entry. Under -fsanitize=signed-integer-overflow:
Vector3 v(1.0f, 1.0f, 1.0f);
int h = v.GetHashCode();   // FloatHash(1.0f) * 3 exceeds INT_MAX: signed overflow

Current tests

The math tests exercise GetHashCode for equal values (for example CurveKeyTests.cpp and BoundingFrustumTests.cpp), but none of them checks the arithmetic, and whether any lane runs the math suites under UBSan was not examined.

Regression test

Sum through unsigned exactly as Vector2::GetHashCode does, and add a case that hashes Vector3(1, 1, 1) (and the other three types) in a UBSan lane, expecting no report and an unchanged hash value.

Blast radius

Any program that hashes these value types (dictionary keys, sets) in a sanitizer build, and any build whose optimiser exploits the undefined behaviour. The hash value that ordinary builds produce is unchanged by the fix.

Workaround

None needed in ordinary builds. In a UBSan build, suppress signed-integer-overflow for these four functions or hash through your own combiner.