Commit 2b9f60a
committed
MeshHandler: Fix
In the Editor's serialization code, normalized weights are quantized as 5-bit UNORM values then packed as ints. This implies flooring - while the sum of the quantized values are guaranteed to be `31` this can lead to a loss of precision where the _de_quantized sum (div by 31 then sum) of weights is less than 1.0.
Such cases can be trivally reproduced by search, for example a tuple of (1,25,5) summed as (1/31+25/31+5/31) in _that exact order_ will yield 0.9999999999999999 instead of 1.0, and ((1/31+25/31+5/31) >= 1.0) will be falsey.
This PR implements the sum of weight with ints instead. Alternatively an epsilon value could be chosen, but this is what the player code does.weightsData dequantization error1 parent e9ed554 commit 2b9f60a
1 file changed
+3
-3
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
583 | 583 | | |
584 | 584 | | |
585 | 585 | | |
586 | | - | |
| 586 | + | |
587 | 587 | | |
588 | 588 | | |
589 | 589 | | |
| |||
596 | 596 | | |
597 | 597 | | |
598 | 598 | | |
599 | | - | |
| 599 | + | |
600 | 600 | | |
601 | 601 | | |
602 | 602 | | |
603 | 603 | | |
604 | 604 | | |
605 | 605 | | |
606 | | - | |
| 606 | + | |
607 | 607 | | |
608 | 608 | | |
609 | 609 | | |
| |||
0 commit comments