Skip to content

improve performance with parsing bigints with lots of digits #826

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Oct 22, 2022

Conversation

pjfanning
Copy link
Member

@pjfanning pjfanning commented Oct 21, 2022

Parsing numbers in Java has poor performance characteristics. Parse time increases exponentially as the number of digits increases.
Using BigDecimalParser to parse BigIntegers has better performance when there are a lot of digits. It is slower for more usual numbers. From benchmarking, BigDecimalParser is more performant when you have approx 1250 digits and as you head to 10k, 100k digits, it is much faster. BigDecimalParser is not O(n) but it is better than new BigInteger(string).

Relates to #814

@cowtowncoder cowtowncoder added this to the 2.14.0 milestone Oct 22, 2022
@cowtowncoder
Copy link
Member

Interesting! Ok, I trust that you have benchmarked this in addition to it being found a useful optimization in other libraries.
Will merge to go on 2.14.0 since it seems like a low-risk improvement.

@cowtowncoder cowtowncoder merged commit 3d7185d into FasterXML:2.14 Oct 22, 2022
@pjfanning pjfanning deleted the bigint-many-digits branch October 22, 2022 18:22
@pjfanning
Copy link
Member Author

pjfanning commented Oct 22, 2022

@cowtowncoder I benchmarked using https://github.com/pjfanning/jackson-number-parse-bench

On my laptop, I got these results

Benchmark                                 Mode  Cnt       Score       Error  Units
BigIntegerParserBench.bigDec1000         thrpt    5   35195.566 ±  2758.478  ops/s
BigIntegerParserBench.bigDec10000        thrpt    5    1232.178 ±   221.485  ops/s
BigIntegerParserBench.bigDec1000000      thrpt    5       0.396 ±     0.031  ops/s
BigIntegerParserBench.bigInt1000         thrpt    5   48308.680 ±   671.193  ops/s
BigIntegerParserBench.bigInt10000        thrpt    5     617.721 ±    14.032  ops/s
BigIntegerParserBench.bigInt1000000      thrpt    5       0.063 ±     0.002  ops/s
BigIntegerParserBench.fastBigDec1000     thrpt    5  138409.766 ± 10781.948  ops/s
BigIntegerParserBench.fastBigDec1000000  thrpt    5       2.342 ±     0.019  ops/s
  • the higher ops/s is the better
  • 1000 in benchmark means testing with integers with 1000 chars, 10000 means 10000 chars, etc.
  • fastBigDec benchmarks use an experimental new parser that is not ready for Jackson 2.14
  • bigDec benchmarks use BigDecimalParser
  • bigInt benchmarks use new BigInteger(string)

@cowtowncoder
Copy link
Member

@pjfanning Cool. Would it make sense to add simple results on README (everyone can of course run it locally too).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants