Squashed 'src/secp256k1/' changes from be8d9c2..699c057e4f

699c057e4f build: Remove #undef hack for ASM in the precomputation programs
b161047ad3 ci: Test `make precomp`
7758af3606 build: Remove CC_FOR_BUILD stuff
6e2bbd9667 build: Prebuild and distribute ecmult_gen table
d874b87ca9 prealloc: Get rid of manual memory management for prealloc contexts
72f7621ca8 ecmult_gen: Tidy precomputed file and save space
a844e377a6 ecmult_gen: Precompute tables for all values of ECMULT_GEN_PREC_BITS
f9c0d1c732 refactor: Make PREC_BITS a parameter of ecmult_gen_build_prec_table
af8db36679 refactor: Move default callbacks to util.h
e84f9d9867 doc: Remove obsolete hint for valgrind stack size
0997db140c exhaustive_tests: Fix with ecmult_gen table with custom generator
2e6770c27b refactor: Make generator a parameter of ecmult_gen_create_prec_table
d44559ccbd refactor: Rename program that generates static ecmult_gen table
0396aa2078 refactor: Rename file that contains static ecmult_gen table
69c1e00ee0 ecmult_gen: Make code consistent with comment
6dc3acf95f ecmult_gen: Simplify ecmult_gen context after making table static
4c5ed7a731 refactor: Decouple table generation and ecmult_gen context
ae1e76f648 ecmult_gen: Move table creation to new file and force static prec
49f608d Merge bitcoin-core/secp256k1#1004: ecmult: fix definition of STRAUSS_SCRATCH_OBJECTS
60bf889 ecmult: fix definition of STRAUSS_SCRATCH_OBJECTS
fecf436 Merge bitcoin-core/secp256k1#1019: build: don't append valgrind CPPFLAGS if not installed (macOS)
2e5e4b6 Merge bitcoin-core/secp256k1#1020: doc: remove use of <0xa0> "no break space"
812ff5c doc: remove use of 0xa0 "no break space"
214042a build: don't append valgrind CPPFLAGS if not installed
793ad90 Merge bitcoin-core/secp256k1#1010: doc: Minor fixes in safegcd_implementation.md
dc9b685 doc: Minor fixes in safegcd_implementation.md
ea5e8a9 Merge bitcoin-core/secp256k1#1012: Fix typos
2332975 Fix typos
7006f1b Merge bitcoin-core/secp256k1#1011: ci: Enable -g if we set CFLAGS manually
72de135 ci: Enable -g if we set CFLAGS manually
74c34e7 Merge bitcoin-core/secp256k1#1009: refactor: Use (int)&(int) in boolean context to avoid compiler warning
16d1322 refactor: Use (int)&(int) in boolean context to avoid compiler warning
c74a7b7 Merge bitcoin-core/secp256k1#1007: doc: Replace apoelstra's GPG key by jonasnick's GPG key
3b157c4 doc: Suggest keys.openpgp.org as keyserver in SECURITY.md
73a7472 doc: Replace apoelstra's GPG key by jonasnick's GPG key
515a5db Merge bitcoin-core/secp256k1#991: Merge all "external" benchmarks into a single bench binary
af6abcb Make bench support selecting which benchmarks to run
9f56bdf Merge bench_schnorrsig into bench
3208557 Merge bench_recover into bench
855e18d Merge bench_ecdh into bench
2a7be67 Combine bench_sign and bench_verify into single bench
8fa4120 Merge bitcoin-core/secp256k1#1002: Make aux_rnd32==NULL behave identical to 0x0000..00.
5324f89 Make aux_rnd32==NULL behave identical to 0x0000..00.
21c188b Merge bitcoin-core/secp256k1#943: VERIFY_CHECK precondition for secp256k1_fe_set_int.
3e7b2ea Merge bitcoin-core/secp256k1#999: bench_ecmult: improve clarity of output
23e2f66 bench: don't return 1 in have_flag() if argc = 1
96b1ad2 bench_ecmult: improve clarity of output
20d791e Merge bitcoin-core/secp256k1#989: Shared benchmark format for command line and CSV outputs
aa1b889 Merge bitcoin-core/secp256k1#996: Fix G.y parity in sage code
044d956 Fix G.y parity in sage code
b4b1306 create csv file from the benchmark output
26a255b Shared benchmark format for command line and CSV outputs
9526874 Merge bitcoin-core/secp256k1#810: Avoid overly-wide multiplications in 5x52 field mul/sqr
920a0e5 Merge bitcoin-core/secp256k1#952: Avoid computing out-of-bounds pointer.
f34b5ca Merge bitcoin-core/secp256k1#983: [RFC] Remove OpenSSL testing support
297ce82 Merge bitcoin-core/secp256k1#966: Make aux_rand32 arg to secp256k1_schnorrsig_sign const
2888640 VERIFY_CHECK precondition for secp256k1_fe_set_int.
d49011f Make _set_fe_int( . , 0 ) set magnitude to 0
bc08599 Remove OpenSSL testing support
10f9bd8 Merge bitcoin-core/secp256k1#987: Fix unused parameter warnings when building without VERIFY
189f6bc Fix unused parameter warnings when building without VERIFY
da0092b Merge bitcoin-core/secp256k1#986: tests: remove `secp256k1_fe_verify` from tests.c and modify `_fe_from_storage` to call `_fe_verify`
d439937 tests: remove `secp256k1_fe_verify` from tests.c and modify `secp256k1_fe_from_storage` to call `secp256k1_fe_verify`
2a3a97c Merge bitcoin-core/secp256k1#976: `secp256k1_schnorrsig_sign_internal` should be static
aa5d34a Merge bitcoin-core/secp256k1#783: Make the public API docs more consistent and explicit
7271387 Add missing static to secp256k1_schnorrsig_sign_internal
db4667d Make aux_rand32 arg to secp256k1_schnorrsig_sign const
9a5a87e Merge bitcoin-core/secp256k1#956: Replace ecmult_context with a generated static array.
20abd52 Add tests for pre_g tables.
6815761 Remove ecmult_context.
f20dcbb Correct typo.
16a3cc0 Generate ecmult_static_pre_g.h
8de2d86 Bump memory limits in advance of making the ecmult context static.
d7ec49a Merge bitcoin-core/secp256k1#969: ci: Fixes after Debian release
5d5c74a tests: Rewrite code to circument potential bug in clang
3d2f492 ci: Install libasan6 (instead of 5) after Debian upgrade
adec5a1 Add missing null check for ctx and input keys in the public API
f4edfc7 Improve consistency for NULL arguments in the public interface
9be7b0f Avoid computing out-of-bounds pointer.
b53e0cd Avoid overly-wide multiplications

git-subtree-dir: src/secp256k1
git-subtree-split: 699c057e4f0f72296641971c43d94f06e7d80eaf

Author: fanquake

Diff for: .cirrus.yml

@@ -278,14 +278,13 @@ task:
   container:
     dockerfile: ci/linux-debian.Dockerfile
     cpu: 1
-    memory: 1G
+    memory: 2G
   env:
     ECDH: yes
     RECOVERY: yes
     EXPERIMENTAL: yes
     SCHNORRSIG: yes
     CTIMETEST: no
-    EXTRAFLAGS: "--disable-openssl-tests"
   matrix:
     - name: "Valgrind (memcheck)"
       env:
@@ -294,8 +293,7 @@ task:
         TEST_ITERS: 16
     - name: "UBSan, ASan, LSan"
       env:
-        CFLAGS: "-fsanitize=undefined,address"
-        CFLAGS_FOR_BUILD: "-fsanitize=undefined,address"
+        CFLAGS: "-fsanitize=undefined,address -g"
         UBSAN_OPTIONS: "print_stacktrace=1:halt_on_error=1"
         ASAN_OPTIONS: "strict_string_checks=1:detect_stack_use_after_return=1:detect_leaks=1"
         LSAN_OPTIONS: "use_unaligned=1"
@@ -330,7 +328,7 @@ task:
     # ./configure correctly errors out when given CC=g++.
     # We hack around this by passing CC=g++ only to make.
     CC: gcc
-    MAKEFLAGS: -j2 CC=g++ CFLAGS=-fpermissive
+    MAKEFLAGS: -j2 CC=g++ CFLAGS=-fpermissive\ -g
     WERROR_CFLAGS:
     EXPERIMENTAL: yes
     ECDH: yes

Diff for: .gitattributes

@@ -0,0 +1,2 @@
+src/ecmult_static_pre_g.h linguist-generated
+src/ecmult_gen_static_prec_table.h linguist-generated

Diff for: .gitignore

@@ -1,18 +1,15 @@
-bench_inv
-bench_ecdh
+bench
 bench_ecmult
-bench_schnorrsig
-bench_sign
-bench_verify
-bench_recover
 bench_internal
 tests
 exhaustive_tests
-gen_context
+gen_ecmult_gen_static_prec_table
+gen_ecmult_static_pre_g
 valgrind_ctime_test
 *.exe
 *.so
 *.a
+*.csv
 !.gitignore
 
 Makefile
@@ -44,7 +41,6 @@ coverage.*.html
 
 src/libsecp256k1-config.h
 src/libsecp256k1-config.h.in
-src/ecmult_static_context.h
 build-aux/config.guess
 build-aux/config.sub
 build-aux/depcomp

Diff for: Makefile.am

@@ -1,3 +1,5 @@
+.PHONY: clean-precomp precomp
+
 ACLOCAL_AMFLAGS = -I build-aux/m4
 
 # AM_CFLAGS will be automatically prepended to CFLAGS by Automake when compiling some foo
@@ -28,6 +30,8 @@ noinst_HEADERS += src/ecmult_const.h
 noinst_HEADERS += src/ecmult_const_impl.h
 noinst_HEADERS += src/ecmult_gen.h
 noinst_HEADERS += src/ecmult_gen_impl.h
+noinst_HEADERS += src/ecmult_gen_prec.h
+noinst_HEADERS += src/ecmult_gen_prec_impl.h
 noinst_HEADERS += src/field_10x26.h
 noinst_HEADERS += src/field_10x26_impl.h
 noinst_HEADERS += src/field_5x52.h
@@ -50,6 +54,7 @@ noinst_HEADERS += src/hash_impl.h
 noinst_HEADERS += src/field.h
 noinst_HEADERS += src/field_impl.h
 noinst_HEADERS += src/bench.h
+noinst_HEADERS += src/basic-config.h
 noinst_HEADERS += contrib/lax_der_parsing.h
 noinst_HEADERS += contrib/lax_der_parsing.c
 noinst_HEADERS += contrib/lax_der_privatekey_parsing.h
@@ -81,13 +86,9 @@ endif
 
 noinst_PROGRAMS =
 if USE_BENCHMARK
-noinst_PROGRAMS += bench_verify bench_sign bench_internal bench_ecmult
-bench_verify_SOURCES = src/bench_verify.c
-bench_verify_LDADD = libsecp256k1.la $(SECP_LIBS) $(SECP_TEST_LIBS) $(COMMON_LIB)
-# SECP_TEST_INCLUDES are only used here for CRYPTO_CPPFLAGS
-bench_verify_CPPFLAGS = $(SECP_TEST_INCLUDES)
-bench_sign_SOURCES = src/bench_sign.c
-bench_sign_LDADD = libsecp256k1.la $(SECP_LIBS) $(SECP_TEST_LIBS) $(COMMON_LIB)
+noinst_PROGRAMS += bench bench_internal bench_ecmult
+bench_SOURCES = src/bench.c
+bench_LDADD = libsecp256k1.la $(SECP_LIBS) $(SECP_TEST_LIBS) $(COMMON_LIB)
 bench_internal_SOURCES = src/bench_internal.c
 bench_internal_LDADD = $(SECP_LIBS) $(COMMON_LIB)
 bench_internal_CPPFLAGS = $(SECP_INCLUDES)
@@ -118,7 +119,7 @@ endif
 if USE_EXHAUSTIVE_TESTS
 noinst_PROGRAMS += exhaustive_tests
 exhaustive_tests_SOURCES = src/tests_exhaustive.c
-exhaustive_tests_CPPFLAGS = -I$(top_srcdir)/src $(SECP_INCLUDES)
+exhaustive_tests_CPPFLAGS = $(SECP_INCLUDES)
 if !ENABLE_COVERAGE
 exhaustive_tests_CPPFLAGS += -DVERIFY
 endif
@@ -127,29 +128,45 @@ exhaustive_tests_LDFLAGS = -static
 TESTS += exhaustive_tests
 endif
 
-if USE_ECMULT_STATIC_PRECOMPUTATION
-CPPFLAGS_FOR_BUILD +=-I$(top_srcdir) -I$(builddir)/src
-
-gen_context_OBJECTS = gen_context.o
-gen_context_BIN = gen_context$(BUILD_EXEEXT)
-gen_%.o: src/gen_%.c src/libsecp256k1-config.h
-	$(CC_FOR_BUILD) $(DEFS) $(CPPFLAGS_FOR_BUILD) $(SECP_CFLAGS_FOR_BUILD) $(CFLAGS_FOR_BUILD) -c $< -o $@
-
-$(gen_context_BIN): $(gen_context_OBJECTS)
-	$(CC_FOR_BUILD) $(SECP_CFLAGS_FOR_BUILD) $(CFLAGS_FOR_BUILD) $(LDFLAGS_FOR_BUILD) $^ -o $@
-
-$(libsecp256k1_la_OBJECTS): src/ecmult_static_context.h
-$(tests_OBJECTS): src/ecmult_static_context.h
-$(bench_internal_OBJECTS): src/ecmult_static_context.h
-$(bench_ecmult_OBJECTS): src/ecmult_static_context.h
-
-src/ecmult_static_context.h: $(gen_context_BIN)
-	./$(gen_context_BIN)
-
-CLEANFILES = $(gen_context_BIN) src/ecmult_static_context.h
-endif
-
-EXTRA_DIST = autogen.sh src/gen_context.c src/basic-config.h
+### Precomputed tables
+EXTRA_PROGRAMS = gen_ecmult_static_pre_g gen_ecmult_gen_static_prec_table
+CLEANFILES = $(EXTRA_PROGRAMS)
+
+gen_ecmult_static_pre_g_SOURCES = src/gen_ecmult_static_pre_g.c
+gen_ecmult_static_pre_g_CPPFLAGS = $(SECP_INCLUDES)
+gen_ecmult_static_pre_g_LDADD = $(SECP_LIBS) $(COMMON_LIB)
+
+gen_ecmult_gen_static_prec_table_SOURCES = src/gen_ecmult_gen_static_prec_table.c
+gen_ecmult_gen_static_prec_table_CPPFLAGS = $(SECP_INCLUDES)
+gen_ecmult_gen_static_prec_table_LDADD = $(SECP_LIBS) $(COMMON_LIB)
+
+# See Automake manual, Section "Errors with distclean".
+# We don't list any dependencies for the prebuilt files here because
+# otherwise make's decision whether to rebuild them (even in the first
+# build by a normal user) depends on mtimes, and thus is very fragile.
+# This means that rebuilds of the prebuilt files always need to be
+# forced by deleting them, e.g., by invoking `make clean-precomp`.
+src/ecmult_static_pre_g.h:
+	$(MAKE) $(AM_MAKEFLAGS) gen_ecmult_static_pre_g$(EXEEXT)
+	./gen_ecmult_static_pre_g$(EXEEXT)
+src/ecmult_gen_static_prec_table.h:
+	$(MAKE) $(AM_MAKEFLAGS) gen_ecmult_gen_static_prec_table$(EXEEXT)
+	./gen_ecmult_gen_static_prec_table$(EXEEXT)
+
+PRECOMP = src/ecmult_gen_static_prec_table.h src/ecmult_static_pre_g.h
+noinst_HEADERS += $(PRECOMP)
+precomp: $(PRECOMP)
+
+# Ensure the prebuilt files will be build first (only if they don't exist,
+# e.g., after `make maintainer-clean`).
+BUILT_SOURCES = $(PRECOMP)
+
+maintainer-clean-local: clean-precomp
+
+clean-precomp:
+	rm -f $(PRECOMP)
+
+EXTRA_DIST = autogen.sh SECURITY.md
 
 if ENABLE_MODULE_ECDH
 include src/modules/ecdh/Makefile.am.include

Diff for: README.md

@@ -66,18 +66,9 @@ libsecp256k1 is built using autotools:
     $ ./autogen.sh
     $ ./configure
     $ make
-    $ make check
+    $ make check  # run the test suite
     $ sudo make install  # optional
 
-Exhaustive tests
------------
-
-    $ ./exhaustive_tests
-
-With valgrind, you might need to increase the max stack size:
-
-    $ valgrind --max-stackframe=2500000 ./exhaustive_tests
-
 Test coverage
 -----------
 
@@ -100,6 +91,18 @@ To create a HTML report with coloured and annotated source code:
     $ mkdir -p coverage
     $ gcovr --exclude 'src/bench*' --html --html-details -o coverage/coverage.html
 
+Benchmark
+------------
+If configured with `--enable-benchmark` (which is the default), binaries for benchmarking the libsecp256k1 functions will be present in the root directory after the build.
+
+To print the benchmark result to the command line:
+
+    $ ./bench_name
+
+To create a CSV file for the benchmark result :
+
+    $ ./bench_name | sed '2d;s/ \{1,\}//g' > bench_name.csv
+
 Reporting a vulnerability
 ------------
 

Diff for: SECURITY.md

@@ -9,7 +9,7 @@ The following keys may be used to communicate sensitive information to developer
 | Name | Fingerprint |
 |------|-------------|
 | Pieter Wuille | 133E AC17 9436 F14A 5CF1  B794 860F EB80 4E66 9320 |
-| Andrew Poelstra | 699A 63EF C17A D3A9 A34C  FFC0 7AD0 A91C 40BD 0091 |
+| Jonas Nick | 36C7 1A37 C9D9 88BD E825  08D9 B1A7 0E4F 8DCD 0366 |
 | Tim Ruffing | 09E0 3F87 1092 E40E 106E  902B 33BC 86AB 80FF 5516 |
 
-You can import a key by running the following command with that individual’s fingerprint: `gpg --recv-keys "<fingerprint>"` Ensure that you put quotes around fingerprints containing spaces.
+You can import a key by running the following command with that individual’s fingerprint: `gpg --keyserver hkps://keys.openpgp.org --recv-keys "<fingerprint>"` Ensure that you put quotes around fingerprints containing spaces.