default HTTPS stack based on platform support

[aajtodd]

Motivation and Context

• Hyper Upgrade #1925
• [tracking] support hyper/http/etc. 1.0 awslabs/aws-sdk-rust#977

Description

Modifies the default TLS provider based on platform support and changes the default for unix to s2n-tls.

I had to introduce a new crate aws-smithy-default-tls to make this work. This is because we can't modify dependencies or add new features based on a feature. Doing it without a new crate seems only possible with rust-lang/cargo#1197.

The way around this is to consider the crate itself a "feature" and then we can make use of rust-lang/cargo#7914 which allow specifying features on a dependency predicated on target.

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[github-actions]

A new generated diff is ready to view.

• AWS SDK (ignoring whitespace)
• Client Test (ignoring whitespace)
• No codegen difference in the Server Test
• No codegen difference in the Server Test Python
• No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

[github-actions]

A new generated diff is ready to view.

• AWS SDK (ignoring whitespace)
• Client Test (ignoring whitespace)
• No codegen difference in the Server Test
• No codegen difference in the Server Test Python
• No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

[github-actions]

A new generated diff is ready to view.

• AWS SDK (ignoring whitespace)
• Client Test (ignoring whitespace)
• No codegen difference in the Server Test
• No codegen difference in the Server Test Python
• No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

[aajtodd]

Did some performance regression testing using the CRT S3 benchmark suite. We are going to close this for now and leave the default as rustls+aws-lc. We'll work with s2n team on profiling and figuring out a path forward. The structural setup in this PR may still prove useful in the future should we need to default to different TLS providers based on platform/target.

Raw numbers for reference:

# 30 GiB download to RAM using hyper1 + aws-lc (s3 tm main branch currently via aws_smithy_experimental)
/home/ec2-user/sandbox/aws-crt-s3-benchmarks/runners/s3-benchrunner-rust/target/release/s3-benchrunner-rust
Run:1 Secs:12.160394 Gb/s:21.191585
Run:2 Secs:7.051622 Gb/s:36.544505
Run:3 Secs:3.356120 Gb/s:76.784502
Run:4 Secs:3.613896 Gb/s:71.307547
Run:5 Secs:6.432426 Gb/s:40.062339
Run:6 Secs:3.375006 Gb/s:76.354836
Run:7 Secs:3.322531 Gb/s:77.560756
Run:8 Secs:3.260616 Gb/s:79.033535
Run:9 Secs:3.239925 Gb/s:79.538282
Run:10 Secs:3.426990 Gb/s:75.196608

[ec2-user@ip-172-31-42-139 aws-crt-s3-benchmarks]$ ./build_and_run.sh
/home/ec2-user/sandbox/aws-crt-s3-benchmarks/runners/s3-benchrunner-rust> cargo build --release
    Finished `release` profile [optimized] target(s) in 0.18s
------ RUNNER_CMD ------
/home/ec2-user/sandbox/aws-crt-s3-benchmarks/runners/s3-benchrunner-rust/target/release/s3-benchrunner-rust
Run:1 Secs:8.413203 Gb/s:30.630193
Run:2 Secs:5.796209 Gb/s:44.459755
Run:3 Secs:3.226865 Gb/s:79.860185
Run:4 Secs:7.317880 Gb/s:35.214848
Run:5 Secs:3.807951 Gb/s:67.673675
Run:6 Secs:8.699537 Gb/s:29.622040
Run:7 Secs:12.795185 Gb/s:20.140235
Run:8 Secs:10.226855 Gb/s:25.198172
Run:9 Secs:6.551659 Gb/s:39.333250
Run:10 Secs:5.101917 Gb/s:50.510038

[ec2-user@ip-172-31-42-139 aws-crt-s3-benchmarks]$ ./build_and_run.sh
/home/ec2-user/sandbox/aws-crt-s3-benchmarks/runners/s3-benchrunner-rust> cargo build --release
    Finished `release` profile [optimized] target(s) in 0.20s
------ RUNNER_CMD ------
/home/ec2-user/sandbox/aws-crt-s3-benchmarks/runners/s3-benchrunner-rust/target/release/s3-benchrunner-rust
Run:1 Secs:9.333234 Gb/s:27.610799
Run:2 Secs:6.331628 Gb/s:40.700121
Run:3 Secs:3.304693 Gb/s:77.979419
Run:4 Secs:4.225890 Gb/s:60.980776
Run:5 Secs:7.156990 Gb/s:36.006484
Run:6 Secs:3.373012 Gb/s:76.399972
Run:7 Secs:3.361471 Gb/s:76.662290
Run:8 Secs:3.472432 Gb/s:74.212553
Run:9 Secs:3.257133 Gb/s:79.118069
Run:10 Secs:3.247230 Gb/s:79.359341

# 30 GiB download to RAM using hyper1 + s2n-tls 
   Compiling s3-benchrunner-rust v0.1.0 (/home/ec2-user/sandbox/aws-crt-s3-benchmarks/runners/s3-benchrunner-rust)
warning: `aws-s3-transfer-manager` (lib) generated 2 warnings (run `cargo fix --lib -p aws-s3-transfer-manager` to apply 2 suggestions)
    Building [=======================> ] 346/347: s3-benchrunner-rust(bin)
    Finished `release` profile [optimized] target(s) in 55.96s
------ RUNNER_CMD ------
/home/ec2-user/sandbox/aws-crt-s3-benchmarks/runners/s3-benchrunner-rust/target/release/s3-benchrunner-rust
Run:1 Secs:9.903571 Gb/s:26.020718
Run:2 Secs:6.913731 Gb/s:37.273367
Run:3 Secs:11.389119 Gb/s:22.626687
Run:4 Secs:10.129107 Gb/s:25.441338
Run:5 Secs:5.770604 Gb/s:44.657027
Run:6 Secs:4.997102 Gb/s:51.569499
Run:7 Secs:5.173330 Gb/s:49.812794
Run:8 Secs:3.974426 Gb/s:64.839056
Run:9 Secs:10.807099 Gb/s:23.845255
Run:10 Secs:3.981825 Gb/s:64.718574

[ec2-user@ip-172-31-42-139 aws-crt-s3-benchmarks]$ ./build_and_run.sh
warning: `aws-s3-transfer-manager` (lib) generated 2 warnings (run `cargo fix --lib -p aws-s3-transfer-manager` to apply 2 suggestions)
    Finished `release` profile [optimized] target(s) in 0.20s
------ RUNNER_CMD ------
/home/ec2-user/sandbox/aws-crt-s3-benchmarks/runners/s3-benchrunner-rust/target/release/s3-benchrunner-rust
Run:1 Secs:7.147009 Gb/s:36.056768
Run:2 Secs:7.993910 Gb/s:32.236795
Run:3 Secs:11.457785 Gb/s:22.491087
Run:4 Secs:12.284546 Gb/s:20.977416
Run:5 Secs:4.334734 Gb/s:59.449560
Run:6 Secs:9.475266 Gb/s:27.196917
Run:7 Secs:3.879103 Gb/s:66.432372
Run:8 Secs:15.262528 Gb/s:16.884361
Run:9 Secs:4.003311 Gb/s:64.371222
Run:10 Secs:3.921189 Gb/s:65.719357

[ec2-user@ip-172-31-42-139 aws-crt-s3-benchmarks]$ ./build_and_run.sh
/home/ec2-user/sandbox/aws-crt-s3-benchmarks/runners/s3-benchrunner-rust> cargo build --release
warning: `aws-s3-transfer-manager` (lib) generated 2 warnings (run `cargo fix --lib -p aws-s3-transfer-manager` to apply 2 suggestions)
    Finished `release` profile [optimized] target(s) in 0.21s
------ RUNNER_CMD ------
/home/ec2-user/sandbox/aws-crt-s3-benchmarks/runners/s3-benchrunner-rust/target/release/s3-benchrunner-rust
Run:1 Secs:9.351953 Gb/s:27.555532
Run:2 Secs:8.337577 Gb/s:30.908024
Run:3 Secs:12.931791 Gb/s:19.927482
Run:4 Secs:7.360418 Gb/s:35.011330
Run:5 Secs:5.921717 Gb/s:43.517453
Run:6 Secs:4.096035 Gb/s:62.914021
Run:7 Secs:4.871492 Gb/s:52.899197
Run:8 Secs:4.261247 Gb/s:60.474788
Run:9 Secs:5.017272 Gb/s:51.362187
Run:10 Secs:7.385893 Gb/s:34.890572

# 30 GiB download to RAM using hyper1 + rustls<aws-lc> (using hyper1 branch)
warning: `aws-s3-transfer-manager` (lib) generated 2 warnings (run `cargo fix --lib -p aws-s3-transfer-manager` to apply 2 suggestions)
    Finished `release` profile [optimized] target(s) in 0.21s
------ RUNNER_CMD ------
/home/ec2-user/sandbox/aws-crt-s3-benchmarks/runners/s3-benchrunner-rust/target/release/s3-benchrunner-rust
Run:1 Secs:6.583336 Gb/s:39.143992
Run:2 Secs:5.620584 Gb/s:45.848979
Run:3 Secs:6.035723 Gb/s:42.695473
Run:4 Secs:7.475112 Gb/s:34.474136
Run:5 Secs:8.656864 Gb/s:29.768058
Run:6 Secs:3.602178 Gb/s:71.539506
Run:7 Secs:9.934056 Gb/s:25.940869
Run:8 Secs:7.221963 Gb/s:35.682550
Run:9 Secs:3.671617 Gb/s:70.186534
Run:10 Secs:3.456191 Gb/s:74.561291

[ec2-user@ip-172-31-42-139 aws-crt-s3-benchmarks]$ ./build_and_run.sh
    Finished `release` profile [optimized] target(s) in 0.19s
------ RUNNER_CMD ------
/home/ec2-user/sandbox/aws-crt-s3-benchmarks/runners/s3-benchrunner-rust/target/release/s3-benchrunner-rust
Run:1 Secs:7.770635 Gb/s:33.163058
Run:2 Secs:5.683668 Gb/s:45.340095
Run:3 Secs:6.484344 Gb/s:39.741573
Run:4 Secs:3.398729 Gb/s:75.821887
Run:5 Secs:6.926989 Gb/s:37.202030
Run:6 Secs:3.239065 Gb/s:79.559380
Run:7 Secs:3.285915 Gb/s:78.425047
Run:8 Secs:3.367883 Gb/s:76.516334
Run:9 Secs:3.434607 Gb/s:75.029845
Run:10 Secs:3.436963 Gb/s:74.978417

[ec2-user@ip-172-31-42-139 aws-crt-s3-benchmarks]$ ./build_and_run.sh
    Finished `release` profile [optimized] target(s) in 0.20s
------ RUNNER_CMD ------
/home/ec2-user/sandbox/aws-crt-s3-benchmarks/runners/s3-benchrunner-rust/target/release/s3-benchrunner-rust
Run:1 Secs:8.087065 Gb/s:31.865458
Run:2 Secs:5.636244 Gb/s:45.721586
Run:3 Secs:6.337981 Gb/s:40.659325
Run:4 Secs:9.155953 Gb/s:28.145408
Run:5 Secs:3.285333 Gb/s:78.438945
Run:6 Secs:3.278316 Gb/s:78.606830
Run:7 Secs:3.393722 Gb/s:75.933760
Run:8 Secs:3.328566 Gb/s:77.420126
Run:9 Secs:3.228309 Gb/s:79.824467
Run:10 Secs:3.318739 Gb/s:77.649389


# 30 GiB download to RAM using hyper1 + rustls<aws-lc> + tcp no delay reverted (using hyper1 branch)
[ec2-user@ip-172-31-42-139 aws-crt-s3-benchmarks]$ ./build_and_run.sh
------ RUNNER_CMD ------
/home/ec2-user/sandbox/aws-crt-s3-benchmarks/runners/s3-benchrunner-rust/target/release/s3-benchrunner-rust
Run:1 Secs:9.068149 Gb/s:28.417932
Run:2 Secs:6.007649 Gb/s:42.894987
Run:3 Secs:6.646025 Gb/s:38.774762
Run:4 Secs:6.962151 Gb/s:37.014142
Run:5 Secs:3.326129 Gb/s:77.476852
Run:6 Secs:7.241287 Gb/s:35.587325
Run:7 Secs:3.335714 Gb/s:77.254225
Run:8 Secs:3.410184 Gb/s:75.567184
Run:9 Secs:3.327003 Gb/s:77.456517
Run:10 Secs:3.385542 Gb/s:76.117207

[ec2-user@ip-172-31-42-139 aws-crt-s3-benchmarks]$ ./build_and_run.sh
/home/ec2-user/sandbox/aws-crt-s3-benchmarks/runners/s3-benchrunner-rust> cargo build --release
    Finished `release` profile [optimized] target(s) in 0.20s
------ RUNNER_CMD ------
/home/ec2-user/sandbox/aws-crt-s3-benchmarks/runners/s3-benchrunner-rust/target/release/s3-benchrunner-rust
Run:1 Secs:7.060977 Gb/s:36.496090
Run:2 Secs:5.632381 Gb/s:45.752946
Run:3 Secs:5.306538 Gb/s:48.562362
Run:4 Secs:3.356194 Gb/s:76.782810
Run:5 Secs:4.733881 Gb/s:54.436953
Run:6 Secs:4.953634 Gb/s:52.022013
Run:7 Secs:3.268732 Gb/s:78.837320
Run:8 Secs:3.299662 Gb/s:78.098311
Run:9 Secs:3.335204 Gb/s:77.266047
Run:10 Secs:3.197857 Gb/s:80.584594

[ec2-user@ip-172-31-42-139 aws-crt-s3-benchmarks]$ ./build_and_run.sh
/home/ec2-user/sandbox/aws-crt-s3-benchmarks/runners/s3-benchrunner-rust> cargo build --release
    Finished `release` profile [optimized] target(s) in 0.20s
------ RUNNER_CMD ------
/home/ec2-user/sandbox/aws-crt-s3-benchmarks/runners/s3-benchrunner-rust/target/release/s3-benchrunner-rust
Run:1 Secs:6.962146 Gb/s:37.014168
Run:2 Secs:5.083043 Gb/s:50.697587
Run:3 Secs:5.757390 Gb/s:44.759527
Run:4 Secs:6.631388 Gb/s:38.860349
Run:5 Secs:3.952984 Gb/s:65.190756
Run:6 Secs:3.373288 Gb/s:76.393722
Run:7 Secs:3.416310 Gb/s:75.431689
Run:8 Secs:7.000567 Gb/s:36.811025
Run:9 Secs:6.449184 Gb/s:39.958240
Run:10 Secs:4.118250 Gb/s:62.574651


# 30 GiB download to RAM using hyper1 + s2n-tls + tcp no delay reverted
----- RUNNER_CMD ------
/home/ec2-user/sandbox/aws-crt-s3-benchmarks/runners/s3-benchrunner-rust/target/release/s3-benchrunner-rust
Run:1 Secs:7.363163 Gb/s:34.998279
Run:2 Secs:8.028486 Gb/s:32.097960
Run:3 Secs:11.408999 Gb/s:22.587260
Run:4 Secs:9.543634 Gb/s:27.002086
Run:5 Secs:14.254566 Gb/s:18.078280
Run:6 Secs:3.920257 Gb/s:65.734988
Run:7 Secs:3.861695 Gb/s:66.731841
Run:8 Secs:3.814444 Gb/s:67.558482
Run:9 Secs:4.087962 Gb/s:63.038261
Run:10 Secs:4.869622 Gb/s:52.919510

[ec2-user@ip-172-31-42-139 aws-crt-s3-benchmarks]$ ./build_and_run.sh
------ RUNNER_CMD ------
/home/ec2-user/sandbox/aws-crt-s3-benchmarks/runners/s3-benchrunner-rust/target/release/s3-benchrunner-rust
Run:1 Secs:7.301555 Gb/s:35.293581
Run:2 Secs:7.647071 Gb/s:33.698921
Run:3 Secs:11.784727 Gb/s:21.867119
Run:4 Secs:11.435470 Gb/s:22.534977
Run:5 Secs:4.910175 Gb/s:52.482459
Run:6 Secs:4.150348 Gb/s:62.090708
Run:7 Secs:4.012899 Gb/s:64.217420
Run:8 Secs:4.157731 Gb/s:61.980454
Run:9 Secs:10.928397 Gb/s:23.580589
Run:10 Secs:3.878708 Gb/s:66.439137

[ec2-user@ip-172-31-42-139 aws-crt-s3-benchmarks]$ ./build_and_run.sh
------ RUNNER_CMD ------
/home/ec2-user/sandbox/aws-crt-s3-benchmarks/runners/s3-benchrunner-rust/target/release/s3-benchrunner-rust
Run:1 Secs:8.711975 Gb/s:29.579750
Run:2 Secs:7.697960 Gb/s:33.476148
Run:3 Secs:13.299428 Gb/s:19.376625
Run:4 Secs:8.806046 Gb/s:29.263762
Run:5 Secs:9.411990 Gb/s:27.379762
Run:6 Secs:4.764589 Gb/s:54.086102
Run:7 Secs:5.164219 Gb/s:49.900677
Run:8 Secs:3.915825 Gb/s:65.809383
Run:9 Secs:4.313274 Gb/s:59.745349
Run:10 Secs:3.979368 Gb/s:64.758538