Why is there NO way to use a self signed Cert by specifying a certificate file?

[russfellows]

In general, this SDK is great. I have a created an S3 client, and am able to do GET, PUT, DELETE, LIST, and other necessary operations. Great, BUT... so far is has proven impossible to specify a CA bundle to support self signed certificates. Has ANYONE done this? Where are examples or documentation? It doesn't exist as far as I can find.

Ideally, it would be great to support operations EXACTLY as the "aws s3" cli does, but doing it at all would be a great start. For example the following method of specifying a ca_bundle:

aws s3 ca_bundle = /path/to/cabundle-2019mar05.pem ls s3://my-bucket

This is documented here: [(https://docs.aws.amazon.com/cli/v1/userguide/cli-configure-files.html)]
However, there is no documented method, or code that supports this in the Rust aws_sdk_s3

Note: The setting an environment variable with this SDK, as is documented with aws cli does NOT work:
AWS_CA_BUNDLE=/path/to/ca-cert/cacert.pem

[aajtodd]

The AWS_CA_BUNDLE is a CLI specific environment variable.

The Rust SDK developer guide now has examples of providing a custom CA root: https://docs.aws.amazon.com/sdk-for-rust/latest/dg/http.html#customizeCertificatesTls

[russfellows]

Aaron, see my subsequent replies. This code sample is WAY too minimal to be useful.

[russfellows]

Thanks Aaron, I will try this. I could find nothing, and after extensive Chat interactions, pointing it at Github examples, the Rust crates, the aws_sdk_s3, rustls, smithy-aws, etc. etc. I ended up with 40 sets of code, none of which would compile. It would be nice to add something to the high level documentation pointing here also, because searching the docs for “CA” “Certs”, Custom, etc. all turned up nothing. Regards, —Russ Russ Fellows Senior Partner, Evaluator Group ***@***.***

…

On Apr 11, 2025, at 12:20 PM, Aaron Todd ***@***.***> wrote: The AWS_CA_BUNDLE is a CLI specific environment variable. The Rust SDK developer guide now has examples of providing a custom CA root: https://docs.aws.amazon.com/sdk-for-rust/latest/dg/http.html#customizeCertificatesTls — Reply to this email directly, view it on GitHub <#1277 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AF64UJ5K4ND7RMHHAGW4O432ZABXRAVCNFSM6AAAAAB26WMOVKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEOBQG42DCNA>. You are receiving this because you authored the thread.

[russfellows]

Sorry, that is not helpful. The code sample is WAY too minimal. If there are going to be examples, it needs to produce code that someone can actually run. The reference is a small code snipet which is useless without a lot more code.

I am attaching a Rust program that "SHOULD" compile and create a useful example. Except of course it does not because there is not enough information in all the crates to create a working example. This is as close as I can come. If it were to compile, it would use environment variables for KEY_ID and SECRET_KEY along with URL to connect to S3 storage over TLS, and list the contents of a given bucket. Importantly, it should also allow the use of a custom certificate, via CLI or environment variable.

Can ANYONE please correct this, and produce an actual working example? I STILL have NEVER seen a working example.

Code attached:
simple_s3_custom_cert.rs.txt

[russfellows]

Update.

Note: I have deleted the code sample that was here, in order to NOT confuse future readers of this thread. The example I posted was incorrect, as pointed out. by aajtodd. He has since provided a fully working example, which I have tested, and works as intended. Thank you Aaron Todd. I will post a link to that thread below and mark this as closed.

[aajtodd]

Couple things stand out:

1. There is no rustls_connector function on the aws_smithy_http_client::Builder type. The comment with the reference is referencing the aws_smithy_http crate which is not the same thing.
2. You are not setting the http_client on the client config
3. If you are overriding the endpoint you don't need an endpoint resolver usually, you just need to override the endpoint_url (you can do this in code or via env variables globally or per/service FYI). Also see the Rust developer guide section on endpoint config
4. Highly recommend you use the aws_config crate for resolving most of your configuration and only overriding the parts you need. In particular using this will remove the need for you to even set the credentials provider as the default credential chain will already handle sourcing it from environment. See the developer guide section on credential providers

The developer guide provides the necessary code for all of these things.

    let http_client = Builder::new()
        .tls_provider(tls::Provider::Rustls(CryptoMode::AwsLc))
        .tls_context(tls_context_from_pem("my-custom-ca.pem"))
        .build_https();
    
    let sdk_config = aws_config::defaults(
        aws_config::BehaviorVersion::latest()
    )
    .http_client(http_client)
    .endpoint_url("https://localhost:9000")
    .load()
    .await;
    
    // create client(s) using sdk_config
    let s3_client = aws_sdk_s3::Client::new(&sdk_config);


    // List objects in the specified bucket
    list_bucket_objects(&s3_client, bucket).await;

[russfellows]

I will try this, thank you for the explanation and code example.

[russfellows]

Aaron.

This code sample is still not complete as shown above. Also, the developer guide you linked to similarly does not provide a working example, only small code snips of specific pieces required. What is needed is a real function that builds a client, that someone can use. We would VERY much appreciate your, or another aws-sdk-s3 library maintainers assistance.

Background: I am working on a project that may / should be interesting to AWS in general, which is to create a Rust library, and from that a Python library that is usable be AI/ML workloads for accessing S3 storage. Note that the current "boto3" libraries are WAY too slow, and only support PyTorch, with no support for others like TensorFlow.

Here is a public link to the GitHub repo for this project: https://github.com/russfellows/dlio_s3_rust

Here is a link to the specific bug listed in this repo, and included in there is a set of code, that maybe close to using the code you outlined above. russfellows/dlio_s3_rust#4 (comment)

Any assistance would be GREATLY appreciated.

[russfellows]

Here is a link to the Open issue where Aaron Todd (aajtodd) posted the solution.

#1286 (comment)

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.