feat(rust/core): add async traits for Driver, Database, Connection, Statement

[if0ne]

Hi, I’m back again. This is a continuation of this PR: #3712

I’ve returned with a draft of the asynchronous traits. They fully mirror the synchronous API (with a couple of differences in input arguments), adding Future + Send as the return type.

As further development, I can propose:

• Abstract away from async executors and introduce an AsyncExecutor trait with a single block_on method. In adbc_core, add helper structures like SyncDriverWrapper<E: AsyncExecutor, D: AsyncDriver> and others, and implement the corresponding synchronous traits for them.

• Add some procedural macro magic to automatically implement the synchronous traits by generating the code described above.

[if0ne]

What do you think? @lidavidm and @eitsupi

[lidavidm]

CC @felipecrv @mbrobbel too

[mbrobbel]

For the Send bound stuff I would suggest using https://docs.rs/trait-variant/0.1.2/trait_variant/ (as suggested in https://blog.rust-lang.org/2023/12/21/async-fn-rpit-in-traits/#async-fn-in-public-traits).

For async vs sync I would suggest using the async trait as the default and provided sync helper traits and structs. But the truth is; there is no elegant solution available today for this.

[if0ne]

trait_variant will help solve the Send bound problem, but it introduces another problem: the macro does not add additional Send bound to function input arguments if they are of an opaque type. Should we use explicit types instead?

[felipecrv]

Once the code in blocking is moved back to the current location I can give another review.

@lidavidm do we really need everything to be a Future, even get/set options?

[felipecrv]

I think we can skip creating the blocking namespace altogether and keep the symbols where they were. "Blocking" has bad connotations and I don't think a blocking database API is as bad as the name "blocking" suggests.

[if0ne]

I think we can skip creating the blocking namespace altogether and keep the symbols where they were. "Blocking" has bad connotations and I don't think a blocking database API is as bad as the name "blocking" suggests.

I’m a bit confused. @mbrobbel suggests that the async traits should be used by default. To me, this means they should be at the root of the crate, with the corresponding sync version in a separate module.

I don’t see anything wrong with using the namespace blocking. I relied on existing crates. For example reqwest

[felipecrv]

async being the default for database APIs is a bad idea IMO. Database connections are stateful objects that accumulates statements in transactions. Async may re-order actual execution of the queries on the connection. A world of pain.

[if0ne]

I made the sync traits the default, but I won’t move them back to the crate root. Let them stay in the sync mod.

[if0ne]

Once the code in blocking is moved back to the current location I can give another review.

@lidavidm do we really need everything to be a Future, even get/set options?

I made the AsyncOptionable because in the DataFusion implementation, setting options involved asynchronous operations. It might be possible to make get_option_* sync, but I would keep set_option async.

https://github.com/if0ne/arrow-adbc/blob/61963f8e450926da1657f31112db9498f789b380/rust/driver/datafusion/src/lib.rs#L247

[lidavidm]

Even get_option may involve I/O in some cases (e.g. if the driver has to run a query to fetch some information), unfortunately