-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature gate object_store
and reqwest
#71
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add to CI variants that test that async-tiff
compiles with each combination of features?
You can see how parquet-wasm uses cargo-all-features
https://github.com/kylebarron/parquet-wasm/blob/e85322b592ab224938861895a06f4f3947a77802/Cargo.toml#L117
Co-authored-by: Kyle Barron <[email protected]>
Co-authored-by: Kyle Barron <[email protected]>
just wanted to note: removing |
it was not fine and is fixed now |
src/reader.rs
Outdated
impl AsyncFileReader for std::fs::File { | ||
fn get_bytes(&self, range: Range<u64>) -> BoxFuture<'_, AsyncTiffResult<Bytes>> { | ||
async move { | ||
let mut file = self.try_clone()?; | ||
file.seek(std::io::SeekFrom::Start(range.start))?; | ||
let len = (range.end - range.start) as usize; | ||
let mut buf = vec![0u8; len]; | ||
file.read_exact(&mut buf)?; | ||
let res = Bytes::copy_from_slice(&buf); | ||
Ok(res) | ||
} | ||
.boxed() | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should not add this because this will perform blocking reads on the main thread.
Instead we should probably restore the tokio integration
Lines 70 to 91 in 6a7a2dc
// #[cfg(feature = "tokio")] | |
// impl<T: tokio::io::AsyncRead + tokio::io::AsyncSeek + Unpin + Debug + Send + Sync> AsyncFileReader | |
// for T | |
// { | |
// fn get_bytes(&self, range: Range<u64>) -> BoxFuture<'_, AsyncTiffResult<Bytes>> { | |
// use tokio::io::{AsyncReadExt, AsyncSeekExt}; | |
// async move { | |
// self.seek(std::io::SeekFrom::Start(range.start)).await?; | |
// let to_read = (range.end - range.start).try_into().unwrap(); | |
// let mut buffer = Vec::with_capacity(to_read); | |
// let read = self.take(to_read as u64).read_to_end(&mut buffer).await?; | |
// if read != to_read { | |
// return Err(AsyncTiffError::EndOfFile(to_read, read)); | |
// } | |
// Ok(buffer.into()) | |
// } | |
// .boxed() | |
// } | |
// } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The problem I had was that the feature testing also wants to test with all features disabled, and I didn't exactly know how to deal with that... added a fallback in util.rs
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also tokio's asyncread require &mut self
, so I changed the impl to tokio::fs::File
, where we can try_clone()
on &self
. There doesn't seem to be a TryClone trait
@kylebarron should be done! |
01b0f2f
to
feefadd
Compare
Thanks. I updated a couple things:
|
As discussed in #6, only feature gate and not pulling coalesce_ranges in-tree