feat(services/gdrive): List shows modified timestamp gdrive

[erickguan]

Which issue does this PR close?

Part of #4746.

Are there any user-facing changes?

None.

[erickguan]

I did some experimentation (patch) with async support for listing operations, which helps reduce runtime as expected. However, extending OpList introduces a "breaking" change. Since OpenDAL has a compatibility package, I’m happy to coordinate with you when you're preparing for a breaking release, to minimize the amount of work on both sides. There's no urgency regarding async support, though.

A few other observations from running the behavior tests (not related to the PR):

1. The behavior test fails on my local machine but passes in CI. I’m investigating the cause casually.
2. The UUID random generation in behavior tests creates multiple levels of UUIDs. While this results in an extremely low chance of collision in the CI for OpenDAL's test accounts, I’d like to reduce the number of levels when running against my environment too.

[Xuanwo]

Hi, sorry for not making the Metakey behavior clearer. (I'm working on this.)

Metakey represents a best effort hint and is not processed server-side. The service merely needs to supply the most comprehensive metadata available during listing.

The Operator will call stat as required, for example:

opendal/core/src/types/list.rs

Lines 150 to 211 in 5e074cc

	fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
	// Returns `None` if we have errored.
	if self.errored {
	return Poll::Ready(None);
	}
	// Trying to pull more tasks if there are more space.
	if self.tasks.has_remaining() {
	// Building future if we have a lister available.
	if let Some(mut lister) = self.lister.take() {
	let fut = async move {
	let res = lister.next_dyn().await;
	(lister, res)
	};
	self.fut = Some(Box::pin(fut));
	}
	if let Some(fut) = self.fut.as_mut() {
	if let Poll::Ready((lister, entry)) = fut.as_mut().poll(cx) {
	self.lister = Some(lister);
	self.fut = None;
	match entry {
	Ok(Some(oe)) => {
	let (path, metadata) = oe.into_entry().into_parts();
	if metadata.contains_metakey(self.required_metakey) {
	self.tasks
	.push_back(StatTask::Known(Some((path, metadata))));
	} else {
	let acc = self.acc.clone();
	let fut = async move {
	let res = acc.stat(&path, OpStat::default()).await;
	(path, res.map(|rp| rp.into_metadata()))
	};
	self.tasks.push_back(StatTask::Stating(Box::pin(fut)));
	}
	}
	Ok(None) => {
	self.lister = None;
	}
	Err(err) => {
	self.errored = true;
	return Poll::Ready(Some(Err(err)));
	}
	}
	}
	}
	}
	// Try to poll tasks
	if let Some((path, rp)) = ready!(self.tasks.poll_next_unpin(cx)) {
	let metadata = rp?;
	return Poll::Ready(Some(Ok(Entry::new(path, metadata))));
	}
	if self.lister.is_some() || self.fut.is_some() {
	Poll::Pending
	} else {
	Poll::Ready(None)
	}
	}
	}

[erickguan]

Thanks for the review. I get a weird Metadata instance problem, that Gdrivebackend::stat returns metadata with a timestamp but Lister::poll_next gets a result of Metadata without the timestamp.

I will debug it a bit.

[erickguan]

@Xuanwo Thanks for the pointer. I've figured out how the PageList and Google Drive backend work together, but I’m currently stuck on a failing test with MinIO in the complete layer. Am I on the right track in continuing this PR?

From what I understand, the `CompleteLayer`` depends on the behavior of different services. Since you mentioned that work on metakey is underway, I’m trying not to alter OpenDAL’s behavior. Unfortunately, adding the new stat behavior in this layer breaks the MinIO service.

[Xuanwo]

Hi, it's intended that we don't send stat for /. The service should always make sure / exists.

[Xuanwo]

The underlying cause might be here:

opendal/core/src/types/metadata.rs

Lines 58 to 61 in 259a19e

	// If mode is dir, we should always mark it as complete.
	if mode.is_dir() {
	metakey |= Metakey::Complete
	}

It seems reasonable to remove it.

[Xuanwo]

Also related to #5287

[erickguan]

Thanks for tentative checking in with metakey!

[erickguan]

After a closer look, can I pause this PR until your metakey work?

[Xuanwo]

After a closer look, can I pause this PR until your metakey work?

Yes, after the metakey is refactored, our work here will be much cleaner.

[Xuanwo]

Yes, after the metakey is refactored, our work here will be much cleaner.

Hi, you can continue your work now.

[erickguan]

Thanks for the ping! I'll pick it up and ping you when it's ready.

[Xuanwo]

Thanks for the ping! I'll pick it up and ping you when it's ready.

Thank you for your patience. Looking forward to your PR.