[disk-buffering] Duplicate export in Windows #1907
Comments
jigarkb
changed the title
|
Found the root cause: Within ToDiskExporter thread, the writableFile is not closed until we try to write new file. Within FromDiskExporter thread, the readableFile class tries to delete the file which is left unclosed in ToDiskExporter. In Unix/Linux this is fine but in Windows its not allowing as references are in separate threads. For this code to work in Windows, we need to close the file within ToDiskExporter thread. I am open to suggestions on best ways to do this. |
|
from @jigarkb Also, after diving deep into the code, I found that the FolderManager is not shared between FromDiskExporter and ToDiskExporter so I am not sure we will ever hit this code block. Let me know if I am missing something. |
yes, that looks wrong @LikeTheSalad wdyt about #1912? |
Not having a shared storage object is wrong, so I think we must fix it. So, thank you for creating this PR, @zeitlinger. Regarding the file not being deleted in Windows, I've been taking a look into it and I think I have an idea of what could be happening, which seems related to an issue in |
|
I've just created this PR where this issue should get fixed. |
Component(s)
disk-buffering
What happened?
Description
On Windows systems, the
exportStoredBatchmethod returnstruewhen a file exists in the buffer directory. However, once the file is fully read, the subsequentfile.delete()call incleanUp()fails silently—returningfalsebut not handled by the code. As a result, the same file is repeatedly re-exported on eachexportStoredBatchcall until a new file is written, leading to redundant exports and significant load on backend APIs.This behavior appears linked to how Windows handles file deletion. After
file.delete()is called, the file’s metadata may reflect 0KB, but the contents persist until a subsequent I/O operation (e.g., file creation) triggers actual cleanup. This suggests possible lazy deletion by the Windows OS, though further investigation at the kernel level would be needed to confirm.Reference: https://github.com/open-telemetry/opentelemetry-java-contrib/blob/v1.39.0/disk-buffering/src/main/java/io/opentelemetry/contrib/disk/buffering/internal/storage/files/ReadableFile.java#L124-L127
Steps to Reproduce
Deploy your code on Windows with
debugEnabledset to true, and you'll consistently see the messageNow exporting batch of ...repeated.Expected Result
The file must be deleted and should not be re-exported.
Actual Result
The file is repeatedly processed and exported.
Component version
v1.39.0
Log output
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: