Stream temp files when saving workbook#2343
Open
1436761858 wants to merge 1 commit into
Open
Conversation
Member
|
Thanks for your PR. Any benchmark data on the performance impact of the changes? Specifically, how much memory is saved, and any speed impact? I suggest test with excelize-benchmark for performance related changes. |
Member
|
The following benchmark report compares memory usage before and after the changes in this PR. The benchmarks were conducted by generating a workbook generated by xuri/excelize-benchmark containing 512K cells. Environment: Workbook generated using the following command: ./v2.11.0-master -func=StreamWriter -rows=102400 -cols=50 -n=6Base version This PR: #2343 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related Issue
#2342
Description
This change updates the save path for temp-file-backed workbook parts.
Currently, when
writeToZipwrites entries fromf.tempFiles, it callsf.readBytes(path). If the entry is backed by a temp file,readBytesopens the temp file, reads the entire content into memory withio.ReadAll, stores it back intof.Pkg, and then writes the resulting byte slice to the ZIP writer.This change avoids materializing the whole temp file in heap. Instead,
writeToZipopens the temp file withf.readTemp(path)and streams it directly to the ZIP writer withio.Copy.The saved workbook content is intended to be functionally equivalent: