Skip to content

Support dill 0.3.6 #5166

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Oct 28, 2022
Merged

Support dill 0.3.6 #5166

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
0716b97
Remove fix on dill >= 0.3.6
albertvillanova Oct 26, 2022
74832b6
Unpin dill
albertvillanova Oct 26, 2022
9637e49
Fix dill save_function for dill 0.3.6
albertvillanova Oct 26, 2022
a363eda
Fix dill save_code for dill 0.3.6
albertvillanova Oct 26, 2022
5cbc6cd
Pin dill<0.3.7 to support it explicitly in their next release
albertvillanova Oct 27, 2022
b3c6c88
Fix CI to test latest dill
albertvillanova Oct 27, 2022
31ae725
Fix style
albertvillanova Oct 27, 2022
694a624
Fix dill _save_regex for dill 0.3.6
albertvillanova Oct 27, 2022
d636609
Fix _save_regex
albertvillanova Oct 27, 2022
188a45f
Uninstall apache-beam for CI latest dependencies
albertvillanova Oct 27, 2022
eb886fa
Merge remote-tracking branch 'upstream/main' into fix-5162
albertvillanova Oct 27, 2022
b460f6d
Fix uninstall of apache-beam
albertvillanova Oct 27, 2022
6b04ba7
Remove raising error when pickling logger
albertvillanova Oct 27, 2022
e336437
Use pickable MagicMock when patching
albertvillanova Oct 27, 2022
278064a
Fix style
albertvillanova Oct 27, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,10 +64,12 @@ jobs:
pip install -r additional-tests-requirements.txt --no-deps
- name: Install dependencies (latest versions)
if: ${{ matrix.deps_versions == 'latest' }}
run: pip install --upgrade pyarrow huggingface-hub
- name: Install depencencies (minimum versions)
run: |
pip uninstall -y apache-beam
pip install --upgrade pyarrow huggingface-hub dill
- name: Install dependencies (minimum versions)
if: ${{ matrix.deps_versions != 'latest' }}
run: pip install pyarrow==6.0.1 huggingface-hub==0.2.0 transformers
run: pip install pyarrow==6.0.1 huggingface-hub==0.2.0 transformers dill==0.3.1.1
- name: Test with pytest
run: |
python -m pytest -rfExX -m ${{ matrix.test }} -n 2 --dist loadfile -sv ./tests/
Expand Down
2 changes: 1 addition & 1 deletion setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@
# Minimum 6.0.0 to support wrap_array which is needed for ArrayND features
"pyarrow>=6.0.0",
# For smart caching dataset processing
"dill<0.3.6", # tmp pin until 0.3.6 release: see https://github.com/huggingface/datasets/pull/4397
"dill<0.3.7", # tmp pin until next 0.3.7 release: see https://github.com/huggingface/datasets/pull/5166
# For performance gains with apache arrow
"pandas",
# for downloading datasets over HTTPS
Expand Down
Loading