-
Notifications
You must be signed in to change notification settings - Fork 191
/
Copy pathchange.log
1237 lines (738 loc) · 72.3 KB
/
change.log
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
18/03/2025
- Launcher now halts if both photo_album and photo_library are configured as these options conflict with each other
20/02/2025
- Launcher now checks that **download_path**, **jpeg_path** and **video_path** exist in /proc/mounts (if they are configured0. This should confirm that these directories are mounted to external locations at container launch. The .mounted file is still a requirement, as this makes sure the container is looking at the correct external mount and also that the volume hasn't dismounted due to errors.
- Remove trailing slashes from variables which could have them, like download_path, jpeg_path, video_path, the Nextcloud URL and target dir
- A load of stuff I lost track of regarding conversion of jpegs and stuff. I'm a bit tipsy and probably broke a bunch of stuff. Previously there was an ugly bunch of hardcoding around HEIC/JPG file extensions... case insensitive searches, but ignoring ".heic" files etc. This uncovered a bunch of other areas where I'd copy/pasted code... so for sideways copying videos and stuff. Find other things which just didn't work in testing at all (unescaped characters in bash expansion). I've no idea ho this worked previously... maybe it's a bash/ash difference which will **** things over in side the container. Yes, I'm an adult, I can put **** if I want to, so go **** yourself if you don't like it.
- Rebased to Alpine Linux 3.21.3 coz apparently that's a thing now.
- Aw man. This just feels really broken, but it's not chucking errors out everywhere in my testing. Sod it. I'll push it to Dockerhub and deal with the fallout tomorrow. I have stuff to do on Saturday so no drinking tomorrow for me. Can fix stuff then if everything blows up.
19/02/2025
- Made notification_type case insensitive. Some dude was using "discord" instead of "Discord". Both will work now.
17/02/2025
- Setting the umask via /etc/passwd didn't work as expected. Now configuring via /etc/profile as it should be consistent whether using interactive or non-interactive shells.
16/02/2025
- Now setting umask based on file permissions. This should result in files being created with correct permissions, if they are created with a non-default umask (022), rather than relying on the functions to correct permissions being run.
- Change the variable name save_ifs to OLDIFS. Think I like that more.
13/02/2025
- Changed the launcher to check that notifications are configured correctly pre-launch.
- Fixed Nextcloud URL log display which was missing a / character between the webroot and desination directory
05/02/2025
- Changed the Telegram bot check so that it only occurs once, during the launch script. Once it successfully verifies the bot, it updates a new variable in the config file and will never check again. For some reason I see it failling to verify my own bot occasionally. I think the check fails randomly for some reason. I've no need to do it each time though as the update number is tracked via file in /config so it should be able to work even if verification fails.
02/02/2025
- Amended initialisation script to hang if it can't create the temporary files
- Changed the way it pulls info about container; using cut to return fields rather than sed to remove the thing I grep'd for
- Container no longer hangs indefinitely if Internet connectivity fails during initialisation. Now waits for 5mins before restarting
- Added 1min wait if version check does not succeed so that it give the end user time to notice the problem
25/01/2025
- Changed the description for --keep-icloud-recent days warning.
23/01/2025
- Bump to icloud-photos-downloader v1.26.1
21/01/2025
- Launcher now waits for apple_id to be added to icloudpd.conf and continues once it's added. This is so I can create test containers with docker --rm and be able to add the apple_id variable and continue using the container. This means I don't have to pre populate a static volume and icloudpd.conf file
20/01/2025
- New feature... Sideways copying of videos. This tells the container to copy/move all the videos it downloads to a different directory. It also has a command line switch so that it can be run against an existing library.
19/01/2025
- Rebase to Alpine Linux 3.21.2
- Scrapped using bash and gone back to ash. It just adds a bit of bloat and I've not used shellcheck as much as I expected to.
- Moved the bits where it creates the requied directories and checks for updates to the launcher, so that it only occurrs during startup. I want the launcher to become a startup script and put all the "pre-flight check" stuff in there... such as checking the telegram bot is enabled, as this has a short pause in it and is annoying when it checks it each time you renew MFA. Same with all the DNS/route cheks. Files permissions. Also check config files variables aren't going to cause issues.
- Massive re-working of initialisation portion of script. It is all performed in the launcher so it is only performed once, instead of each time the sync-icloud.sh script is run. This should make manual execution a lot quicker.
- Disabled all the permissions stuff in the main script. The initialisation script should make sure the /tmp/icloudpd, /config, download and JPEG directories have the corect permissions. As long as I make sure the script switches the user correctly when creating new files, all should be good. Did this quite some time ago, but never disabled the permissions stuff. Hopefully it will work.
- Massive bunch of changes in an attempt to make sensitive items partially obsucred so that they may still be verified by someone who knows the info, but still unrecognisable if it gets posted in part of a log file on GitHub
18/01/2025
- Found a bug in the helthcheck code. If the container is uninitialised, then it can't read the exit code checks. If these files don't exist, then presume healthy, as the container should be waiting for user input.
14/01/2025
- Something not right with the new option as it was enabling by default and configuring a value which would delete all files from icloud.com. Have changed the init_config.sh script so it doesn't set a defualt of 0, but defaults to blank and also so that it invokes the command on the presence of the variable.
- Changed the command builder to use an array
- Change the logging to not display the keep_icloud_recent_days variable if it is not set
- Added an extra variable, keep_icloud_recent_only which has to be switched on to enable keep_icloud_recent_days. This is a dangerous feature and enabling it can just blow away entire icloud accounts. Also added two minute warning so it can be cancelled before any damage is done.
07/01/2025
- Updated parameter name **keep_recent_days** to **keep_icloud_recent_days**.
06/01/2025
- Changed some IFS declarations so they're all consistent (echo \b\n to $'\n')
- Re-implemented Synology Photos App Fix which hadn't been brought in since the move to icloudpd.conf file configuration
- Updated the bug report template
05/01/2025
- Added **keep_icloud_recent_days** option.
04/01/2025
- Messed about with the Telegram polling times so that when re-authenticating, the polling frequency increases so it's more responsive. Polling every 60s makes re-authentication take about 3mins. Should now be loads quicker.
- Also noticed that additional sleeps in the Telegram polling sequence would mean the sync time slips by 2 seconds every 60. I've not checked logs to confirm, so this is only a suspicion. Have changed it anyway so that it should now be more accurate and slippage should only be by the amount of time it takes to process all the commands... so hopefully can be measured in miliseconds.
29/12/2024
- Bump icloud-photos-downloader to v1.25.1 and pushed to GitHub.
25/12/2024
- No update, just Merry Christmas!
18/12/2024
- Another ImageMagick bug... grr.
- Added additional checks at container first launch when creating the /config directory and /config/icloudpd.conf file. Script will now hang before initialisation if checks on these locations fail.
15/12/2024
- Still seems like there is a bug in the JPG file conversion code. Doesn't seem to create the subdirectories it needs. Likely only ever worked if folder_structure was set to none.
13/12/2024
- Imagemagick changed the "convert" command to "magick convert". I'm sure they must have a reason for it, but it seems a little daft... and annoying.
09/12/2024
- Well that was weird. I just re-built the "upload library to nextcloud" function using the newly reworked "create directories" and "upload files" functions as a template and the function worked first time in testing. Ther'es gotta be something wrong with it, surely?
08/12/2024
- A bunch more Nextcloud stuff done over the weekend. All functions now reworked except for the one that uploads the entire library. Not as much space saved as I'd liked. The upload function was actually pretty well written, plus I broke them into separate functions for creating directories and uploading files. I also found a potential issue regarding case sensitivity and the HEIC conversion tool, so added a bunch of code to handle that. Hopefully I'll get some time to finish the last function tomorrow/Tuesday.
05/12/2024
- Bit of work regarding the Nextcloud stuff. Rewritten the function that creates the directories prior to uploading. It now generates the list of encoded directory paths before attempting to create them. Seems a lot cleaner. Code has gone from about 40 lines to 30 lines, a reduction of over 200 characters. Should be able to make similar gains surrounding the nextcloud_upload, nextcloud_delete and double the amoutn for the upload_library_to_nextcloud functions... iirc.
- Also, Alpine Linux 3.21.0 has dropped so rebuilt with that.
03/12/2024
- Bump icloud-photos-downloader to v.125.0 and pushed to GitHub.
- Then found nextcloud upload handling doesn't work when the album name has a space in it. I've got directoy creation and upload working, but it's an absolute mess. I've no idea how it's actally working and seems way, way overly complicated. Needs some time spending on it.
24/11/2024
- Did remember.
23/11/2024
- Well, seems like remote auth was royally fucked with the switch from ash to bash due to the regex matching. Think it's fixed. Pretty sure there were PRs surrounding the photo_album and photo_lirary features that used the same regex based matches. they could be broke, but it's late, so I'm going to bed. I'm also a bit drunk so I'll probably forget to check if its broken tomorrow.
22/11/2024
- Bump to iCloud Photos Downloader v1.24.4
10/11/2024
- Changed the format of a lot of code.
- Added bash shell and now running scripts in bash. This is because I've started using shellcheck to test the code and it doesn't support Alpine.
22/10/2024
- Removed all the "Change: " prefixes in this file. I originally planned to have different stuff, and there probably are a few still in here, but they serve no value.
- Changed the search for "True" and "False" in the config file replacements so that they are now case insensitive. TRUE will also now be changed to true. As FaLSe will be changed to false.
13/10/2024
- Issue reported with a "] if statement preventing launch. Odd, as greppimg the main script for it returned a second result which must have been there for many, mant releases.
11/10/2024
- Added the **webhook_insecure** option to allow use of self-signed certificates.
- Some logging changes.
17/09/2024
- Added PR from @Teknicallity which fixes a rare condition which would lead to an empty download_path variable and some other logging output changes.
16/09/2024
- Rebase to Alpine Linux 3.20.3
- Installed libheif 1.18.2-r0 from Edge repository which should allow for HEIC to JPEG conversion on iOS 18
11/09/2024
- Merged PR submitted by RICwang which adds the Chinese translations for the remote sync feature and also enables the feature.
08/09/2024
- Upload Library to Nextcloud function now checks for the existence of the file and skips if it's already there. Hopefully this wil speed-up the time it takes, at the expense of overwriting what's already on the server.
- Amended Nextcloud offline notification to contain username. I've had two notifications for a possible five accounts. Not sure which accounts to look at.
- Added a bit to detect whether a command line parameter has been specified. If so, it sets telegram_polling to false, so there's no delay when launching the script with command line parameters such as init and upload-library-to-nextcloud.
02/09/2024
- Bump to iCloud Photos Downloader v1.23.4
01/09/2024
- Bump to iCloud Photos Downloader v1.23.2
- Telegram remote authentication is working perfectly now... only related issue is that it's spamming the logs. Need to figure out how to quieten it down. Task for another day, I think. I've done so many MFA re-authentications today, it won't be long before Apple block me again.
- Bump to iCloud Photos Downloader v.123.3
- Fixed an odd bug that when the container is waiting for authentication to complete, the cookie was being removed. Think this behaviour has changed somwhere down the line. Previously I think the file was being overwritten, no I think it removes the file at the start of the process. This resulted in a grep command not being able to find the file. Now, set the grep command to silence errors and just echo a value indicating non-detection instead.
- A buncu of remote authentication tweaks. It's looking a lot nicer now. No debug info printed and logging nicely too.
31/08/2024
- Fake user agent to fix IYUU notifications in China.
30/08/2024
- Latest version of icloudpd changed the way that the phone numbers were output (specifically the ***** *****) part, so the remote auth feature would not fire out the Telegram notification with the number on.
26/88/2024
- Reworked Telegram notifications so that they have a base URL instead of notification URL, this is to remove duplicated lines of code surrounding notifications and polling.
- Amended health check to instpect the same cookie expiry value as the main script.
- Changed log output so that next synchronisation dates which are more than 20 hours in the futuer display the locale's data and time.
24/08/2024
- Now possible to configure Telegram to use HTTP instead of HTTPS by setting **telegram_http=true**
23/08/2024
- Amended some error logs.
- Amended cookie logic to pull expiry date back for X-APPLE-WEBAUTH-USER instead of X-APPLE-WEBAUTH-PCS-Photos. Seems the latter's expiry date is extended upon each sync where the former is the true expiry date of the cookie.
22/08/2024
- Moved the logic for creating the files in the JPEG path. Seems it was attempting to create a blank directory for people who did not have this configured.
31/07/2024
- Might have figured out the Telgram remote authentication expect scripts bollocks.
23/07/2024
- Possible bug with jpeg_path veriable. Have added some stuff to check and set permissions.
21/07/2024
- Got rid of the config_dir variable and hardcoded /config. Some systems dont' seem to be honoring it, even though it's specified in the container.
- Bumped to v1.22.0
- icloud-photos-downloader version output text has changed. Updated command so it's correctly displayed.
10/07/2024
- Bug with Alternative size.
09/07/2024
- Missed one of the new photo sizes. Alternative. Duh.
08/07/2024
- Reworked user/group creation. As the config file is now user to specify the user/group and not variables, it would default to creating a user:1000 and group:1000 for the user and group. Then a user could come along and change say the username, but it would fail to change this due to the UID already being is use. Script has been changed to overwrite the /etc/passwd and /etc/group files with minimal ones (only a root user and only root, tty and shadow for groups). It will then create the new user. This should allow much better user switching within the container.
- v1.19.0 brought in support for downloading multiple sizes of photo. Have now updated the container to support that too.
06/07/2024
- Bumped to v1.21.0
03/07/2024
- Seems one of the error logs was not using the error log function, but debug log function... fixed.
- Got rid of full stops on ends of some log lines. What moron puts full stops at the end of lines nowadays?
- Couple of issues in the init config file. debug_logging=AAAAA was one of them. No idea how that got there. Plus group_id=group. That's prob gonna break things.
- Anoher config file related change. I now set the download_path variable default in the config file where previously it was blank. Now it checks for the user variable in the config, if there isn't one, it uses the user variable, if there isn't one of those either, it defaults to /home/user/iCloud.
01/07/2024
- Bumped to v1.20.4
- Script now checks for presence of config file before running. Exits if it doesn't exist as it indicates failure to write to /config, the directory where the container keeps its config files/cookies/keyring etc.
- Removed some commented out code relating to different virtual environments when china_fix branch was also implemented.
- Changed the cookie creation command from performing a dry-run and downloading 0 recent photos, to just performing the authentication only option.
- Added an extra test to init_config.sh as changes made on 24th June did not account for first run.
- Added an update check to startup.
30/06/2024
- Added a bunch of stuff to set defaults for empty variables in icloudpd.conf.
- Added force_gid to config file as it was missing.
- Added live_photo_size too.
- Removed some code relating to the old NextcloudCLI container I've deprecated a long while back.
29/06/2024
- Bumped to v1.20.3
24/06/2024
- Think I fixed an issue where files with spaces in the name won't upload to Nextcloud... edit: yeah, pretty much sorted it.
- Added a test to init_config.sh to check file permissions before launch.
23/06/2024
- Not uploading to Nextcloud if the filename has a space in it. Made amendments to see if I can work out why. Seems that spaces are not being changed to %20. Made some logging changes to to check.
- Found and fixed the Nextcloud upload issue.
- Just found that the function which deletes empty directories wasn't actually implemented correctly. Now fixed.
17/06/2024
- Bumped to v1.20.0
- Amended init_config.sh so that it won't match two search results when searching for variables e.g. 'name=' previously matched 'name=' and 'nextcloud_username='.
- Amended the profile so that the path variable now includes the new location for icloudpd.
- Amended startup part of main script to get rid of default values in script. All these should now be set by the init script and pulled from the icldoupd.conf file.
- Implemented the --keep-unicode-in-filenames functionality.
- Implemented the --live-photo-mov-filename-policy functionality.
- Implemented the --file-match-policy functionality.
- Implemented the --align-raw functionality.
- Was counting lines of the download check with grep. Swapped to wc, just in case something iffy is happening with "grep -c ^".
10/06/2024
- Fixed bug introduced by SMTP notifcation.
- Fixed bug in config file initilisation script.
06/06/2024
- Bumped to v1.19.1
- Merged pull request from @klesh to add SMTP notification support.
01/06/2024
- Bumped to v1.19.0
28/05/2024
- Bumped to v1.18.0
26/05/2024
- Just removed some obsolete stuff from the dockerfile.
- Added date to build number. Some dude was running a version that was 3 years old. I removed the date from the build number a year or two ago, coz I thought most people would be fairly up to date. Obviously not, so I've put it back. Will help identifiy people doing the same thing.
- Changed the date format due to the change above, and also coz it's just easier to read when units go from smallest to largest. Reverse dates work best on filenames so they are automatically stored chronoligically, but that's not really needed here.
25/05/2024
- Bumped to v1.17.7 and relocated python venv from /opt/icloudpd_latest to /opt/icloudpd as I'm no longer installing two versions into the container. Also... noticed Alpine is now at version 3.20
24/05/2024
- Bumped to v1.17.6
16/05/2024
- Amended the log output for when an error is detected to instruct the end user to gather a debug log before posting the issue to GitHub. Too many muppets posting a single line of error message and expecting me to know what they've messed up to break it in that way.
09/05/2024
- Added a check to see if the download path is writable. A lot of users have reported seeing the error: "Invalid value for '-d' / '--directory': Path '/download_path' is not readable" and although some users it appears to be due to SMB file mounts, I suspect it is the age old issue that their permissions are just inadequate... like SMB permissions don't just have file permissions, they have user permissions via the share itself, and it's likely just that. I now check that the download directory is writable. If not, then I attempt to set the permissions as root for the configured user, then re-test. If it fails, I just bail and kick out an error.
05/05/2024
- Added nano, nano-syntax and findutils to the container. BusyBox fnd doesn't support -printf and I need that to add the nano syntax highlighting stuff to /etc/nanorc.
- Altered the script to add some delays in where people attempting to run as root. Running as root is not supported as this script expects to switch user at times and it breaks the su command.
- Added /opt/icloudpd_latest to path so icloudpd and icloud can be run without having to cd to their locations.
04/05/2024
- Some stuff to make command line life easier.
- Removed a bunch of redundant stuff from the Dockerfile.
28/04/2024
- I got rid of the bit which attempts to detect TCP connectivity to a DNS server which curl has been found to use. It seems that for some people, curl works just fine with UDP lookups. The people who are having problems with curl can firgure out their issue instead.
- The reauth script wasn't setting the cookie directory, so the re-authed cookie would land in the root user's home directory instead. Also, the re-auth command would be run as root, so permissions could get messy. Changed script to use a run_as function, so it's runs as the configured user.
- Have modified the script so that it handles the new behaviour for remotely authenticating when using SMS based MFA. Haven't tested it much yet, so probably broken lol. I need to push to Dockerhub so I can test on the accounts I do SMS MFA from, as these don't run my testing builds.
05/04/2024
- Amended logic for remote authentication coz the first way I did it was pretty stupid. This allows for users to have numbers in their Apple ID, which is nice.
31/03/2024
- reauth.sh script now checks for auth_china variable and configures authentication domain.
27/03/2024
- Telgram messaging can now re-authenticate your icloud.com account.
- Added reauth.sh script so that re-initialising the cookie is easier.
20/01/2024
- Gotify can now be configured for HTTP or HTTPS connections.
07/01/2024
- Merged PR from @timrettop which fixes HEIC to JPEG conversion in Alpine 3.19.0
- Created option to disable the startup notification.
- Removed the silencing of the set_mempolicy error. The convert application does not generate these errors any more after the jpeg package was installed in the container. It seems that the convert application was just converting the HEIC file to another HEIC file, but with the JPG file extension. Installing the package means the conversion processes a lot more quickly and correctly outputs the files.
- Made the functions which convert HEIC files to JPEG, remove all HEIC files, etc do case insensitive searches. This should conver *.HEIC and *.heic to *.JPG.
06/01/2024
- Rebuild container to icloudpd 1.17.3
01/01/2024
- Altered how Nextcloud path is built. Seems there are double trailing slashes. Now removes the one in the variable and appends one manually.
- Seems like deleting from Nextcloud wasn't working correctly. Was always coming up with "File not found" which is bizarre coz it defo worked when I first implemented it. I've changed it so it uses the same encoding method as uploads, instead of doing it using a standard file path (which wouldn't work for files/paths with spaces in their names). Anyway, seems to be deleting things OK again now.
- The curl command performs TCP DNS lookups, whereas everything else like ping and nslookup uses UDP. If TCP requests are blocked to the DNS server, the container will pass the DNS checks, but then the curl commands to send the startup notifications will hang/fail. Bind-tools has been installed so that the full version of nslookup can be used and force the requests over TVP (using the -vc parameter)
30/12/2023
- Removed a log entry which was malformed.
- Output value of bot_check into debug log.
- Added random delay of 0-15 seconds (inclusive) if Telegram polling is enabled. I believe starting multiple containers at the same time and them all hitting the APi at the same time may cause some requests to be ignored.
27/12/2023
- Documented the --remove-all-jpgs command line option. It's been there ages, not sure why I never documented it. Probably because it could delete data that might be needed and I don't want people losing stuff.
24/12/2023
- I think the upgrade to Alpine 3.19 may have broke Imagmagick. Adding the additional package, libheif-dev, got it working again.
- Also, found that due to the newer version of ImageMagick, an error is generated when converting HEIC to JPEG. The conversion still works, so I've just silenced that error, specfically.
20/12/2023
- Upgraded to use Apline Linux 3.19.0 and icloudpd 1.17.0 which now includes the authentication fix. Won't work for China as they were having to use an old version of iCloudpd which is packaged into my container. New version may work ok, but script will need significant re-write for that to occur. Probably look at it over Christmas.
19/10/2023
- Added a new Nextcloud URL encode function. If a user is downloading to Photo Albums, or Shared Libraries, or even just has files with special characters (like spaces) in them, then the Nextcloud URL needs to be encoded.
18/10/2023
- Trailing slashes on $download_path messes up Nextcloud uploads. Added a bit into the upload function which should account for this.
10/10/2023
- Downloading albums not working correctly. Fixed it up and added a bunch more logging. Probably needs sames for libraries function.
- Done the libraries function too.
09/10/2023
- Bug found in health check... Was using wrong filename for error checks.
- Another bug in health check... Filename followed with a } making it invalid.
- icloudpd seems to error with an exit code of 0 when MFA is required but aborted due to being a non-interactive session. Have now added a check to see if the error report file has size.
05/10/2023
- Some users attempting to set the user for sync to root. This breaks things as it can't create the local user account or switch users correctly, so now it will default to user:1000 group:1000 if root:0 is detected.
04/10/2023
- Just some logging stuff.
- Found a couple of bugs surrounding downloading from photo albums/libraries. Squished.
02/10/2023
- A bunch of stuff regarding the new shared library support.
- Fixed missing --enable-debugging and --disabled-debugging launch parameter validation.
30/09/2023
- Found a bug which was causing videos to be skipped. Fixed it.
29/09/2023
- Found that the path of the python_keyring firectory can be configured by setting the XDG_DATA_HOME variable. Configured that and then removed a bunch of code relating to moving it from /home/${user}/.local.
27/09/2023
- Thought I'd just commented out all the lines required for the shared library downloads. Turns out I'd removed the bit which validates command line options, so re-added those.
- Also, realised that there was a mismatch between what the allowed command line variables were, and the actual variables. This affected all variables with more than one word in them, except for the latest one I added to upload Nextcloud files.
26/09/2023
- Re-enabled the libraries feature now upstream code has been merged to support shared libraries. Untested though, as I don't use libraries myself.
20/08/2023
- Added a bit to create some extra temporary files as some people are having issues with them.
14/08/2023
- Added Nextcloud connectivity check. My Nextcloud container was paused during a large offsite backup, so Nextcloud connectivity borked. This meant fails failed to upload and will not longer be retried. I can fix by running the function to upload the entire library, it's just a bit of a ballache, so I've added a connectivity check which will wait for the Nextcloud server to come back online. It will send a notification on the first failure, then again after an hour.
- Added '--help' command line option which displays the configuration information. This is coz I couldn't remember the name of the function which uploads all files to Nextcloud when I was writing the above change log entry lol.
02/08/2023
- Amended the code so that it now checks for, and removes empty folders from the Nextcloud server after it's removed stuff from Recently Deleted. Works on the test uns I've done. Not nuked my photo store, which is good.
- Also amended the code which parses the download logs for downloaded/deleted files. I used to extract column 5 to know what files to process. This doesn't work for file paths with spaces in them. I've now amended it so that it cuts from column 9 upwards, which should give me the whole file path, including spaces. Haven't tested anything yet. Wish me luck. Wow. It worked first time! Tested downloads, deletes, uploads to Nextcloud, deletes from Nextcloud. Life is good.
- Written a function which allows uploading of the entire library to Nextcoud in one go... Invoke with sync-cloud.sh --upload-library-to-nextcloud.
- Amended the command line parameters so that ones that are multiple word, have spaces separated by hyphens like the command above.
- Removed a few more checks to see if variables are present, over an acual value of true/false.
01/08/2023
- Well, that's the code in to delete empty folders. It's a bit of a hack. Basically I parse the XML that is returned and count the number of links it contains, if it only has a single link (a reference to the folder itself) then the directory is removed. If there is more than one link, the directory is left in place. This is probably going to delete loads of stuff by mistake, but here goes.
31/07/2023
- So, it turns out that when uploading to Nextcloud via WebDAV, the file paths are not automatically created when you 'put' a file. In addition to that, when you create a pathname, you can't automatically create all the folders in the path at once (e.g. mkdir --parents /folder1/folder2/folder3). New change now creates a unique list of the directory names from the list of newly downloaded files. It then checks if each directory in turn exists, and if not, creates it. I guess I need to look at pulling the properties of the folder, checking if it is empty, and if it is, removing it during the file removal function. Otherwise I may end up with empty folders on my Nextcloud server.
24/07/2023
- Simplified the logic for auth_china. I want to eliminate chcking for the presence of a variable as being **true** and absence of a variable as **false*. This results in a tendancy towards **true*, because if it contains any value at all, it's considered **true**. Someone had configured a variable to be **false**, which means it is technically **true**. Logic is now that `if [ "${auth_china:=false}" = true ]`. This kinda flips it. Now, it's only true if it's set to **true**. If it's set to **motorboat** it's now technically **false**.
23/07/2023
- Changed how the icloud_china variable is processed. I was just checking if the variable was present, so setting it to **false** was equivalent to **true**.
- Oops. Didn't test that one very thoroughly. Forgot that notifications check if the **icloud_china** variable is empty if they want to send English notifications. Pushed a new version to correct that.
22/07/2023
- Did some stuff around cookie creation. Script would progress immediately after cookie file was created, however, this happens when the user is prompted to authenticate, so background script would continue as user is confirming 2FA.
- Changed 2FA to MFA. The correct term is Multi-Factor Authentication, not two factor authenticaion.
- Introduced a bug as I forgot to change the healthcheck.sh... oops.
- I'm not sure when I did this, but I scrapped the older_function thing as I'd changed how this process worked and it is no longer needed.
21/07/2023
- Added a function to check if the cookie exists. Two of my containers' cookies expired and the auto-heal feature kept restaring them. This meant when they were starting up they were attempting to connect to iCloud so they could list the available abums. This was causing an MFA prompt. New function just pauses the script if a cookie does not exist.
11/07/2023
- Synology systems create multiple directories which contain extended attributes, such as thumbnails. These will be seen as "eaDir*" directories. New variable added to skip changing permissions on these directories. PR submitted by tsanie.
09/07/2023
- Rewrite of remot sync function. Should be a little bit more reliable, as the previous one would get differeing results back from the Telegram API. Now it goes by update ID and stores it's progress to a file, so that it doesn't fail to retrive a correct value at boot.
- Added function to remove deleted files from Nextcloud.
04/07/2023
- Re-enabled **delete_after_download* functionality as recent fix has been merged upstream.
- Altered the clean up of the *notification_title* so that it does not apply to the commonly used Chinese notification methods.
- New function to Nextcloud server. This allows uploading directly to a Nextcloud server, rather than just dropping a file for a different container to monitor and then start a manual sync.
03/07/2023
- Rebuilt with iCloud Photos Downloader 1.14.2
27/06/2023
- Rebuilt with Alpine Linux 3.18.2 and iCloud Photos Downloader 1.13.4
26/06/2023
- Just chown the tmp and configu folders as some people seem to be having issues with keyring file which is solved by moving the file from the true location to the /config dir location. Possibly a permissions issue. Not sure.
- The Telegram polling is still acting weird. Sometimes, the latest message reported by the telegram API will actually be an older message, then a second call to Telegram's API will return a higher number. Also, after a while, all messages will just disappear, meaning null values are passed back. Hopfully these should be fixed now.
- New variable added **albums_with_dates** which should allow albums to be downloaded to folders with the album name, and then split by date.
22/05/2023
- Send notification when remote sync command received.
04/05/2023
- Out of range issue polling for Telegram wake command.
26/04/2023
- Added **skip_album** variable so that certain albums (such as the Apple default ones) can be excluded from the download.
25/04/2023
- Unexpected behaviour when latest message is an update about group membership. This change will set ${latest_message} to ${current_message} if ${latest_message} is empty.
- Now supports downloading from multiple folder albums, and saving them into folders named accordingly.
24/04/2023
- Bug fix for folder structure.
- Possible solution for Telegram polling issue.
23/04/2023
- Set the function to configure file ownership to use user and group ids (instead of names) as this was failing to set the correct permisisons prior to the accounts being created (brand new containers).
202304022
- So, I messed up and accidentally pushed a version I was working on, to Dockerhub and to Github, which contained my Telegram API token. I had to delete the latest tag from Dockerhub, reset the master branch on GitHub and rotate my Telegram token. What a pain in the arse. That means this latest version is a little rushed, so don't be surprised if it's broken. It does bring within it a cracking new feature though... I've now coded it so it polls the Telegram chat (the one it normally sends notifications to), and if the latest message is the same as the **user** variable, it will force an immediate sync. That way, if you've taken a load of photos on your phone, you can tell your NAS at home to sync the queue once you're done.
- This is just a change to the documentation. There has been a variable for Telegram **telegram_silent_file_notifications** which sends the telegram notifications for download/delete previews silenetly, I just noticed it's not been documented so I thought I'd change that.
- I've re-enabled the **delete_after_download** option as it's been implemented correctly upstream and newly released.
- I've also removed the i386 build platform from my images. They're failing to compile as they can't build a wheel for cryptography. I'm hoping nobody complains, but it they do I'll look at trying to re-add it in.
10/04/2023
- So many changes, I've forgotten most of them... Um... now I use Python virtual environments to install mutliple versions of icloud-photos-downloader... It incorporates v1.12.0, which is used for downloading photos globally. It also uses a modified version of v1.7.3 which allows authentication against the China servers. Also rebased the image to use Alpine 3.17.3.
03/04/2023
- Changed any "True/False" variable to "true/false" and also find/replace title cased variables in icloudpd.conf to lower case.
01/04/2023
- Some stuff around how build tags are handled in docker buildx. Mainly so that I have versioning in Dockerhub, instead of only a single "latest" container.
- Delete after download option removed due to bug where it deletes files from iCloud that have not been downloaded yet.
01/04/2023
- New build refers to --china-mainland rather than --domain cn.
- FFS, I typo'd it as --china_mainland.
31/03/2023
- Rebuilt using icloudpd and integrating PR 608 https://github.com/icloud-photos-downloader/icloud_photos_downloader/pull/608 in an attempt to fix the on going China authentication issue.
30/03/2023
- Got rid of auth_china variable, as it doesn't work.
29/03/2023
- Altered the way the log file creation works so that it can add missing/future variables and sort it into alphabetical order.
28/03/2023
- Urgh. It's filth, but it might just work... Docker container downloads a forked copy of icloud_photos_downloader which has been customised to work with China's authentication servers. The cookie generation function will now swap out/in these versions of the files if the auth_china variable is set to True, or just set really, I'm not fussy on this one.
- Added a --enabledebugging and a --disabledebugging command line option which will edit the config file to enable/disable debugging for people who may find editing a file a little trickier on some systems... I'm thinking NAS devices may not make this so easy.
- Made it so that the script reads the value of the debug_logging variable prior to each sync. That way debug logging can be switched on using the command line and it will apply on the next run.
26/03/2023
- Complete overhaul of logging system. A DEBUG log now introduced which removes a lot of the standard logging and puts it behind this option. When it's enabled, it will hide some of the sensitive data such as e-mail address, Apple ID and tokens.
- Complete overhaul of variables. Docker variables are now pushed into the icloudpd.conf log file stored in the config directory. This allows users to toggle options by just changing the log file and restarting the container. This will allow people to switch on/off deug logging, or change the notification method without having to remove and recreate the container.
- The --ConvertAllHEICs function was missing a command to create the destination folders. This broke the cuntion is being used in conjunction with the jpeg_path variable.
15/03/2023
- Rebuild to Apline Linux 3.17.2 and icloudpd 1.12.0
- Added delete_after_download variable. This moves all downloaded files to 'Recently Deleted'. Cannot be used with auto_delete variable.
- Added support for Telgram proxy server using the variable telegram_server.
08/02/2023
- Added function to delete all JPGs that have an accompanying HEIC. Use --RemoveAllJPGs to remove that data. Be warned, JPGs that have matching names with a HEIC file in the same folder will be removed. They may not necessarily be the same photo. Use with car and only if you know what you are doing.
28/12/2022
- Added details in readme warning users about enabling Apple's new Advanced Data Protection feature in iOS 16.2
- Changed the way permissions and ownership is applied so that it it only applies permissions to the download directory after a sync, rather than at multiple different times.
- Slight change needed to user creation due to rebase to Alpine 3.17
03/12/2022
- Added --Init so I'm less likely to typo it.
- When downloading from libraries it now lists available libraries to the log file.
- The iOS16 shared libraries patches have been moved to a new container tagged as "testing", something isn't quite right as many people are reporting errors.
21/11/2022
- iOS 16 shared libaries can now be downloaded with the --library option. PR courtesy of Patrice Neff https://github.com/pneff.
- Added support for WeCom proxy via PR submitted by Alano-i and Sowevo.
08/10/2022
- Incorrectly named variable. changed bark_key to bark_device_key.
26/09/2022
- Amended the log entries so that the warning about setting icloud_china variable only appears if it's not already set. Also changed it to prompt the user to log into iCloud.com and make sure there are no pop-up notifications.
23/09/2022
- Bug in --RemoveKeyring preventing it from launching.
12/09/2022
- Documentation updated, changed --DeleteKeyring to --RemoveKeyring.
- Added feature to place converted JPEGs into different folder from the download location.
08/09/2022
- Keyring file wasn't actually being removed during container initialisation... Bizarre. Always thought it was. Maybe I removed it while ago, but forgot. Added a --DeleteKeying command line switch to remove the keyring file, if it exists.
28/08/2022
- Bark modifications.
27/08/2022
- Amended error relating to unindexed an library. Error says to wait minutes, it should say hours.
24/08/2022
- Added Bark notifications (untested)
- Two fixes required for Python 3.10+ support.
- Pull request: Merged PR from rmlanghopto to add Gotify notifications.
28/07/2022
- Set the function, which corrects the owner and permissions, to run after a successful download. Now permissions are corrected immediately after files are downloaded, rather than at the start of the next scheduled synchronisation.
- Moved the Nextcloud trigger to the donwloaded/deleted files notification sections (as it's essentially notifying Nextcloud that it needs to sync), but also because where it was currently executing, meant that it would only trigger a sync on successful downloads, not if files had simply been removed. Now it will trigger a Nextcloud oneither of those two events.
27/07/2022
- Added trigger_nextlcoudcli_synchronisation variable.
20/07/2022
- Changed the notification events so that they can have a larger range of icons and different ones too.
- Gone live with some changes to WeCom notifications.
- Amended the find command to ignore symbolic links when changing the group on downloaded files and their folders. This is due to the Synology Photos app creating symlinks in the download folders.
19/07/2022
- Partially added some Chinese translations, still a few questions about some other changest that have been requested, so holding off on all of them.
- Cookie expiration is not working for expired cookies, only the notifications about expiry. This change should change the icon in the notification to the alert symbol when cookie has actually expired.
- Reverted the changes that were put in place to support root user. Running as root user is really bad, and I probably should have just sacked it off when it was requested, so I'm putting it back to how it was because some people using unRAID devices have seen a load of issues, because they're actually running the thing as root... Why even is that?
17/07/2022
- Evaluate the home directory location of the user... If user is root, home is located at /root. On CoreElec/LibreElec/OpenElec, then the user's home directory is in /storage. This change should work for either of those scenarios.
16/07/2022
- Allow badnames when creating user id.
- UNTESTED BETA: Added WeCom, the official method of WeChat notifications.
20/05/2022
- Add option to download from single album.
18/05/2022
- Bug in logic about 2FA renewal reminders means they aren't being sent out daily. I've changed logging to help identify where the problem lies.
30/04/2022
- Added single_pass variable which runs the sync process once before exiting out.
28/04/2022
- Merged PR from eliempje (https://github.com/boredazfcuk/docker-icloudpd/pull/155) which adds Openhab functionality.
11/04/2022
- Apparently Synology devices have a Photos app which doesn't update when a download takes place. The newly created files need to be touched for it to trigger a re-index. I've added a new variable which will touch a file to trigger the update, then set it back to what it was to maintain integrity.
04/03/2022
- Added a check for the IP address lookup performed during initialisation. Previously this could fail if the DNS server was not available (due to container starting on same system as a DNS container). It will now retry the lookup for 2mins before failing.
- Changed cookie generation name from Generate2FACookie to GenerateCookie as it generates both types of cookie and is run no matter what cookie type you use. Also, changed the log output so it doesn't say "Generating 2FA Cookie" when generating a Web cookie.
- Urgh. I dunno about you, but sometimes I delete a bunch of stupid memes off my phone and it ends up leaving my folder_structure with empty directories. I've now added a variable delete_empty_directories which will tidy those up.
20/02/2022
- The timezone variable is not being set globally if it's not specified, so the underlying Python script generates a warning. The TZ variable is now defaulted to UTC in the Dockerfile which should take care of this.
- Added an extra synchronisation interval of 21600, which synchronises every 6 hours. This generates a warning message detailing the potential issue and fix, and includes a 2 minute wait on container start time to make the warning prominent.
- Pushover notifications not showing downloaded/deleted file previews.
- Added function to allow deletion of HEIC accompanying files. These would be the converted JPG files (if using the HEIC to JPG conversion feature) and the_HEVC.MOV files which come with live photos.
- Bug found with sending Telegram notifications if e-mail address has an _ character in it. The _ character wasn't escaped correctly when passing the text to the Telegram API.
- The Notify function is always called, even if notifications aren't configured. A log entry was still being processed and the logic for determining a success/failure of the sent message was being performed too, resulting in an 'out of range' error, as an empty variable was being processsed.
18/02/2022
- Cookie expiry notification period was being defaulted in the Initialise function and also for every notification type configured in the ConfigureNotifications function. Removed these as they're just duplicates. Also corrected a bug which would override user configured period, configuring it to 7.
- Logic for skipping the file check was checking if value it true, then following two if statements had to check if it was set to false. Don't really like the OR tests that needed to be done so adjusted the logic two set the check_exit code and check_files_count to values that will always perform a sync, as they should never change due to CheckFiles function never being run.
- Script doesn't seem to reliably send the "once per day" notifications during the nofitication expiry period. So have added a log entry which details the next notification time.
- Huge overhaul of notification system. Probably broken everything lol. I've tested Telegra, Prowl, Discord and Pushover. I don't have any of the other methods to test though, so can't guarantee they work, or are formatted correctly. I've standardised the way notifications are sent to the Notify function. This way, only one notification command is needed to send notifications for all types and the formatting of the messages is now handled by the Notify function.
11/02/2022
- Introduced a bug with my error handling from the last update. Now I create the error and exit_code log files in the Initialisation function of the script so that the correct permisions are applied. Problem was that the files were being created as root, and then the python script didn't have permission to them.
- Status code checking for the curl command when sending notifications would only accept 200 as a success. Discord reports 204. I've amended the checking code to just accept 2xx.
10/02/2022
- Handle errors differently. Added suggestions for authentication failure issues.
- Eliminated full stops from log entries. Some lines had them, others didn't, so standardised on not having them. We don't need them. They take up precious bytes. They giving me flashbacks to COBOL.
09/02/2022
- Broke out logging into three separate functions. Just because. I dunno. Having tonnes of $(date '+%Y-%m-%d %H:%M:%S') commands in the script just looked a bit weird. Plus if I even need to change it, it's three lines instead of a find/replace of a bazillion.
- Set ConfigureNotifications function to check for presence of Dingtalk token, instead of ID, to decide whether to fail or not. Same as the other methods (except webhook, which doesn't use tokens).
- Altered priority for Pushover warning/error notifications. Previously they were set to -2, which gives no alert, only changes the icon badge number. Now it sends a standard priority alert for all notification types, except the one that alerts you that your cookie will run out that day, which is a "quiet hours breaking" notification (same for Prowl).
06/02/2022
- Amended the code which sends to Pushover notifications. Originally this was two separate commands, depending on whether the pushover_sound variable was sent. This is not needed, as Pushover will use the default sound is a blank value is sent for the "sound" argument.
04/02/2022
- Changed notification failure log messages to be more accurate and display http status code.
- Prowl notifications logging not quite right.
- Standard notification log message.
- Moved some stuff to prevent duplication of code.
23/01/2022
- Added variable to set custom sounds for Pushover notification type. This can be used to silence notifications.
12/12/2021
- Fix bug introduced by moving to useradd/groupadd.
11/12/2021
- Amended domain and route checks to be compatible with icloud_china variable. No point checking the route to icloud.com is OK when the container downloads from icloud.com.cn.
- Change to IYUU notifications, which should now get these working, and integrated changes to live.
- Users and Groups now created with useradd/groupadd instead of adduser/addgroup to allow the creation of users/group with an id greater than 256,000.
30/11/2021
- Final text alert for cookie expiration not received. Made a couple of changes. Will check again in 3 months time.
27/11/2021
- Added the IYUU notification type. This is a 3rd party service which supports forwarding messages to WeChat.
25/11/2021
- Added variable icloud_china so that photos are downloaded from icloud.com.cn instead of icloud.com
- Removed readme information about obsolete variable interavtive_only. The readme is nearing it's maximum size on Dockerhub, so this helps keep tthe number of characters down.
21/11/2021
- Amended healthcheck so that it does not set the status to unhealthy if notification days is less than 7. This was causing my auto-heal container to automatically restart it every 5mins. This behaviour now only occurs when the cookie has actually expired.
- Made script launch parameters case insensitive and also allowed for non-English spelling of Initialise.
24/00/2021
- Set icloud.com DNS query type to 'a' records only so that it returns the IPv4 IP address and not IPv6. Also, removed the local loopback addresses, just to remove more unwanted data.
20/10/2021
- Discord notifications added by @ibtvt.
17/10/2021
- Added detection for when a file has been downloaded, but also exists in 'Recently Deleted', so gets removed post download. In these cases, it should get rid of the missing file errors.
21/09/2021
- Added DNS lookup and traceroute to check network connectivity to icloud.com.
15/09/2021
- Removed the code which removes hyphen characters from the Telegram chat ID. The hyphen character is used to denote the bot has been added to a group to send messages, rather than a standard user chat.
13/09/2021
- Replicated recent whitespace changes to additional functions for completeness.
- Force convert HEICs functions now remove pre-existing JPG files as I don't think the convertion tool is overwriting pre-existing files.
12/09/2021
- Bug introduced by recent HEIC to JPEG change. Fixed.
- Added set owner and permissions function to run after HEIC to JPEG conversion functions.
- Added --ForceConvertAllmntHEICs.
- Fixed bug in above new command line option as it wasn't handling spaces safely.
06/09/2021
- HEIC to JPEG coversion utility has a bug and rotates JPEG files when it shouldn't. Replaced with ImageMagick which doesn't have this problem.
- Added --ForceConvertAllHEICs command line option to overwrite JPEG files. This is so over-rotated files can be replaced with crorrectly oriented versions.
02/09/2021
- Added a log entry to show the UID of the account that is running the script. It seems some users are having permissions issues and this will help with debugging.
- Also changed the symbolic link command to force linking of python_keyring directory if something that isn't a link exists in its place.
01/09/2021
- Added logging for --ConvertAllHEICs function.
- Amended way script handles expired cookies.
30/08/2021
- Amended script to wait for the cookie file to be created, then continuing once it's appeared. Container will auto restart after 30mins if file does not appear, just in case something has gone wrong.
27/08/2021
- Better way of calculating elapsed time.
- Amended the logging so it lists the expiry time of expired 2FA cookies for clarity.
- Amended the way the script waits for the keyring and failsafe files to be created. Instead of checking every 5 minutes before proceeding, script now checks every 5 seconds. This way, script is a bit more responsive when being initialised. If the script doesn't detect the file it's checking for after 30 minutes, it will exit the container, just in case there is a detection problem.
- Added logging info explaining that the download check takes a long time on large collections and that nothing is being downloaded during that phase.
- PR Merge: Changed how the webhook notification is handled for compatibility across more notification platforms.
- Script now records time taken per synchronisation and subtracts this from the synchronisation_interval. This means a sync with a 12 hour interval will start 12 hours later and not skew by the amount of time it takes the sync to complete.
27/08/2021
- Locked version of tzlocal to 2.1 as lateat version introduces dependency which is not available in Python 3.8
- Added variable manipulation on the Telegram chat ID to remove minus sign if it's included in the variable. It needs to be omitted when posting the data to the API.
01/08/2021
- Added some extra logging info.
24/07/2021
- Added function to sanitise command line parameters passed to script.
19/07/2021
- Download and Delete notification webhook payload JSON fix.
- Remove ambiguity in DownloadedFilesNotification.
16/07/2021
- Sanitise notification_title variable.
- Rebase to Alpine Linux 3.14.0
13/07/2021
- Modify 20210709 JSON escaping and fix minor change.log text.
12/07/2021
- Removed apple_password variable. Password must now be added to the system keyring.
- Default mode now to download, interactive_only variable removed. This process was far more complex than it needed to be for Synology/QNAP devices, so a small change to the default operation makes things easier for everyone.
- --Generate2FACookie command line option changed to --Initialise as it is also used for saving the password to the system keyring. Not just cookie generation.
09/07/2021
- Changed JSON escaping to prevent further issues.
08/07/2021
- Download check and actual download functions both exit with same error message. Changed for clarity about which section is failing.
05/07/2021
- JPEG quality mismatch in documentation and script. Default JPEG quality now referred to correctly in README.MD as 90.
17/06/2021
- Added script hash to make sure that the sync-icloud.sh file is correct file as date stamp seems to be affected by timezone.
- Oddities occuring when creating password/cookie on devices which always run interactively.
14/06/2021
- Accurate health check output when approaching cookie expiration date.
- Cookie expiration date checking improved.
- Wasn't setting notification period for some of the notification options.
- Notifications weren't firing in the notification period. Should now be fixed.
07/06/2021
- Code to remove empty keyring file requires logic to detect f file actually exists, otherwise brand new containers enter a file deletion loop. Logic amended.
22/05/2021
- Rebase to Alpine Linux 3.13.5
- Now logs full version ID rather than just major/minor version.
- Reddit user Vinnipinni made a comment that the documentation was great... so I've revisited it again to make sure it's up to date, corrected some spellings and grammar, and made a few improvements for clarity.
22/05/2021
- --size parameter was incorrectly set as --photo-size.
19/05/2021
- Added code to remove keyring file if it doesn't contain credentials.
15/05/2021
- New download link for Boost source as location changed.
15/05/2021
- Amended log output for when attempting to add a user to a pre-existing system group.
22/03/2021
- Amended log text after cookie generation. Script no longer claims 2FA cookie generated for all cases, but states 2FA/Web based on which was actually generated.
- Chnage: Add Litecoin address. BTC network fees are expensive AF!
04/03/2021
- Added notification_title variable.
- Readme changes to add full stops on lines where they were missing.
28/01/2021
- Added logging of $TERM variable.
27/01/2021
- Modified when the script check for the the --Generate2FACookie so that it can be invoked from NAS devices such a Synology and QNAP.
19/01/2021
- Added the --Generate2FACookie command line option.
18/01/2021
- Rebuilt on Alpine 3.13
21/12/2020
- Convert HEIC to JPEG functions now allow specifying the quality with and integer value from 0 to 100.
16/12/2020
- Sanitise apple_id variable so all letters are lower case. This prevents a case mismatch between the cookie file name that is saved, and the name that the icloudpd Python script creates the cookie file with, due to it respecting case.
04/12/2020
- Removed Pushbullet notifications. The Pushbullet service has introduced version 2 of its API so notifications no longer work. Pushbullet have also removed their app from the iOS App Store, so if it's no longer supported on iOS, there's no way to realistically support it here.
03/12/2020
- Upgraded pip.
- Pushover notification bugfix. PR courtesy of @larstomas.
- Webhook notification notification standardisation. PR courtesy of @larstomas again.
- Altered failure piority for pushover notifications from -2 to 2 so they become emergency level notifications rather than silent. Change recommended by @larstomas.
- Added variable to allow skipping of the new file check. On large photo collections, this can significantly increase the time taken to download.
28/11/2020
- Added log line to show the exact command line used to invole iCloudPD.
21/11/2020
- Due to changes in iOS14 the 2FA synchronisation interval is now limited to a maximum of once every 12 hours. It seems that 2FA authentication is now restricted to a maximum of 26 logins before re-authentication is required. Synchronisation is now restricted to specific time periods with the shortest being 12 hours.
- New variable synchronisation_delay added. This is so that the first synchronisation can be dalayed for a number of minutes. This is so that multiple containers can stagger their downloads.