Skip to content

Unable to import .gramps file to gramps web #630

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
maestrx opened this issue Mar 21, 2025 · 10 comments
Closed

Unable to import .gramps file to gramps web #630

maestrx opened this issue Mar 21, 2025 · 10 comments

Comments

@maestrx
Copy link

maestrx commented Mar 21, 2025

Ive exported my tree from Gramps AIO64-6.0.0--1 using "Grams XML (family tree)" to data.gramps file. Trying to import it to fresh new instance of self hosted gramps web:

Gramps 5.2.4
Gramps Web API 2.8.2
Gramps Web Frontend 25.3.1
Gramps QL 0.3.0
Sifts 1.0.0
locale: en
multi-tree: false
task queue: true
OCR: true
chat: false

When I select the .gramps file in the admin section "Import", I get almost immediately a red cross icon with description "Failed importing GRAMPS file".

docker output logs:
grampsweb_celery | [2025-03-21 10:06:25,251: INFO/MainProcess] Task gramps_webapi.api.tasks.import_file[b0528977-e0e9-4aa7-9121-3c0876aa65d5] received
grampsweb_celery | [2025-03-21 10:06:25,267: ERROR/ForkPoolWorker-1] Task gramps_webapi.api.tasks.import_file[b0528977-e0e9-4aa7-9121-3c0876aa65d5] raised unexpected: ValueError('Failed importing gramps file')
grampsweb_celery | Traceback (most recent call last):
grampsweb_celery | File "/usr/local/lib/python3.11/dist-packages/celery/app/trace.py", line 453, in trace_task
grampsweb_celery | R = retval = fun(*args, **kwargs)
grampsweb_celery | ^^^^^^^^^^^^^^^^^^^^
grampsweb_celery | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/util/celery.py", line 20, in call
grampsweb_celery | return self.run(*args, **kwargs)
grampsweb_celery | ^^^^^^^^^^^^^^^^^^^^^^^^^
grampsweb_celery | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/tasks.py", line 207, in import_file
grampsweb_celery | raise ValueError(f"Failed importing {extension} file")
grampsweb_celery | ValueError: Failed importing gramps file

@DavidMStraub
Copy link
Member

The import function used by Gramps Web is the same as the one used by Gramps Desktop. You can try importing it in Gramps Desktop to check if you get an error there as well.

@maestrx
Copy link
Author

maestrx commented Mar 21, 2025

I can load the export in teh desktop app without any errors

@DavidMStraub
Copy link
Member

I suspect you have a setup issue and your celery container can't access the cache volume. Try creating a file in /app/cache/export and see if it shows up in the celery container under the same path.

@maestrx
Copy link
Author

maestrx commented Mar 21, 2025

Ive setup the infra using the default docker-compose.yml file as instructed in your install manual: https://www.grampsweb.org/install_setup/deployment/. So I dont expect any setup issue.

Import of the .ged file is working fine. When I try to load the media files for it, I get red cross again and this time I see this error:

grampsweb_celery | [2025-03-21 11:01:18,094: INFO/MainProcess] Task gramps_webapi.api.tasks.import_file[6e8f2fa4-ab7a-4f85-966c-d2800a786500] received
grampsweb_celery | [2025-03-21 11:01:20,102: INFO/ForkPoolWorker-1] Task gramps_webapi.api.tasks.import_file[6e8f2fa4-ab7a-4f85-966c-d2800a786500] succeeded in 2.0073128490000727s: None
grampsweb_celery | [2025-03-21 11:01:29,146: INFO/MainProcess] Task gramps_webapi.api.tasks.import_media_archive[785fa339-468b-41a4-a5bc-35cc3a59b7e1] received
grampsweb_celery | [2025-03-21 11:01:29,295: ERROR/ForkPoolWorker-1] Task gramps_webapi.api.tasks.import_media_archive[785fa339-468b-41a4-a5bc-35cc3a59b7e1] raised unexpected: <HTTPException '403: Forbidden'>
grampsweb_celery | Traceback (most recent call last):
grampsweb_celery | File "/usr/local/lib/python3.11/dist-packages/celery/app/trace.py", line 453, in trace_task
grampsweb_celery | R = retval = fun(*args, **kwargs)
grampsweb_celery | ^^^^^^^^^^^^^^^^^^^^
grampsweb_celery | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/util/celery.py", line 20, in call
grampsweb_celery | return self.run(*args, **kwargs)
grampsweb_celery | ^^^^^^^^^^^^^^^^^^^^^^^^^
grampsweb_celery | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/tasks.py", line 343, in import_media_archive
grampsweb_celery | result = importer(progress_cb=progress_callback_count(self))
grampsweb_celery | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grampsweb_celery | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/media_importer.py", line 214, in call
grampsweb_celery | fixed = self._fix_missing_checksums(temp_dir, missing_files)
grampsweb_celery | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grampsweb_celery | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/media_importer.py", line 138, in _fix_missing_checksums
grampsweb_celery | update_object(self.db_handle, new_object, trans)
grampsweb_celery | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/util.py", line 1043, in update_object
grampsweb_celery | abort_with_message(HTTPStatus.FORBIDDEN, "Forbidden: database is read-only")
grampsweb_celery | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/util.py", line 716, in abort_with_message
grampsweb_celery | raise exc
grampsweb_celery | werkzeug.exceptions.HTTPException: 403 Forbidden: Forbidden: database is read-only

I belive that cache folder access is working just fine. This is content of all volumes post clean install, import of the .ged file and attempt of import the media files:

root@databox:/mnt/.ix-apps/docker/volumes# find grampsweb_gramps_*
grampsweb_gramps_cache
grampsweb_gramps_cache/_data
grampsweb_gramps_cache/_data/reports
grampsweb_gramps_cache/_data/export
grampsweb_gramps_cache/_data/export/10a8b9f9-378a-4cb5-8a69-16b10dc3083f.zip
grampsweb_gramps_db
grampsweb_gramps_db/_data
grampsweb_gramps_db/_data/950c8115-e371-47b8-9e63-aa7274f5147d
grampsweb_gramps_db/_data/950c8115-e371-47b8-9e63-aa7274f5147d/name.txt
grampsweb_gramps_db/_data/950c8115-e371-47b8-9e63-aa7274f5147d/database.txt
grampsweb_gramps_db/_data/31c5b113-c2c5-4e82-b080-cb7e12cc2e70
grampsweb_gramps_db/_data/31c5b113-c2c5-4e82-b080-cb7e12cc2e70/sqlite.db
grampsweb_gramps_db/_data/31c5b113-c2c5-4e82-b080-cb7e12cc2e70/undo.db
grampsweb_gramps_db/_data/31c5b113-c2c5-4e82-b080-cb7e12cc2e70/name.txt
grampsweb_gramps_db/_data/31c5b113-c2c5-4e82-b080-cb7e12cc2e70/database.txt
grampsweb_gramps_index
grampsweb_gramps_index/_data
grampsweb_gramps_index/_data/search_index.db
grampsweb_gramps_media
grampsweb_gramps_media/_data
grampsweb_gramps_secret
grampsweb_gramps_secret/_data
grampsweb_gramps_secret/_data/secret
grampsweb_gramps_thumb_cache
grampsweb_gramps_thumb_cache/_data
grampsweb_gramps_thumb_cache/_data/2029240f6d1128be89ddc32729463129
grampsweb_gramps_tmp
grampsweb_gramps_tmp/_data
grampsweb_gramps_tmp/_data/pymp-5rdcqv2j
grampsweb_gramps_tmp/_data/tmpo2u1_5ch
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I20.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I26_1.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I223.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/F70.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/F70_2.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I11_6.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I11_1.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/F1_2.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I232.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I31.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I225.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/F76.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I26.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I0.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I219.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/F72_2.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I0-I18.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I233.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I234.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/F1_3.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I223_1.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I34.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I11-I177.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I33.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/F1.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I11_4.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I11_3.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/F72.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I22.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/F72_1.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I20_1.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/F7.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/F1_1.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I0_1.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I231.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I11-I178-I215.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I177.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I23.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I11.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I220.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I11_2.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/E9.pdf
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I229.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I18.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/F70_1.jpg
grampsweb_gramps_tmp/_data/tmpo2u1_5ch/gramps_media/I11_5.jpg
grampsweb_gramps_users
grampsweb_gramps_users/_data
grampsweb_gramps_users/_data/users.sqlite

@maestrx
Copy link
Author

maestrx commented Mar 21, 2025

I'm also seeing this message frequently:

grampsweb-grampsweb-1 | ERROR:root:Error parsing list of recent DBs from file /root/.gramps/recent-files-gramps.xml: no element found: line 1, column 0.
grampsweb-grampsweb-1 | This might indicate a damage to your files.
grampsweb-grampsweb-1 | If you're sure there is no problem with other files, delete it, and restart Gramps.

@maestrx
Copy link
Author

maestrx commented Mar 21, 2025

whenever I try to load media files that match the media from the .GED export, I get the error:

grampsweb_celery | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/util.py", line 1043, in update_object
grampsweb_celery | abort_with_message(HTTPStatus.FORBIDDEN, "Forbidden: database is read-only")
grampsweb_celery | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/util.py", line 716, in abort_with_message
grampsweb_celery | raise exc
grampsweb_celery | werkzeug.exceptions.HTTPException: 403 Forbidden: Forbidden: database is read-only

Loading media that are not referenced by teh GED file (wrong path) is working fine...

@maestrx
Copy link
Author

maestrx commented Mar 21, 2025

Please help me fix this

@DavidMStraub
Copy link
Member

 grampsweb_celery | werkzeug.exceptions.HTTPException: 403 Forbidden: Forbidden: database is read-only

Ok, this is a bug I believe. We should not have readonly=True here

https://github.com/gramps-project/gramps-web-api/blob/master/gramps_webapi/api/tasks.py#L337

But it's unrelated to your original problem.

@maestrx
Copy link
Author

maestrx commented Mar 21, 2025

I see that this "bug" is not a recent code change. Meaning I have no recent version to downgrade to to skip the impact of this bug.

When do you expect this bug would be fixed ?

Thanks
Vit

@DavidMStraub
Copy link
Member

Please don't ask this question.

Feel free to submit a PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants