Can't make calendar events.

[koen860]

Impacted versions

• Modoboa: 1.15.0
• installer used: Yes
• Webserver: Nginx

Steps to reproduce

I'm running a fresh install on CentOS 7 with external Mysql DB.

Current behavior

I installed a fresh copy of Modoboa and noticed that the calendar function isn't working correctly. After some investigation i noticed that radicale is not started. When i start Radicale i saw this message:
[2020-06-08 19:33:57 +0200] [28669] [CRITICAL] Invalid configuration: Invalid option 'debug' in section 'logging' in config file '/etc/radicale/config'

I commented out the Debug line in /etc/radicale/config and received the following error. And this is the one where i'm stuck on:

[2020-06-08 19:37:17 +0200] [29280] [CRITICAL] An exception occurred during server startup: Failed to load storage module 'radicale_storage_by_index': cannot import name 'Collection'

Expected behavior

I hope someone can help me to start radicale. So that i can use the calendar function.

PS. I also noticed that the web interface is taking very long to create a calendar (but does eventually creates one).

Update:

During a second install on a clean, new centos 7 vm i noticed these error messages:

ERROR: Can't open/parse the config file /etc/freshclam.conf
...
Job for clamd@amavisd.service failed because the control process exited with error code. See "systemctl status clamd@amavisd.service" and "journalctl -xe" for details.

ERROR: Command errored out with exit status 1:
command: /srv/modoboa/env/bin/python3 /srv/modoboa/env/lib64/python3.6/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpwrgjsy6s
cwd: /tmp/pip-install-7ukr84j4/django-braces
Complete output (16 lines):
Traceback (most recent call last):
File "/srv/modoboa/env/lib64/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in
main()
File "/srv/modoboa/env/lib64/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/srv/modoboa/env/lib64/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 114, in get_requires_for_build_wheel
return hook(config_settings)
File "/tmp/pip-build-env-k5g46gwu/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 148, in get_requires_for_build_wheel
config_settings, requirements=['wheel'])
File "/tmp/pip-build-env-k5g46gwu/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 128, in _get_build_requires
self.run_setup()
File "/tmp/pip-build-env-k5g46gwu/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 143, in run_setup
exec(compile(code, file, 'exec'), locals())
File "setup.py", line 3, in
import braces
ModuleNotFoundError: No module named 'braces'

ERROR: Command errored out with exit status 1: /srv/modoboa/env/bin/python3 /srv/modoboa/env/lib64/python3.6/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpwrgjsy6s Check the logs for full command output.

...

ERROR: modoboa-contacts 0.8.1 has requirement caldav==0.6.2, but you'll have caldav 0.5.0 which is incompatible.
ERROR: modoboa-radicale 1.4.3 has requirement caldav==0.6.2, but you'll have caldav 0.5.0 which is incompatible.

And the same issues are present is this install as well. Upgrading to caldav 0.6.2 also doesn't help.

Video/Screenshot link (optional)

[tonioo]

@koen860 A new major version of Radicale has been released a few weeks ago and it breaks compatibility with old versions. The installer was not forcing the version to use so it deploys the latest one. I've fixed the installer for now but you must downgrade the Radicale version to 2.1.12. I suggest you update your local copy of the installer using git, and then run it again.

[koen860]

Hello,

Thank you for your fast response! But the installer prints that radicale is already in radicale-storage-by-index so it wont install the correct version:

Requirement already satisfied: radicale in ./env/lib/python3.6/site-packages (from radicale-storage-by-index) (3.0.3)

And when I manually install radicale 2 I get this error:
[2020-06-09 10:55:02 +0200] [14400] [CRITICAL] An exception occurred during server startup: Failed to load auth module 'radicale_dovecot_auth': No module named 'radicale_dovecot_auth'

[tonioo]

@koen860 The storage-by-index plugin looks deprecated too. I suggest you uninstall it manually before you run the installer again.

[koen860]

It looks like it didn't install anything at the moment:

Installing radicale
Package supervisor-3.4.0-1.el7.noarch already installed and latest version
Creating mailbox file: File exists
Package python3-3.6.8-13.el7.x86_64 already installed and latest version
Collecting pip
Downloading https://files.pythonhosted.org/packages/43/84/23ed6a1796480a6f1a2d38f2802901d078266bda38388954d01d3f2e821d/pip-20.1.1-py2.py3-none-any.whl (1.5MB)
100% |████████████████████████████████| 1.5MB 396kB/s
Collecting setuptools
Downloading https://files.pythonhosted.org/packages/95/95/f657b6e17f00c3f35b5f68b10e46c3a43af353d8856bd57bfcfb1dbb3e92/setuptools-47.1.1-py3-none-any.whl (583kB)
100% |████████████████████████████████| 583kB 1.8MB/s
Installing collected packages: pip, setuptools
Found existing installation: pip 9.0.3
Uninstalling pip-9.0.3:
Successfully uninstalled pip-9.0.3
Found existing installation: setuptools 39.2.0
Uninstalling setuptools-39.2.0:
Successfully uninstalled setuptools-39.2.0
Successfully installed pip-20.1.1 setuptools-47.1.1
/bin/sh: 3: No such file or directory
Redirecting to /bin/systemctl stop supervisord.service
Redirecting to /bin/systemctl start supervisord.service

Radicale Directory:
radicale@.nl:/env/bin# ls
activate activate.csh activate.fish easy_install easy_install-3.6 pip pip3 pip3.6 python python3
radicale@.nl:/env/bin# pwd
/srv/radicale/env/bin

radicale@.sddc01.nl:~/env# ./bin/pip list
Package Version

---

pip 20.1.1
setuptools 47.1.1

When i changed line 34 of the radical install script from:
packages = ["Radicale<3", "radicale-dovecot-auth", "pytz"]
to:
packages = ["Radicale==2.1.10", "radicale-dovecot-auth", "pytz"]

It started the install and Radicale is running without any errors but it still doesnt create calendar event or calendars.

[tonioo]

@koen860 Can you check logs?

[koen860]

From what services do you need logs?

[koen860]

This is de uwsgi log:

server.sdhd.email [pid: 22011|app: 1|req: 2/3] 172.24.2.30 () {64 vars in 1101 bytes} [Tue Jun 9 14:18:05 2020] GET /jsi18n/ => generated 3223 bytes in 86 msecs (HTTP/1.0 200) 5 headers in 171 bytes (1 switches on core 0)
server.sdhd.email [pid: 22012|app: 1|req: 2/4] 172.24.2.30 () {68 vars in 1291 bytes} [Tue Jun 9 14:18:07 2020] GET /calendars/ => generated 6118 bytes in 700 msecs (HTTP/1.0 200) 5 headers in 163 bytes (1 switches on core 0)
server.sdhd.email [pid: 22012|app: 1|req: 3/5] 172.24.2.30 () {64 vars in 1101 bytes} [Tue Jun 9 14:18:08 2020] GET /jsi18n/ => generated 3223 bytes in 124 msecs (HTTP/1.0 200) 5 headers in 171 bytes (1 switches on core 0)
server.sdhd.email [pid: 22012|app: 1|req: 4/6] 172.24.2.30 () {68 vars in 1284 bytes} [Tue Jun 9 14:18:08 2020] GET /api/v1/shared-calendars/ => generated 569 bytes in 188 msecs (HTTP/1.0 200) 6 headers in 195 bytes (1 switches on core 0)
server.sdhd.email [pid: 22011|app: 1|req: 3/7] 172.24.2.30 () {68 vars in 1280 bytes} [Tue Jun 9 14:18:08 2020] GET /api/v1/user-calendars/ => generated 1879 bytes in 233 msecs (HTTP/1.0 200) 6 headers in 196 bytes (1 switches on core 0)
Tue Jun 9 14:19:09 2020 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /api/v1/shared-calendars/6/events/?start=2020-06-07T00%3A00%3A00%2B02%3A00&end=2020-06-14T00%3A00%3A00%2B02%3A00&=1591705089610 (ip 172.24.2.30) !!!
Tue Jun 9 14:19:09 2020 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /api/v1/shared-calendars/6/events/?start=2020-06-07T00%3A00%3A00%2B02%3A00&end=2020-06-14T00%3A00%3A00%2B02%3A00&=1591705089610 (172.24.2.30)
OSError: write error
server.sdhd.email [pid: 22011|app: 1|req: 4/8] 172.24.2.30 () {66 vars in 1418 bytes} [Tue Jun 9 14:18:09 2020] GET /api/v1/shared-calendars/6/events/?start=2020-06-07T00%3A00%3A00%2B02%3A00&end=2020-06-14T00%3A00%3A00%2B02%3A00&=1591705089610 => generated 0 bytes in 60567 msecs (HTTP/1.0 500) 5 headers in 0 bytes (0 switches on core 0)
Tue Jun 9 14:19:09 2020 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /api/v1/shared-calendars/3/events/?start=2020-06-07T00%3A00%3A00%2B02%3A00&end=2020-06-14T00%3A00%3A00%2B02%3A00&=1591705089609 (ip 172.24.2.30) !!!
Tue Jun 9 14:19:09 2020 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /api/v1/shared-calendars/3/events/?start=2020-06-07T00%3A00%3A00%2B02%3A00&end=2020-06-14T00%3A00%3A00%2B02%3A00&=1591705089609 (172.24.2.30)
OSError: write error
server.sdhd.email [pid: 22012|app: 1|req: 5/9] 172.24.2.30 () {66 vars in 1418 bytes} [Tue Jun 9 14:18:09 2020] GET /api/v1/shared-calendars/3/events/?start=2020-06-07T00%3A00%3A00%2B02%3A00&end=2020-06-14T00%3A00%3A00%2B02%3A00&=1591705089609 => generated 0 bytes in 60592 msecs (HTTP/1.0 500) 5 headers in 0 bytes (0 switches on core 0)

[koen860]

And this is the Radicale Log:

[7fcc20e1b740] INFO: Starting Radicale
[7fcc20e1b740] INFO: Authentication type is 'radicale_dovecot_auth'
[7fcc20e1b740] INFO: Storage type is 'multifilesystem'
[7fcc20e1b740] INFO: Rights type is 'from_file'
[7fcc20e1b740] INFO: Web type is 'none'
[7fcc20e1b740] INFO: Listening to 'localhost.localdomain' on port 5232
[7fcc20e1b740] INFO: Radicale server ready

Also, the django debug log:

Traceback:

File "/srv/modoboa/env/lib64/python3.6/site-packages/django/core/handlers/exception.py" in inner
34. response = get_response(request)

File "/srv/modoboa/env/lib64/python3.6/site-packages/django/core/handlers/base.py" in _get_response
145. response = self.process_exception_by_middleware(e, request)

File "/srv/modoboa/env/lib64/python3.6/site-packages/django/core/handlers/base.py" in _get_response
143. response = response.render()

File "/srv/modoboa/env/lib64/python3.6/site-packages/django/template/response.py" in render
106. self.content = self.rendered_content

File "/srv/modoboa/env/lib64/python3.6/site-packages/rest_framework/response.py" in rendered_content
70. ret = renderer.render(self.data, accepted_media_type, context)

File "/srv/modoboa/env/lib64/python3.6/site-packages/rest_framework/renderers.py" in render
724. context = self.get_context(data, accepted_media_type, renderer_context)

File "/srv/modoboa/env/lib64/python3.6/site-packages/rest_framework/renderers.py" in get_context
655. raw_data_post_form = self.get_raw_data_form(data, view, 'POST', request)

File "/srv/modoboa/env/lib64/python3.6/site-packages/rest_framework/renderers.py" in get_raw_data_form
563. data = serializer.data.copy()

File "/srv/modoboa/env/lib64/python3.6/site-packages/rest_framework/serializers.py" in data
562. ret = super().data

File "/srv/modoboa/env/lib64/python3.6/site-packages/rest_framework/serializers.py" in data
256. raise AssertionError(msg)

Exception Type: AssertionError at /api/v1/user-calendars/18/events/
Exception Value: When a serializer is passed a data keyword argument you must call .is_valid() before attempting to access the serialized .data representation.
You should either call .is_valid() first, or access .initial_data instead.

[koen860]

I have fixed the issue by just completely reinstalling my CentOS and begin from scratch. Then i noticed that my Apple calendar wont connect to it while some other apps do. What i did to fix it was adding this config to the NGINX configuration. I discovered this by looking at the acces log while trying to logon.

location /principals/users/ {
    proxy_pass http://localhost:5232/; # The / is important!
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Script-Name /radicale;
    proxy_pass_header Authorization;
}

[tonioo]

@koen860 Do you know if it is a standard url?

[koen860]

The standard URL for discovery is /.well-known/ but that was in my case already in use for let's encrypt so I saw that the client was trying this url (/principals/users/). So I added this to the NGINX config because it works for me and this is a personal VPS. It's probably nicer to comply with the RFC5785 and redirect the caldav client via the .well-known url as described here: https://sabre.io/dav/service-discovery/