Skip to content

supports JSON logging for gunicorn process within Docker#8779

Merged
akshay-joshi merged 1 commit into
pgadmin-org:masterfrom
AllenSH12:docker-json-logging
Jun 3, 2025
Merged

supports JSON logging for gunicorn process within Docker#8779
akshay-joshi merged 1 commit into
pgadmin-org:masterfrom
AllenSH12:docker-json-logging

Conversation

@AllenSH12

@AllenSH12 AllenSH12 commented May 21, 2025

Copy link
Copy Markdown
Contributor

For #8665

This attempts to bridge the gap between the application and gunicorn config to ensure both are logging in a consistent way.

Before:

$ grep CONSOLE_LOG_LEVEL web/config_local.py
CONSOLE_LOG_LEVEL = logging.INFO

$ grep JSON_LOGGER web/config.py 
JSON_LOGGER = True

$ ./venv/bin/gunicorn  -c ./pkg/docker/gunicorn_config.py --chdir web pgAdmin4:app                                
[2025-05-21 13:20:52 -0400] [36038] [INFO] Starting gunicorn 23.0.0
[2025-05-21 13:20:52 -0400] [36038] [INFO] Listening at: http://127.0.0.1:8000 (36038)
[2025-05-21 13:20:52 -0400] [36038] [INFO] Using worker: sync
[2025-05-21 13:20:52 -0400] [36039] [INFO] Booting worker with pid: 36039
{"time": "2025-05-21 13:20:53,405", "message": "########################################################", "level": "INFO"}
{"time": "2025-05-21 13:20:53,405", "message": "Starting pgAdmin 4 v9.3...", "level": "INFO"}
{"time": "2025-05-21 13:20:53,405", "message": "########################################################", "level": "INFO"}
...

After:

$ grep CONSOLE_LOG_LEVEL web/config_local.py
CONSOLE_LOG_LEVEL = logging.INFO

$ grep JSON_LOGGER web/config.py 
JSON_LOGGER = True

$ ./venv/bin/gunicorn -c ./pkg/docker/gunicorn_config.py --chdir web pgAdmin4:app
[2025-05-21 13:22:30 -0400] [36080] [INFO] Starting gunicorn 23.0.0
{"time": "2025-05-21 13:22:31,529", "message": "Listening at: http://127.0.0.1:8000 (36080)", "level": "INFO"}
{"time": "2025-05-21 13:22:31,529", "message": "Using worker: sync", "level": "INFO"}
{"time": "2025-05-21 13:22:31,532", "message": "Booting worker with pid: 36081", "level": "INFO"}
{"time": "2025-05-21 13:22:31,609", "message": "########################################################", "level": "INFO"}
{"time": "2025-05-21 13:22:31,609", "message": "Starting pgAdmin 4 v9.3...", "level": "INFO"}
{"time": "2025-05-21 13:22:31,610", "message": "########################################################", "level": "INFO"}

And via docker with a locally built image:

$ docker run -d --name=pgadmin4 --rm \
    -e "PGADMIN_DEFAULT_EMAIL=a@b.co" \
    -e "PGADMIN_DEFAULT_PASSWORD=admin" \
    -e "PGADMIN_CONFIG_JSON_LOGGER=True" \
    -e "PGADMIN_CONFIG_CONSOLE_LOG_LEVEL=10" \
    -p 8080:80 \
    allensh12/pgadmin4

$ docker logs -f pgadmin4
...
[2025-05-21 17:24:38 +0000] [1] [INFO] Starting gunicorn 23.0.0
{"time": "2025-05-21 17:24:40,663", "message": "Listening at: http://[::]:80 (1)", "level": "INFO"}
{"time": "2025-05-21 17:24:40,663", "message": "Using worker: gthread", "level": "INFO"}
{"time": "2025-05-21 17:24:40,667", "message": "Booting worker with pid: 121", "level": "INFO"}
{"time": "2025-05-21 17:24:40,876", "message": "########################################################", "level": "INFO"}
{"time": "2025-05-21 17:24:40,876", "message": "Starting pgAdmin 4 v9.3...", "level": "INFO"}
{"time": "2025-05-21 17:24:40,876", "message": "########################################################", "level": "INFO"}

A few outstanding things:

  1. The first line from gunicorn ("Starting gunicorn...") does not respect the format, given that this is much less common than access log perhaps that is okay to leave, but I will investigate further.
  2. A few other places where printing happens outside of the logger, have marked these with "issues/8665" in the code, can see example below
  3. No control over postfix logs, the only one I've seen so far is "postfix/postlog: starting the Postfix mail system" but I have not exercised it more than that. IMO an issue that is worth addressing separately.
$ docker logs -f pgadmin4
email config is {'CHECK_EMAIL_DELIVERABILITY': False, 'ALLOW_SPECIAL_EMAIL_DOMAINS': [], 'GLOBALLY_DELIVERABLE': True}
NOTE: Configuring authentication for SERVER mode.

pgAdmin 4 - Application Initialisation
======================================

postfix/postlog: starting the Postfix mail system
[2025-05-21 17:38:41 +0000] [1] [INFO] Starting gunicorn 23.0.0
{"time": "2025-05-21 17:38:43,326", "message": "Listening at: http://[::]:80 (1)", "level": "INFO"}
{"time": "2025-05-21 17:38:43,327", "message": "Using worker: gthread", "level": "INFO"}
{"time": "2025-05-21 17:38:43,330", "message": "Booting worker with pid: 119", "level": "INFO"}

Curious to hear what y'all think!

@yogeshmahajan-1903 yogeshmahajan-1903 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess you have removed changes in the entrypoint.sh while cleaning up the code.

@AllenSH12

Copy link
Copy Markdown
Contributor Author

Yes everything is happening in gunicorn_config.py now, makes it easier to share the app config that way but I think there is a better solution still, need to study the Python logging module more

@AllenSH12 AllenSH12 force-pushed the docker-json-logging branch 2 times, most recently from 3ca994e to 238aea1 Compare May 27, 2025 22:32
@AllenSH12

Copy link
Copy Markdown
Contributor Author

@yogeshmahajan-1903 have this working as desired now, here are some example logs. Can see both gunicorn and pgadmin are logging in JSON, and I created #8801 to follow up with other components, although I have no immediate plans to work on the issue myself.

$ docker run -d --name=pgadmin4 --rm \
    -e "PGADMIN_DEFAULT_EMAIL=a@b.co" \
    -e "PGADMIN_DEFAULT_PASSWORD=admin" \
    -e "PGADMIN_CONFIG_JSON_LOGGER=True" \
    -e "PGADMIN_CONFIG_CONSOLE_LOG_LEVEL=20" \
    -p 8080:80 \
    allensh12/pgadmin4

email config is {'CHECK_EMAIL_DELIVERABILITY': False, 'ALLOW_SPECIAL_EMAIL_DOMAINS': [], 'GLOBALLY_DELIVERABLE': True}
{"time": "2025-05-27 22:20:55,923", "message": "########################################################", "level": "INFO"}
{"time": "2025-05-27 22:20:55,923", "message": "Starting pgAdmin 4 v9.3...", "level": "INFO"}
{"time": "2025-05-27 22:20:55,923", "message": "########################################################", "level": "INFO"}
{"time": "2025-05-27 22:20:56,964", "message": "Registering blueprint module: <AboutModule 'about'>", "level": "INFO"}
{"time": "2025-05-27 22:20:56,964", "message": "Registering blueprint module: <AuthenticateModule 'authenticate'>", "level": "INFO"}
{"time": "2025-05-27 22:20:56,965", "message": "Registering blueprint module: <BrowserModule 'browser'>", "level": "INFO"}
{"time": "2025-05-27 22:20:57,626", "message": "Registering blueprint module: <DashboardModule 'dashboard'>", "level": "INFO"}
{"time": "2025-05-27 22:20:57,634", "message": "Registering blueprint module: <HelpModule 'help'>", "level": "INFO"}
{"time": "2025-05-27 22:20:57,634", "message": "Registering blueprint module: <MiscModule 'misc'>", "level": "INFO"}
{"time": "2025-05-27 22:20:58,340", "message": "Registering blueprint module: <PreferencesModule 'preferences'>", "level": "INFO"}
{"time": "2025-05-27 22:20:58,342", "message": "Registering blueprint module: <PgAdminModule 'redirects'>", "level": "INFO"}
{"time": "2025-05-27 22:20:58,342", "message": "Registering blueprint module: <SettingsModule 'settings'>", "level": "INFO"}
{"time": "2025-05-27 22:20:58,344", "message": "Registering blueprint module: <ToolsModule 'tools'>", "level": "INFO"}
NOTE: Configuring authentication for SERVER mode.

pgAdmin 4 - Application Initialisation
======================================

postfix/postlog: starting the Postfix mail system
guni config True 20 OrderedDict({'time': 'asctime', 'message': 'message', 'level': 'levelname'})
{"time": "2025-05-27 22:21:03,696", "message": "Starting gunicorn 23.0.0", "level": "INFO"}
{"time": "2025-05-27 22:21:03,697", "message": "Listening at: http://[::]:80 (1)", "level": "INFO"}
{"time": "2025-05-27 22:21:03,697", "message": "Using worker: gthread", "level": "INFO"}
{"time": "2025-05-27 22:21:03,700", "message": "Booting worker with pid: 121", "level": "INFO"}
{"time": "2025-05-27 22:21:03,907", "message": "########################################################", "level": "INFO"}
{"time": "2025-05-27 22:21:03,908", "message": "Starting pgAdmin 4 v9.3...", "level": "INFO"}
{"time": "2025-05-27 22:21:03,908", "message": "########################################################", "level": "INFO"}
{"time": "2025-05-27 22:21:04,570", "message": "Registering blueprint module: <AboutModule 'about'>", "level": "INFO"}
{"time": "2025-05-27 22:21:04,570", "message": "Registering blueprint module: <AuthenticateModule 'authenticate'>", "level": "INFO"}
{"time": "2025-05-27 22:21:04,571", "message": "Registering blueprint module: <BrowserModule 'browser'>", "level": "INFO"}
{"time": "2025-05-27 22:21:05,297", "message": "Registering blueprint module: <DashboardModule 'dashboard'>", "level": "INFO"}
{"time": "2025-05-27 22:21:05,305", "message": "Registering blueprint module: <HelpModule 'help'>", "level": "INFO"}
{"time": "2025-05-27 22:21:05,306", "message": "Registering blueprint module: <MiscModule 'misc'>", "level": "INFO"}
{"time": "2025-05-27 22:21:05,992", "message": "Registering blueprint module: <PreferencesModule 'preferences'>", "level": "INFO"}
{"time": "2025-05-27 22:21:05,993", "message": "Registering blueprint module: <PgAdminModule 'redirects'>", "level": "INFO"}
{"time": "2025-05-27 22:21:05,994", "message": "Registering blueprint module: <SettingsModule 'settings'>", "level": "INFO"}
{"time": "2025-05-27 22:21:05,995", "message": "Registering blueprint module: <ToolsModule 'tools'>", "level": "INFO"}
{"time": "2025-05-27 22:21:18,001", "message": "::ffff:172.17.0.1 - - [27/May/2025:22:21:18 +0000] \"GET / HTTP/1.1\" 302 213 \"-\" \"curl/8.7.1\"", "level": "INFO"}

@AllenSH12 AllenSH12 force-pushed the docker-json-logging branch from 238aea1 to 88648eb Compare May 27, 2025 22:38
@AllenSH12 AllenSH12 force-pushed the docker-json-logging branch from 88648eb to 0700303 Compare May 27, 2025 22:41
@akshay-joshi akshay-joshi merged commit 0a2db9b into pgadmin-org:master Jun 3, 2025
32 checks passed
@AllenSH12

Copy link
Copy Markdown
Contributor Author

Thank you @akshay-joshi!

@AllenSH12 AllenSH12 deleted the docker-json-logging branch July 14, 2025 21:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants