Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make it easier to run in Kubernetes #203

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 19 additions & 2 deletions core/files/entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -68,5 +68,22 @@ export PHP_SESSION_COOKIE_SAMESITE=${PHP_SESSION_COOKIE_SAMESITE:-Lax}
export NGINX_X_FORWARDED_FOR=${NGINX_X_FORWARDED_FOR:-false}
export NGINX_SET_REAL_IP_FROM=${NGINX_SET_REAL_IP_FROM}

# start supervisord using the main configuration file so we have a socket interface
/usr/bin/supervisord -c /etc/supervisor/supervisord.conf
if [ -n "$KUBERNETES_SERVICE_HOST" ]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd change this to if [ -n "$KUBERNETES_SERVICE_HOST" ] && [ -n "$CONTAINER_NAME" ]; then to ensure we just do the default thing if CONTAINER_NAME is missing. This ensures existing setups aren't immediately broken by updating.

case "$CONTAINER_NAME" in
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really a shame the container name isn't exposed to the container environment by default.

nginx*)
exec /entrypoint_k8s_nginx.sh
;;
php*)
# Not ideal, but let supervisord manage the workers still
mv /etc/supervisor/conf.d/10-supervisor.conf{.k8s,}
/usr/bin/supervisord -c /etc/supervisor/supervisord.conf &
exec /entrypoint_k8s_fpm.sh
;;
cron*)
exec /entrypoint_cron.sh
;;
esac
else
# start supervisord using the main configuration file so we have a socket interface
/usr/bin/supervisord -c /etc/supervisor/supervisord.conf
fi
5 changes: 5 additions & 0 deletions core/files/entrypoint_cron.sh
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,11 @@ if [[ ! -p /tmp/cronlog ]]; then
mkfifo -m 777 /tmp/cronlog
fi

if [ -n "$KUBERNETES_SERVICE_HOST" ]; then
tail -f /tmp/cronlog &
exec cron -l -f
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're missing some setup for the default cron jobs to be able to run here (www-data /var/www/MISP/app/Console/cake Admin updateGalaxies and friends). Not sure it's feasible to run these in their own container.

Maybe the best solution is to rewrite them all to run API calls / pymisp against localhost with supplied admin_key?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, a better solution would befor us to supply an example set of CronJob manifests to deploy alongside MISP.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, ok, thanks. Admittedly I didn't look too far into the cron jobs.

fi

# Build another fifo for the cron pipe
if [[ ! -p /tmp/cronpipe ]]; then
mkfifo /tmp/cronpipe
Expand Down
9 changes: 9 additions & 0 deletions core/files/entrypoint_fpm.sh
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ change_php_vars() {
sed -i "s|.*session.save_path = .*|session.save_path = '$(echo $REDIS_HOST | grep -E '^\w+://' || echo tcp://$REDIS_HOST):$REDIS_PORT?auth=${ESCAPED}'|" "$FILE"
sed -i "s/session.sid_length = .*/session.sid_length = 64/" "$FILE"
sed -i "s/session.use_strict_mode = .*/session.use_strict_mode = 1/" "$FILE"
sed -i "s|session.cookie_domain = .*|session.cookie_domain = ${BASE_URL}|" "$FILE"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

done

for FILE in /etc/php/*/fpm/pool.d/www.conf
Expand Down Expand Up @@ -57,9 +58,17 @@ change_php_vars() {
echo "Configure PHP | Disabling 'pm.status_listen'"
sed -i -E "s/^pm.status_listen =/;pm.status_listen =/" "$FILE"
fi
if [[ -n "$PHP_FPM_SOCK_FILE" ]]; then
echo "Configure PHP | Setting 'listen' to ${PHP_FPM_SOCK_FILE}"
sed -i "/^listen =/s@=.*@= ${PHP_FPM_SOCK_FILE}@" "$FILE"
fi
done
}

if [ -n "${BASH_SOURCE[0]}" ]; then
return
fi

echo "Configure PHP | Change PHP values ..." && change_php_vars

echo "Configure PHP | Starting PHP FPM"
Expand Down
26 changes: 26 additions & 0 deletions core/files/entrypoint_k8s_fpm.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
#!/bin/bash -e

source /entrypoint_nginx.sh
source /entrypoint_fpm.sh

# Initialize MySQL
echo "INIT | Initialize MySQL ..." && init_mysql

# Initialize MISP
echo "INIT | Initialize MISP files and configurations ..." && init_misp_data_files
echo "INIT | Update MISP app/files directory ..." && update_misp_data_files
echo "INIT | Enforce MISP permissions ..." && enforce_misp_data_permissions

# Run configure MISP script
echo "INIT | Configure MISP installation ..."
/configure_misp.sh

if [[ -x /custom/files/customize_misp.sh ]]; then
echo "INIT | Customize MISP installation ..."
/custom/files/customize_misp.sh
fi

echo "Configure PHP | Change PHP values ..." && change_php_vars

echo "Configure PHP | Starting PHP FPM"
exec /usr/sbin/php-fpm8.2 -R -F
11 changes: 11 additions & 0 deletions core/files/entrypoint_k8s_nginx.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/bin/bash -e

source /entrypoint_nginx.sh

# Initialize nginx
echo "INIT | Initialize NGINX ..." && init_nginx
echo "INIT | Flip NGINX live ..." && flip_nginx true true

# launch nginx as current shell process in container
exec nginx -g 'daemon off;'

11 changes: 10 additions & 1 deletion core/files/entrypoint_nginx.sh
Original file line number Diff line number Diff line change
Expand Up @@ -210,13 +210,19 @@ flip_nginx() {
echo "... nginx docroot set to ${NGINX_DOC_ROOT}"
sed -i "s|root.*var/www.*|root ${NGINX_DOC_ROOT};|" /etc/nginx/includes/misp

if [[ "$reload" = "true" ]]; then
if [[ "$reload" = "true" ]] && [[ -z "$KUBERNETES_SERVICE_HOST" ]]; then
echo "... nginx reloaded"
nginx -s reload
fi
}

init_nginx() {
# Optional location of PHP-FPM sock file
if [[ -n "$PHP_FPM_SOCK_FILE" ]]; then
echo "... setting 'fastcgi_pass' to unix:${PHP_FPM_SOCK_FILE}"
sed -i "s@fastcgi_pass .*;@fastcgi_pass unix:${PHP_FPM_SOCK_FILE};@" /etc/nginx/includes/misp
fi

# Adjust timeouts
echo "... adjusting 'fastcgi_read_timeout' to ${FASTCGI_READ_TIMEOUT}"
sed -i "s/fastcgi_read_timeout .*;/fastcgi_read_timeout ${FASTCGI_READ_TIMEOUT};/" /etc/nginx/includes/misp
Expand Down Expand Up @@ -352,6 +358,9 @@ init_nginx() {
flip_nginx false false
}

if [ -n "${BASH_SOURCE[0]}" ]; then
return
fi

# Initialize MySQL
echo "INIT | Initialize MySQL ..." && init_mysql
Expand Down
12 changes: 12 additions & 0 deletions core/files/etc/supervisor/conf.d/10-supervisor.conf.k8s
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
[supervisord]
nodaemon=true
user=root
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0

[inet_http_server]
port=127.0.0.1:9001
username=supervisor
password=supervisor
Loading