Running Doris via Docker Compose

[avidongre]

I have been struggling to get the setup right for a multi node setup via Docker for Doris 2.1.8. It works well the first time but never after a restart. It starts giving an FE waiting for catalog error in a loop. I have fixed IP for all the nodes, mounted volumes for data storage and doris meta and everything works fine on first startup. I have tried enabling metadata recovery and priority networks too via fe.conf but it hasn't helped. Is there a reference working compose file ? Also, is it viable to run Doris on Docker or k8s in production ? Any pointers from the community really appreciated.

[yuanyuan8983]

HI, you mentioned this problem, it may be due to the restart failure caused by the IP change. In this case, you need to use the mode of Meta Recovery, which is generally not recommended, because it will cause metadata confusion.
In general, it is not recommended that Docker deploy Doris cluster, but K8s can be used for deployment. The Doris official website also provides a tutorial about K8s Operator deployment cluster, you can refer to: url

[yuanyuan8983]

https://doris.apache.org/docs/install/deploy-on-kubernetes/install-doris-operator

[avidongre]

Thank you for your response. The IP definitely is still the same. My compose file looks like below -

services:
fe-01:
image: "apache/doris:fe-2.1.8.1"
container_name: "doris-fe-01"
hostname: "fe-01"
environment:
- FE_SERVERS=fe1:172.30.80.2:9010,fe2:172.30.80.3:9010,fe3:172.30.80.4:9010
- FE_ID=1
ports:
- 8031:8030
- 9031:9030
volumes:
- ./data/fe-01/doris-meta:/opt/apache-doris/fe/doris-meta
- ./data/fe-01/log:/opt/apache-doris/fe/log
- ./conf/fe.conf:/opt/apache-doris/fe/conf/fe.conf
networks:
doris_net:
ipv4_address: 172.30.80.2
fe-02:
image: "apache/doris:fe-2.1.8.1"
container_name: "doris-fe-02"
hostname: "fe-02"
environment:
- FE_SERVERS=fe1:172.30.80.2:9010,fe2:172.30.80.3:9010,fe3:172.30.80.4:9010
- FE_ID=2
ports:
- 8032:8030
- 9032:9030
volumes:
- ./data/fe-02/doris-meta:/opt/apache-doris/fe/doris-meta
- ./data/fe-02/log:/opt/apache-doris/fe/log
networks:
doris_net:
ipv4_address: 172.30.80.3
fe-03:
image: "apache/doris:fe-2.1.8.1"
container_name: "doris-fe-03"
hostname: "fe-03"
environment:
- FE_SERVERS=fe1:172.30.80.2:9010,fe2:172.30.80.3:9010,fe3:172.30.80.4:9010
- FE_ID=3
ports:
- 8033:8030
- 9033:9030
volumes:
- ./data/fe-03/doris-meta:/opt/apache-doris/fe/doris-meta
- ./data/fe-03/log:/opt/apache-doris/fe/log
networks:
doris_net:
ipv4_address: 172.30.80.4
be-01:
image: "apache/doris:be-2.1.8.1"
container_name: "doris-be-01"
hostname: "be-01"
depends_on:
- fe-01
- fe-02
- fe-03
environment:
- FE_SERVERS=fe1:172.30.80.2:9010,fe2:172.30.80.3:9010,fe3:172.30.80.4:9010
- BE_ADDR=172.30.80.5:9050
ports:
- 8041:8040
volumes:
- ./data/be-01/storage:/opt/apache-doris/be/storage
- ./data/be-01/script:/docker-entrypoint-initdb.d
- ./data/be-01/log:/opt/apache-doris/be/log
networks:
doris_net:
ipv4_address: 172.30.80.5
be-02:
image: "apache/doris:be-2.1.8.1"
container_name: "doris-be-02"
hostname: "be-02"
depends_on:
- fe-01
- fe-02
- fe-03
environment:
- FE_SERVERS=fe1:172.30.80.2:9010,fe2:172.30.80.3:9010,fe3:172.30.80.4:9010
- BE_ADDR=172.30.80.6:9050
ports:
- 8042:8040
volumes:
- ./data/be-02/storage:/opt/apache-doris/be/storage
- ./data/be-02/script:/docker-entrypoint-initdb.d
- ./data/be-02/log:/opt/apache-doris/be/log
networks:
doris_net:
ipv4_address: 172.30.80.6
be-03:
image: "apache/doris:be-2.1.8.1"
container_name: "doris-be-03"
hostname: "be-03"
depends_on:
- fe-01
- fe-02
- fe-03
environment:
- FE_SERVERS=fe1:172.30.80.2:9010,fe2:172.30.80.3:9010,fe3:172.30.80.4:9010
- BE_ADDR=172.30.80.7:9050
ports:
- 8043:8040
volumes:
- ./data/be-03/storage:/opt/apache-doris/be/storage
- ./data/be-03/script:/docker-entrypoint-initdb.d
- ./data/be-03/log:/opt/apache-doris/be/log
networks:
doris_net:
ipv4_address: 172.30.80.7
networks:
doris_net:
ipam:
config:
- subnet: 172.30.80.0/24

I have tried the k8s operator as well - with the helm chart - the BE node isn't able to reach the FE node due to incorrect hostname for FE service. Its not using the right k8s host prefix like <POD_NAME>.<HEADLESS_SERVICE_NAME>..svc.cluster.local. Is there a way to set this via environment variables ?

[avidongre]

I was finally able to get the helm chart going by patching the service name AFTER the helm chart was deployed

kubectl -n doris set env statefulset/doriscluster-be ENV_FE_ADDR="doriscluster-fe-internal.doris.svc.cluster.local"

Any values set in values.yaml would just get overridden during the helm install. Is there a more elegant way of handling this ?