Skip to content

Commit bb2dcc7

Browse files
Haaroleangitbook-bot
Haarolean
authored and
gitbook-bot
committed
GitBook: [#4] No subject
1 parent b582e9f commit bb2dcc7

26 files changed

+1535
-0
lines changed

‎README.md

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# About
2+
3+
## **About Kafka-UI**
4+
5+
**Versatile, fast and lightweight web UI for managing Apache Kafka® clusters. Built by developers, for developers.**
6+
7+
****
8+
9+
**UI for Apache Kafka is a free, open-source web UI to monitor and manage Apache Kafka clusters.**
10+
11+
UI for Apache Kafka is a simple tool that makes your data flows observable, helps find and troubleshoot issues faster and deliver optimal performance. Its lightweight dashboard makes it easy to track key metrics of your Kafka clusters - Brokers, Topics, Partitions, Production, and Consumption.

‎SUMMARY.md

+41
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# Table of contents
2+
3+
## 🎓 Overview
4+
5+
* [About](README.md)
6+
* [Features](overview/features.md)
7+
* [Getting started](overview/getting-started.md)
8+
9+
## 🛣 Project
10+
11+
* [Code of Conduct](project/code-of-conduct.md)
12+
* [Roadmap](project/roadmap.md)
13+
14+
## 🛠 Development
15+
16+
* [Contributing](development/contributing.md)
17+
* [Building](development/building/README.md)
18+
* [Prerequisites](development/building/prerequisites.md)
19+
* [WIP: Setting up git](development/building/wip-setting-up-git.md)
20+
* [With Docker](development/building/with-docker.md)
21+
* [Without Docker](development/building/without-docker.md)
22+
* [WIP: Testing](development/building/wip-testing.md)
23+
24+
## 👷♂ Configuration
25+
26+
* [Configuration](configuration/configuration.md)
27+
* [SSL](configuration/ssl.md)
28+
* [Authentication](configuration/authentication/README.md)
29+
* [OAuth2](configuration/authentication/oauth2.md)
30+
* [AWS IAM](configuration/authentication/aws-iam.md)
31+
* [SSO Guide](configuration/authentication/sso-guide.md)
32+
* [SASL\_SCRAM](configuration/authentication/sasl\_scram.md)
33+
* [RBAC (Role based access control)](configuration/rbac-role-based-access-control.md)
34+
* [Data masking](configuration/data-masking.md)
35+
* [Serialization / SerDe](configuration/serialization-serde.md)
36+
* [Protobuf setup](configuration/protobuf-setup.md)
37+
38+
## ❓ FAQ
39+
40+
* [Common problems](faq/common-problems.md)
41+
* [FAQ](faq/faq.md)

‎configuration/authentication/README.md

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
# Authentication
2+

‎configuration/authentication/aws-iam.md

+45
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
---
2+
description: How to configure AWS IAM Authentication
3+
---
4+
5+
# AWS IAM
6+
7+
UI for Apache Kafka comes with built-in [aws-msk-iam-auth](https://github.com/aws/aws-msk-iam-auth) library.
8+
9+
You could pass sasl configs in properties section for each cluster.
10+
11+
More details could be found here: [aws-msk-iam-auth](https://github.com/aws/aws-msk-iam-auth)
12+
13+
### Examples:
14+
15+
Please replace
16+
17+
* \<KAFKA\_URL> with broker list
18+
* \<PROFILE\_NAME> with your aws profile
19+
20+
#### Running From Docker Image
21+
22+
```
23+
docker run -p 8080:8080 \
24+
-e KAFKA_CLUSTERS_0_NAME=local \
25+
-e KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=<KAFKA_URL> \
26+
-e KAFKA_CLUSTERS_0_PROPERTIES_SECURITY_PROTOCOL=SASL_SSL \
27+
-e KAFKA_CLUSTERS_0_PROPERTIES_SASL_MECHANISM=AWS_MSK_IAM \
28+
-e KAFKA_CLUSTERS_0_PROPERTIES_SASL_CLIENT_CALLBACK_HANDLER_CLASS=software.amazon.msk.auth.iam.IAMClientCallbackHandler \
29+
-e KAFKA_CLUSTERS_0_PROPERTIES_SASL_JAAS_CONFIG=software.amazon.msk.auth.iam.IAMLoginModule required awsProfileName="<PROFILE_NAME>"; \
30+
-d provectuslabs/kafka-ui:latest
31+
```
32+
33+
#### Configuring by application.yaml
34+
35+
```yaml
36+
kafka:
37+
clusters:
38+
- name: local
39+
bootstrapServers: <KAFKA_URL>
40+
properties:
41+
security.protocol: SASL_SSL
42+
sasl.mechanism: AWS_MSK_IAM
43+
sasl.client.callback.handler.class: software.amazon.msk.auth.iam.IAMClientCallbackHandler
44+
sasl.jaas.config: software.amazon.msk.auth.iam.IAMLoginModule required awsProfileName="<PROFILE_NAME>";
45+
```

‎configuration/authentication/oauth2.md

+80
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# OAuth2
2+
3+
## Examples to set up different oauth providers
4+
5+
### Cognito
6+
7+
```
8+
kafka:
9+
clusters:
10+
- name: local
11+
bootstrapServers: localhost:9092
12+
# ...
13+
14+
auth:
15+
type: OAUTH2
16+
oauth2:
17+
client:
18+
cognito:
19+
clientId: xxx
20+
clientSecret: yyy
21+
scope: openid
22+
client-name: cognito
23+
provider: cognito
24+
redirect-uri: http://localhost:8080/login/oauth2/code/cognito
25+
authorization-grant-type: authorization_code
26+
issuer-uri: https://cognito-idp.eu-central-1.amazonaws.com/eu-central-1_xxx
27+
jwk-set-uri: https://cognito-idp.eu-central-1.amazonaws.com/eu-central-1_xxx/.well-known/jwks.json
28+
user-name-attribute: username
29+
custom-params:
30+
type: cognito
31+
logoutUrl: https://<XXX>>.eu-central-1.amazoncognito.com/logout
32+
```
33+
34+
### Google
35+
36+
```
37+
kafka:
38+
clusters:
39+
- name: local
40+
bootstrapServers: localhost:9092
41+
# ...
42+
43+
auth:
44+
type: OAUTH2
45+
oauth2:
46+
client:
47+
google:
48+
provider: google
49+
clientId: xxx.apps.googleusercontent.com
50+
clientSecret: GOCSPX-xxx
51+
user-name-attribute: email
52+
custom-params:
53+
type: google
54+
allowedDomain: provectus.com
55+
56+
```
57+
58+
### Github:
59+
60+
```
61+
kafka:
62+
clusters:
63+
- name: local
64+
bootstrapServers: localhost:9092
65+
# ...
66+
67+
auth:
68+
type: OAUTH2
69+
oauth2:
70+
client:
71+
github:
72+
provider: github
73+
clientId: xxx
74+
clientSecret: yyy
75+
scope:
76+
- read:org
77+
user-name-attribute: login
78+
custom-params:
79+
type: github
80+
```

‎configuration/authentication/sasl_scram.md

+63
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
---
2+
description: How to configure SASL SCRAM Authentication
3+
---
4+
5+
# SASL\_SCRAM
6+
7+
You could pass sasl configs in properties section for each cluster.
8+
9+
### Examples:
10+
11+
Please replace
12+
13+
* \<KAFKA\_NAME> with cluster name
14+
* \<KAFKA\_URL> with broker list
15+
* \<KAFKA\_USERNAME> with username
16+
* \<KAFKA\_PASSWORD> with password
17+
18+
#### Running From Docker Image
19+
20+
```
21+
docker run -p 8080:8080 \
22+
-e KAFKA_CLUSTERS_0_NAME=<KAFKA_NAME> \
23+
-e KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=<KAFKA_URL> \
24+
-e KAFKA_CLUSTERS_0_PROPERTIES_SECURITY_PROTOCOL=SASL_SSL \
25+
-e KAFKA_CLUSTERS_0_PROPERTIES_SASL_MECHANISM=SCRAM-SHA-512 \
26+
-e KAFKA_CLUSTERS_0_PROPERTIES_SASL_JAAS_CONFIG=org.apache.kafka.common.security.scram.ScramLoginModule required username="<KAFKA_USERNAME>" password="<KAFKA_PASSWORD>"; \
27+
-d provectuslabs/kafka-ui:latest
28+
```
29+
30+
#### Running From Docker-compose file
31+
32+
```yaml
33+
34+
version: '3.4'
35+
services:
36+
37+
kafka-ui:
38+
image: provectuslabs/kafka-ui
39+
container_name: kafka-ui
40+
ports:
41+
- "888:8080"
42+
restart: always
43+
environment:
44+
- KAFKA_CLUSTERS_0_NAME=<KAFKA_NAME>
45+
- KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=<KAFKA_URL>
46+
- KAFKA_CLUSTERS_0_PROPERTIES_SECURITY_PROTOCOL=SASL_SSL
47+
- KAFKA_CLUSTERS_0_PROPERTIES_SASL_MECHANISM=SCRAM-SHA-512
48+
- KAFKA_CLUSTERS_0_PROPERTIES_SASL_JAAS_CONFIG=org.apache.kafka.common.security.scram.ScramLoginModule required username="<KAFKA_USERNAME>" password="<KAFKA_PASSWORD>";
49+
- KAFKA_CLUSTERS_0_PROPERTIES_PROTOCOL=SASL
50+
```
51+
52+
#### Configuring by application.yaml
53+
54+
```yaml
55+
kafka:
56+
clusters:
57+
- name: local
58+
bootstrapServers: <KAFKA_URL>
59+
properties:
60+
security.protocol: SASL_SSL
61+
sasl.mechanism: SCRAM-SHA-512
62+
sasl.jaas.config: org.apache.kafka.common.security.scram.ScramLoginModule required username="<KAFKA_USERNAME>" password="<KAFKA_PASSWORD>";
63+
```

‎configuration/authentication/sso-guide.md

+85
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
# SSO Guide
2+
3+
## How to configure SSO
4+
5+
SSO require additionaly to configure TLS for application, in that example we will use self-signed certificate, in case of use legal certificates please skip step 1.
6+
7+
### Step 1
8+
9+
At this step we will generate self-signed PKCS12 keypair.
10+
11+
```bash
12+
mkdir cert
13+
keytool -genkeypair -alias ui-for-apache-kafka -keyalg RSA -keysize 2048 \
14+
-storetype PKCS12 -keystore cert/ui-for-apache-kafka.p12 -validity 3650
15+
```
16+
17+
### Step 2
18+
19+
Create new application in any SSO provider, we will continue with [Auth0](https://auth0.com).
20+
21+
![](https://user-images.githubusercontent.com/1494347/172255269-94cb9e3a-042b-49bb-925e-a06344840662.png)
22+
23+
After that need to provide callback URLs, in our case we will use `https://127.0.0.1:8080/login/oauth2/code/auth0`
24+
25+
![](https://user-images.githubusercontent.com/1494347/172255294-86af29b9-642b-4fb5-9ba8-212185e3fdfc.png)
26+
27+
This is a main parameters required for enabling SSO
28+
29+
![](https://user-images.githubusercontent.com/1494347/172255315-4f12ac92-ca13-4206-ab68-48092e562092.png)
30+
31+
### Step 3
32+
33+
To launch UI for Apache Kafka with enabled TLS and SSO run following:
34+
35+
```bash
36+
docker run -p 8080:8080 -v `pwd`/cert:/opt/cert -e AUTH_TYPE=LOGIN_FORM \
37+
-e SECURITY_BASIC_ENABLED=true \
38+
-e SERVER_SSL_KEY_STORE_TYPE=PKCS12 \
39+
-e SERVER_SSL_KEY_STORE=/opt/cert/ui-for-apache-kafka.p12 \
40+
-e SERVER_SSL_KEY_STORE_PASSWORD=123456 \
41+
-e SERVER_SSL_KEY_ALIAS=ui-for-apache-kafka \
42+
-e SERVER_SSL_ENABLED=true \
43+
-e SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_AUTH0_CLIENTID=uhvaPKIHU4ZF8Ne4B6PGvF0hWW6OcUSB \
44+
-e SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_AUTH0_CLIENTSECRET=YXfRjmodifiedTujnkVr7zuW9ECCAK4TcnCio-i \
45+
-e SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_AUTH0_ISSUER_URI=https://dev-a63ggcut.auth0.com/ \
46+
-e SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_AUTH0_SCOPE=openid \
47+
-e TRUST_STORE=/opt/cert/ui-for-apache-kafka.p12 \
48+
-e TRUST_STORE_PASSWORD=123456 \
49+
provectuslabs/kafka-ui:latest
50+
```
51+
52+
In the case with trusted CA-signed SSL certificate and SSL termination somewhere outside of application we can pass only SSO related environment variables:
53+
54+
```bash
55+
docker run -p 8080:8080 -v `pwd`/cert:/opt/cert -e AUTH_TYPE=OAUTH2 \
56+
-e SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_AUTH0_CLIENTID=uhvaPKIHU4ZF8Ne4B6PGvF0hWW6OcUSB \
57+
-e SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_AUTH0_CLIENTSECRET=YXfRjmodifiedTujnkVr7zuW9ECCAK4TcnCio-i \
58+
-e SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_AUTH0_ISSUER_URI=https://dev-a63ggcut.auth0.com/ \
59+
-e SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_AUTH0_SCOPE=openid \
60+
provectuslabs/kafka-ui:latest
61+
```
62+
63+
### Step 4 (Load Balancer HTTP) (optional)
64+
65+
If you're using load balancer/proxy and use HTTP between the proxy and the app, you might want to set `server_forward-headers-strategy` to `native` as well (`SERVER_FORWARDHEADERSSTRATEGY=native`), for more info refer to [this issue](https://github.com/provectus/kafka-ui/issues/1017).
66+
67+
### Step 5 (Azure) (optional)
68+
69+
For Azure AD (Office365) OAUTH2 you'll want to add additional environment variables:
70+
71+
```bash
72+
docker run -p 8080:8080 \
73+
-e KAFKA_CLUSTERS_0_NAME="${cluster_name}"\
74+
-e KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS="${kafka_listeners}" \
75+
-e KAFKA_CLUSTERS_0_ZOOKEEPER="${zookeeper_servers}" \
76+
-e KAFKA_CLUSTERS_0_KAFKACONNECT_0_ADDRESS="${kafka_connect_servers}"
77+
-e AUTH_TYPE=OAUTH2 \
78+
-e SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_AUTH0_CLIENTID=uhvaPKIHU4ZF8Ne4B6PGvF0hWW6OcUSB \
79+
-e SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_AUTH0_CLIENTSECRET=YXfRjmodifiedTujnkVr7zuW9ECCAK4TcnCio-i \
80+
-e SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_AUTH0_SCOPE="https://graph.microsoft.com/User.Read" \
81+
-e SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_AUTH0_ISSUER_URI="https://login.microsoftonline.com/{tenant-id}/v2.0" \
82+
-d provectuslabs/kafka-ui:latest"
83+
```
84+
85+
Note that scope is created by default when Application registration is done in Azure portal. You'll need to update application registration manifest to include `"accessTokenAcceptedVersion": 2`

‎configuration/configuration.md

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
# Configuration
2+

0 commit comments

Comments
 (0)