Skip to content

[trino][catalogs] allow user catalogs own setup #325

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 37 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
83d9fad
New squashed commit
luismacosta Feb 12, 2025
ff97f8d
squash
luismacosta Feb 12, 2025
07839bb
Merge branch 'main' into secrets-trino
luismacosta Feb 12, 2025
e34e56e
Delete charts/trino/cenko.yaml
luismacosta Feb 12, 2025
6e79f81
Update README.md
luismacosta Feb 12, 2025
3a0df08
Update README.md
luismacosta Feb 12, 2025
ead587d
Update README.md
luismacosta Feb 12, 2025
5e5b068
Update secret.yaml
luismacosta Feb 12, 2025
051fe3b
remove validation
luismacosta Feb 12, 2025
c5faf82
add tests
luismacosta Feb 13, 2025
1c665a5
add tests
luismacosta Feb 13, 2025
149ace3
add tests
luismacosta Feb 13, 2025
660ed49
Update values.yaml
luismacosta Feb 18, 2025
7dc7fee
Update README.md
luismacosta Mar 2, 2025
2ac0d31
Update values.yaml
luismacosta Mar 2, 2025
baa2a98
Update secret.yaml
luismacosta Mar 2, 2025
467783c
Update values.yaml
luismacosta Mar 2, 2025
b7e9189
Update test-catalogs-secrets-values.yaml
luismacosta Mar 2, 2025
00b2d04
Update test-catalogs-secrets-values.yaml
luismacosta Mar 2, 2025
feff8d7
Update deployment-worker.yaml
luismacosta Mar 12, 2025
ae07835
Update deployment-coordinator.yaml
luismacosta Mar 12, 2025
a3ba746
Update deployment-coordinator.yaml
luismacosta Mar 12, 2025
93daba7
Update deployment-worker.yaml
luismacosta Mar 12, 2025
9de99f9
Update deployment-coordinator.yaml
luismacosta Mar 12, 2025
e591ad9
Update deployment-worker.yaml
luismacosta Mar 12, 2025
a892ebd
Merge branch 'trinodb:main' into secrets-trino
luismacosta Mar 18, 2025
72f8214
Merge branch 'trinodb:main' into secrets-trino
luismacosta Apr 9, 2025
bb54aad
Delete tests/trino/test-catalogs-secrets-values.yaml
luismacosta Apr 9, 2025
4a7561f
Update values.yaml
luismacosta Apr 9, 2025
a31f168
Update charts/trino/values.yaml
luismacosta Apr 9, 2025
1a43135
Update charts/trino/README.md
luismacosta Apr 9, 2025
196ea67
Update charts/trino/templates/deployment-coordinator.yaml
luismacosta Apr 9, 2025
da4344a
Update charts/trino/templates/deployment-worker.yaml
luismacosta Apr 9, 2025
28bda70
Update charts/trino/templates/secret.yaml
luismacosta Apr 9, 2025
6218061
revert values.yaml
luismacosta Apr 10, 2025
e49de08
Update charts/trino/values.yaml
luismacosta Apr 10, 2025
9eec1f1
all
luismacosta Apr 10, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/ci-cd.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,11 @@ jobs:

# Everything above is CI, everything here and below is for releases and runs only on non-pull-request events
sync-readme:
<<<<<<< HEAD
needs: [lint, test, test-gateway, docs]
=======
needs: [lint, test, docs]
>>>>>>> 83d9fad (New squashed commit)
runs-on: ubuntu-latest
if: github.event_name != 'pull_request'
steps:
Expand Down Expand Up @@ -120,7 +124,11 @@ jobs:
git push

release:
<<<<<<< HEAD
needs: [lint, test, test-gateway, docs, sync-readme]
=======
needs: [lint, test, docs, sync-readme]
>>>>>>> 83d9fad (New squashed commit)
runs-on: ubuntu-latest
if: github.event_name != 'pull_request'
steps:
Expand Down
28 changes: 28 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: Lint and Test Chart
on: push

jobs:
lint-test:
runs-on: ubuntu-latest
steps:
- name: Cleanup node
uses: AutoModality/[email protected]
- name: Checkout
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Set up Helm
uses: azure/setup-helm@v1
with:
version: v3.4.0
- uses: actions/setup-python@v2
with:
python-version: 3.7
- name: Set up chart-testing
uses: helm/[email protected]
- name: Run chart-testing (lint)
run: ct lint --charts=charts/trino --validate-maintainers=false
- name: Create kind cluster
uses: helm/[email protected]
- name: Run chart-testing (install)
run: ct install --charts=charts/trino
29 changes: 29 additions & 0 deletions .github/workflows/release.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Release Trino Chart

on:
push:
branches:
- main

jobs:
release:
runs-on: ubuntu-latest
steps:
- name: Cleanup node
uses: AutoModality/[email protected]
- name: Checkout
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Configure Git
run: |
git config user.name "$GITHUB_ACTOR"
git config user.email "[email protected]"
- name: Install Helm
uses: azure/setup-helm@v1
with:
version: v3.5.2
- name: Run chart-releaser
uses: helm/[email protected]
env:
CR_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
30 changes: 30 additions & 0 deletions .github/workflows/sync-readme.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: Sync Readme

on:
push:
branches:
- 'master'
paths:
- 'README.md'

jobs:
sync-readme:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
- name: copy README.md
run: |
cp -f README.md ${{ runner.temp }}/README.md
- name: Checkout gh-pages
uses: actions/checkout@v2
with:
ref: gh-pages
- name: commit
run: |
cp -f ${{ runner.temp }}/README.md .
git config user.name trino_community
git config user.email "[email protected]"
git add README.md
git commit --signoff -m "Sync README from main"
git push
16 changes: 16 additions & 0 deletions .helmignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,28 @@
*.swp
*.bak
*.tmp
<<<<<<< HEAD
<<<<<<< HEAD
*.orig
=======
>>>>>>> 1cc98e4 (Initial commit)
=======
*.orig
>>>>>>> 83d9fad (New squashed commit)
*~
# Various IDEs
.project
.idea/
*.tmproj
<<<<<<< HEAD
<<<<<<< HEAD
.vscode/
# helm-doc template
README.md.gotmpl
=======
>>>>>>> 1cc98e4 (Initial commit)
=======
.vscode/
# helm-doc template
README.md.gotmpl
>>>>>>> 83d9fad (New squashed commit)
145 changes: 145 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@

<<<<<<< HEAD
<<<<<<< HEAD
Trino Community Kubernetes Helm Charts
===========
[![CI/CD](https://github.com/trinodb/charts/actions/workflows/ci-cd.yaml/badge.svg?branch=main)](https://github.com/trinodb/charts/actions/workflows/ci-cd.yaml)
Expand All @@ -14,12 +16,48 @@ included:
[Helm](https://helm.sh) must be installed to use the charts.
Refer to Helm's [documentation](https://helm.sh/docs/) to get started.

<<<<<<< HEAD
Once Helm is set up properly, add the repo as follows:
=======
| Parameter | Description | Default |
| ------------------------ | ----------------------- | -------------- |
| `image.repository` | | `"trinodb/trino"` |
| `image.pullPolicy` | | `"IfNotPresent"` |
| `image.tag` | | `"latest"` |
| `server.workers` | | `2` |
| `server.node.environment` | | `"production"` |
| `server.node.dataDir` | | `"/data/trino"` |
| `server.node.pluginDir` | | `"/usr/lib/trino/plugin"` |
| `server.log.trino.level` | | `"INFO"` |
| `server.config.path` | | `"/etc/trino"` |
| `server.config.http.port` | | `8080` |
| `server.config.query.maxMemory` | | `"4GB"` |
| `server.config.query.maxMemoryPerNode` | | `"1GB"` |
| `server.config.query.maxTotalMemoryPerNode` | | `"2GB"` |
| `server.config.memory.heapHeadroomPerNode` | | `"1GB"` |
| `server.jvm.maxHeapSize` | | `"8G"` |
| `server.jvm.gcMethod.type` | | `"UseG1GC"` |
| `server.jvm.gcMethod.g1.heapRegionSize` | | `"32M"` |
| `additionalNodeProperties` | | `{}` |
| `additionalJVMConfig` | | `{}` |
| `additionalConfigProperties` | | `{}` |
| `additionalLogProperties` | | `{}` |
| `additionalCatalogs` | | `{}` |
| `securityContext.runAsUser` | | `1000` |
| `securityContext.runAsGroup` | | `1000` |
| `service.type` | | `"ClusterIP"` |
| `service.port` | | `8080` |
| `resources` | | `{}` |
| `nodeSelector` | | `{}` |
| `tolerations` | | `[]` |
| `affinity` | | `{}` |
>>>>>>> 7f0562e (extend chart, add GHA for CI and Release)

```console
helm repo add trino https://trinodb.github.io/charts/
```

<<<<<<< HEAD
Run `helm search repo trino` to see the latest charts with the string `trino` in
the name to get an output similar to the following:

Expand All @@ -28,6 +66,10 @@ NAME CHART VERSION APP VERSION DESCRIPTION
trino/trino 1.38.0 474 Fast distributed SQL query engine for big data ...
trino/trino-gateway 1.15.1 15 A Helm chart for Trino Gateway
```
=======
---
_Documentation generated by [Frigate](https://frigate.readthedocs.io)._
>>>>>>> 7f0562e (extend chart, add GHA for CI and Release)

Use `helm search repo trino -l` for information about all available versions.

Expand Down Expand Up @@ -94,3 +136,106 @@ you [install the pre-commit binary](https://pre-commit.com/#install), then run:
pre-commit install
pre-commit install-hooks
```
=======
Trino
=======
Trino Community Kubernetes Helm Charts
>>>>>>> 83d9fad (New squashed commit)
===========
[![CI/CD](https://github.com/trinodb/charts/actions/workflows/ci-cd.yaml/badge.svg?branch=main)](https://github.com/trinodb/charts/actions/workflows/ci-cd.yaml)

A repository of Helm charts for the Trino community. The following charts are
included:

* `trino/trino` for [Trino](https://trino.io/)
* `trino/trino-gateway` for [Trino Gateway](https://trinodb.github.io/trino-gateway)

## Usage

[Helm](https://helm.sh) must be installed to use the charts.
Refer to Helm's [documentation](https://helm.sh/docs/) to get started.

Once Helm is set up properly, add the repo as follows:

```console
helm repo add trino https://trinodb.github.io/charts/
```

Run `helm search repo trino` to see the latest charts with the string `trino` in
the name to get an output similar to the following:

```
NAME CHART VERSION APP VERSION DESCRIPTION
trino/trino 1.37.0 470 Fast distributed SQL query engine for big data ...
trino/trino-gateway 1.13.2 13 A Helm chart for Trino Gateway
```

<<<<<<< HEAD
>>>>>>> 1cc98e4 (Initial commit)
=======
Use `helm search repo trino -l` for information about all available versions.

After configuring your Kubernetes cluster, you can install Trino with the chart
`trino/trino` using:

```console
helm install my-trino trino/trino --version 1.37.0
```

Also, you can check the manifests using:

```console
helm template my-trino trino/trino --namespace <YOUR_NAMESPACE>
```

Similarly install Trino Gateway with the `trino/trino-gateway` chart.

## Documentation

More information about Trino, Trino Gateway, and the charts is available in the
following resources:

* [Trino Kubernetes documentation](https://trino.io/docs/current/installation/kubernetes.html)
* [trino/trino chart configuration](./charts/trino/README.md)
* [Trino documentation](https://trino.io/docs/current/index.html)
* [Trino Gateway Kubernetes documentation](https://trinodb.github.io/trino-gateway/installation/#helm)
* [trino/trino-gateway chart configuration](./charts/gateway/README.md)
* [Trino Gateway documentation](https://trinodb.github.io/trino-gateway)

## Development

To test the chart, install it into a Kubernetes cluster. Use `kind` to create a
Kubernetes cluster running in a container, and `chart-testing` to install the
chart and run [tests](charts/trino/templates/tests).

```console
brew install helm kind chart-testing
kind create cluster
ct install
```

To run tests with specific values:
```console
ct install --helm-extra-set-args "--set image.tag=470"
```

Use the `test.sh` script to run a suite of tests, with different chart values.
If some of the tests fail, use the `-s` flag to skip cleanup and inspect the
resources installed in the Kubernetes cluster. Use `-n` to use a specific
namespace, not a randomly generated one. Use `-t` to run only selected tests.
See the command help (`-h`) for a list of available tests.

Example:
```console
./test.sh -n trino -s -t default
```

The documentation is automatically generated from the chart files. Install a
git hook to have it automatically updated when committing changes. Make sure
you [install the pre-commit binary](https://pre-commit.com/#install), then run:

```console
pre-commit install
pre-commit install-hooks
```
>>>>>>> 83d9fad (New squashed commit)
5 changes: 5 additions & 0 deletions charts/gateway/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,13 @@ apiVersion: v2
name: trino-gateway
description: A Helm chart for Trino Gateway
type: application
<<<<<<< HEAD
version: "1.15.1"
appVersion: "15"
=======
version: "1.13.2"
appVersion: "13"
>>>>>>> 83d9fad (New squashed commit)

icon: https://trino.io/assets/images/logos/trino-gateway-small.png

Expand Down
12 changes: 12 additions & 0 deletions charts/gateway/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
# trino-gateway

<<<<<<< HEAD
![Version: 1.15.1](https://img.shields.io/badge/Version-1.15.1-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 15](https://img.shields.io/badge/AppVersion-15-informational?style=flat-square)
=======
![Version: 1.13.2](https://img.shields.io/badge/Version-1.13.2-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 13](https://img.shields.io/badge/AppVersion-13-informational?style=flat-square)
>>>>>>> 83d9fad (New squashed commit)

A Helm chart for Trino Gateway

Expand Down Expand Up @@ -47,7 +51,15 @@ A Helm chart for Trino Gateway
* `config.dataStore.password` - string, default: `"mysecretpassword"`
* `config.dataStore.driver` - string, default: `"org.postgresql.Driver"`
* `config.clusterStatsConfiguration.monitorType` - string, default: `"INFO_API"`
<<<<<<< HEAD
* `command` - list, default: `["java","-XX:MinRAMPercentage=80.0","-XX:MaxRAMPercentage=80.0","-jar","/usr/lib/trino-gateway/gateway-ha-jar-with-dependencies.jar","/etc/trino-gateway/config.yaml"]`
=======
* `config.modules[0]` - string, default: `"io.trino.gateway.ha.module.HaGatewayProviderModule"`
* `config.modules[1]` - string, default: `"io.trino.gateway.ha.module.ClusterStateListenerModule"`
* `config.modules[2]` - string, default: `"io.trino.gateway.ha.module.ClusterStatsMonitorModule"`
* `config.managedApps[0]` - string, default: `"io.trino.gateway.ha.clustermonitor.ActiveClusterMonitor"`
* `command` - list, default: `["java","-XX:MinRAMPercentage=80.0","-XX:MaxRAMPercentage=80.0","-jar","/usr/lib/trino/gateway-ha-jar-with-dependencies.jar","/etc/gateway/config.yaml"]`
>>>>>>> 83d9fad (New squashed commit)

Startup command for Trino Gateway process. Add additional Java options and other modifications as desired.
* `service` - object, default: `{"ports":[{"name":"gateway","protocol":"TCP"}],"type":"ClusterIP"}`
Expand Down
Loading