Modifications to Run integration Tests With Sail Operator #300
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| platform: openshift | ||
| autoscaling: | ||
| enabled: false | ||
|
|
||
| meshConfig: | ||
| accessLogFile: /dev/stdout | ||
|
|
||
| service: | ||
| type: ClusterIP |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| platform: openshift | ||
| autoscaling: | ||
| enabled: false | ||
|
|
||
| meshConfig: | ||
| accessLogFile: /dev/stdout | ||
|
|
||
| service: | ||
| ports: | ||
| - port: 15021 | ||
| targetPort: 15021 | ||
| name: status-port | ||
| - port: 80 | ||
| targetPort: 8080 | ||
| name: http2 | ||
| - port: 443 | ||
| targetPort: 8443 | ||
| name: https | ||
| # This is the port where sni routing happens | ||
| - port: 15443 | ||
| targetPort: 15443 | ||
| name: tls | ||
| ## Extra ports for testing | ||
| - port: 15012 | ||
| targetPort: 15012 | ||
| name: tls-istiod | ||
| - port: 15017 | ||
| targetPort: 15017 | ||
| name: tls-webhook | ||
| - port: 31400 | ||
| targetPort: 31400 | ||
| name: tcp |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| apiVersion: sailoperator.io/v1 | ||
| kind: IstioCNI | ||
| metadata: | ||
| name: default | ||
| spec: | ||
| namespace: ${ISTIOCNI_NAMESPACE} | ||
| version: ${ISTIO_VERSION} | ||
| profile: openshift | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| apiVersion: admissionregistration.k8s.io/v1 | ||
| kind: ValidatingWebhookConfiguration | ||
fjglira marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| metadata: | ||
| name: istiod-default-validator | ||
| labels: | ||
| app: istiod | ||
| release: istio | ||
| webhooks: | ||
| - name: rev.validation.istio.io | ||
| clientConfig: | ||
| service: | ||
| name: istiod | ||
| namespace: istio-system | ||
| path: /validate | ||
| caBundle: <base64-encoded-CA-cert> | ||
| rules: | ||
| - apiGroups: ["security.istio.io", "networking.istio.io", "telemetry.istio.io", "extensions.istio.io"] | ||
| apiVersions: ["*"] | ||
| operations: ["CREATE", "UPDATE"] | ||
| resources: ["*"] | ||
| scope: "*" | ||
| failurePolicy: Fail | ||
| matchPolicy: Equivalent | ||
| admissionReviewVersions: ["v1"] | ||
| sideEffects: None | ||
| timeoutSeconds: 10 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,185 @@ | ||
| #!/bin/bash | ||
|
|
||
| # Copyright 2019 Istio Authors | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| # The integration test runtime is calling this script two times if istio.test.kube.controlPlaneInstaller parameter set. One call is with | ||
| # install and another is with cleanup. On install script is used to convert istio in-cluster operator config to sail operator config and install istiod, istio-cni and gateways. | ||
| # On cleanup istiod, istio-cni, istio-ingressgateway and istio-engressgateway are cleaned | ||
| # The output log of this script is printed under working directory set by: --istio.test.work_dir/sail-operator-setup.log | ||
ctartici marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| # Upstream WoW to call this script is documented in here: https://github.com/openshift-service-mesh/istio/tree/master/tests/integration#running-tests-on-custom-deployment | ||
|
|
||
| LOG_FILE="$2/sail-operator-setup.log" | ||
| # Redirect stdout and stderr to the log file | ||
| exec > >(awk '{print strftime("[%Y-%m-%d %H:%M:%S]"), $0}' | tee -a "$LOG_FILE") 2>&1 | ||
|
|
||
| # Exit immediately for non zero status | ||
| set -e | ||
| # Check unset variables | ||
| set -u | ||
| # Print commands | ||
| set -x | ||
ctartici marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| # fail if any command in the pipeline fails | ||
| set -o pipefail | ||
|
|
||
| SKIP_CLEANUP="${SKIP_CLEANUP:-"false"}" | ||
|
|
||
|
|
||
| function usage() { | ||
| echo "Usage: $0 <install|cleanup> <input_yaml>" | ||
| echo "Example: $0 install /path/to/iop.yaml" | ||
| exit 1 | ||
| } | ||
|
|
||
| if [[ $# -lt 2 ]]; then | ||
| echo "Error: Missing required arguments." | ||
| usage | ||
| fi | ||
|
|
||
| if ! command -v yq &>/dev/null; then | ||
| echo "Error: 'yq' is not installed. Please install it before running the script." | ||
| exit 1 | ||
| fi | ||
|
|
||
| if ! command -v helm &> /dev/null; then | ||
| echo "Helm is not installed. Please install Helm before proceeding." | ||
| exit 1 | ||
| fi | ||
|
|
||
| WD=$(dirname "$0") | ||
| PROW="$(dirname "$WD")" | ||
| ROOT="$(dirname "$PROW")" | ||
|
|
||
| WORKDIR="$2" | ||
| # iop.yaml is the static file name for istiod config created by upstream integration test runtime | ||
| IOP_FILE="$2"/iop.yaml | ||
| SAIL_IOP_FILE="$(basename "${IOP_FILE%.yaml}")-sail.yaml" | ||
|
|
||
| CONVERTER_BRANCH="${CONVERTER_BRANCH:-main}" | ||
|
|
||
| # get istio version from versions.yaml | ||
| VERSION_FILE="https://raw.githubusercontent.com/istio-ecosystem/sail-operator/$CONVERTER_BRANCH/pkg/istioversion/versions.yaml" | ||
| if [ -z "${ISTIO_VERSION:-}" ]; then | ||
| ISTIO_VERSION="$(curl -s "$VERSION_FILE" | grep -E 'name: v[0-9]+\.[0-9]+' | sed -E 's/.*(v[0-9]+\.[0-9]+).*/\1/' | sort -Vr | head -n1)-latest" | ||
| fi | ||
|
Comment on lines
+71
to
+75
There was a problem hiding this comment. Very small change: I think it will be better to set also |
||
|
|
||
| NAMESPACE="${NAMESPACE:-istio-system}" | ||
| ISTIOCNI_NAMESPACE="${ISTIOCNI_NAMESPACE:-istio-cni}" | ||
|
|
||
| ISTIOCNI="${PROW}/config/sail-operator/istio-cni.yaml" | ||
| INGRESS_GATEWAY_VALUES="${PROW}/config/sail-operator/ingress-gateway-values.yaml" | ||
| EGRESS_GATEWAY_VALUES="${PROW}/config/sail-operator/egress-gateway-values.yaml" | ||
|
|
||
| CONVERTER_ADDRESS="https://raw.githubusercontent.com/istio-ecosystem/sail-operator/$CONVERTER_BRANCH/tools/configuration-converter.sh" | ||
| CONVERTER_SCRIPT=$(basename "$CONVERTER_ADDRESS") | ||
|
|
||
| function download_execute_converter(){ | ||
| cd "${PROW}" | ||
| curl -fsSL "$CONVERTER_ADDRESS" -o "$CONVERTER_SCRIPT" || { echo "Failed to download converter script"; exit 1; } | ||
| chmod +x "$CONVERTER_SCRIPT" | ||
| bash "$CONVERTER_SCRIPT" "$IOP_FILE" -v "$ISTIO_VERSION" -n "$NAMESPACE" || { echo "Failed to execute converter script"; exit 1; } | ||
| rm "$CONVERTER_SCRIPT" | ||
| } | ||
|
|
||
| function install_istio_cni(){ | ||
| oc create namespace "${ISTIOCNI_NAMESPACE}" || true | ||
| TMP_ISTIOCNI=$WORKDIR/istio-cni.yaml | ||
| cp "$ISTIOCNI" "$TMP_ISTIOCNI" | ||
| yq -i ".spec.namespace=\"$ISTIOCNI_NAMESPACE\"" "$TMP_ISTIOCNI" | ||
| yq -i ".spec.version=\"$ISTIO_VERSION\"" "$TMP_ISTIOCNI" | ||
| oc apply -f "$TMP_ISTIOCNI" | ||
| echo "istioCNI created." | ||
| } | ||
|
|
||
| function install_istio(){ | ||
| # overwrite sailoperator version before applying it | ||
| oc create namespace "${NAMESPACE}" || true | ||
| if [ "${SAIL_API_VERSION:-}" != "" ]; then | ||
| yq -i eval ".apiVersion = \"sailoperator.io/$SAIL_API_VERSION\"" "$WORKDIR/$SAIL_IOP_FILE" | ||
| fi | ||
| patch_config | ||
| oc apply -f "$WORKDIR/$SAIL_IOP_FILE" || { echo "Failed to install istio"; kubectl get istio default -o yaml;} | ||
| oc -n "$NAMESPACE" wait --for=condition=Available deployment/istiod --timeout=240s || { sleep 60; } | ||
| echo "istiod created." | ||
| } | ||
|
|
||
| SECRET_NAME="istio-ca-secret" | ||
| WEBHOOK_FILE="$PROW/config/sail-operator/validatingwebhook.yaml" | ||
|
|
||
| function patch_config() { | ||
| # adds some control plane values that are mandatory and not available in iop.yaml | ||
| if [[ "$WORKDIR" == *"telemetry-tracing-zipkin"* ]]; then | ||
| # Workaround until https://github.com/istio/istio/pull/55408 is merged | ||
| yq eval ' | ||
| .spec.values.meshConfig.enableTracing = true | | ||
| .spec.values.pilot.traceSampling = 100.0 | | ||
| .spec.values.global.proxy.tracer = "zipkin" | ||
| ' -i "$WORKDIR/$SAIL_IOP_FILE" | ||
| echo "Configured tracing for Zipkin." | ||
| fi | ||
|
|
||
| # Workaround until https://github.com/istio-ecosystem/sail-operator/issues/749 is fixed | ||
| CA_BUNDLE=$(kubectl get secret "$SECRET_NAME" -n "$NAMESPACE" -o yaml 2>/dev/null | grep "ca-cert" | awk '{print $2}') | ||
|
|
||
| # If not found, sleep for 5 seconds and retry once | ||
| if [ -z "$CA_BUNDLE" ]; then | ||
| echo "Secret not found. Sleeping for 5 seconds before retrying..." | ||
| sleep 5 | ||
|
|
||
| # Retry once | ||
| CA_BUNDLE=$(kubectl get secret "$SECRET_NAME" -n "$NAMESPACE" -o yaml 2>/dev/null | grep "ca-cert" | awk '{print $2}') | ||
|
|
||
| if [ -z "$CA_BUNDLE" ]; then | ||
| echo "Secret still not found after retry. Exiting." | ||
| exit 1 | ||
| fi | ||
| fi | ||
|
|
||
| sed -i "s|<base64-encoded-CA-cert>|$CA_BUNDLE|g" "$WEBHOOK_FILE" | ||
| kubectl apply -f "$WEBHOOK_FILE" | ||
| sed -i "s|$CA_BUNDLE|<base64-encoded-CA-cert>|g" "$WEBHOOK_FILE" | ||
| } | ||
|
|
||
| # Install ingress and egress gateways | ||
| function install_gateways(){ | ||
| helm template -n "$NAMESPACE" istio-ingressgateway "${ROOT}"/manifests/charts/gateway --values "$INGRESS_GATEWAY_VALUES" > "${WORKDIR}"/istio-ingressgateway.yaml | ||
| oc apply -f "${WORKDIR}"/istio-ingressgateway.yaml | ||
| helm template -n "$NAMESPACE" istio-egressgateway "${ROOT}"/manifests/charts/gateway --values "$EGRESS_GATEWAY_VALUES" > "${WORKDIR}"/istio-egressgateway.yaml | ||
| oc apply -f "${WORKDIR}"/istio-egressgateway.yaml | ||
| oc -n "$NAMESPACE" wait --for=condition=Available deployment/istio-ingressgateway --timeout=60s || { echo "Failed to start istio-ingressgateway"; oc get pods -n "$NAMESPACE" -o wide; oc describe pod $(oc get pods -n istio-system --no-headers | awk "$3==\"ErrImagePull\" {print $1}" | head -n 1) -n istio-system; exit 1;} | ||
| oc -n "$NAMESPACE" wait --for=condition=Available deployment/istio-egressgateway --timeout=60s || { echo "Failed to start istio-egressgateway"; kubectl get istios; oc get pods -n "$NAMESPACE" -o wide; exit 1;} | ||
| echo "Gateways created." | ||
| } | ||
|
|
||
| function cleanup_istio(){ | ||
| kubectl delete all --all -n "$ISTIOCNI_NAMESPACE" | ||
| kubectl delete all --all -n "$NAMESPACE" | ||
| kubectl delete istios.sailoperator.io --all --all-namespaces --wait=true | ||
| kubectl get clusterrole | grep istio | awk '{print $1}' | xargs kubectl delete clusterrole | ||
| kubectl get clusterrolebinding | grep istio | awk '{print $1}' | xargs kubectl delete clusterrolebinding | ||
| echo "Cleanup completed." | ||
| } | ||
|
|
||
| if [ "$1" = "install" ]; then | ||
| download_execute_converter || { echo "Failed to execute converter"; exit 1; } | ||
| install_istio_cni || { echo "Failed to install Istio CNI"; exit 1; } | ||
| install_istio || { echo "Failed to install Istio"; exit 1; } | ||
| install_gateways || { echo "Failed to install gateways"; exit 1; } | ||
| elif [ "$1" = "cleanup" ]; then | ||
| if [ "$SKIP_CLEANUP" = "true" ]; then | ||
| echo "Skipping cleanup because SKIP_CLEANUP is set to true." | ||
| else | ||
| cleanup_istio || { echo "Failed to cleanup cluster"; exit 1; } | ||
| fi | ||
| fi | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Question: is a profile needed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure about this. Got the yaml from our jenkins config files. I can remove and test it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's test in a consecutive PR