Issue 3138 - Conformance Tests for BackendTLSPolicy - normative #3212
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,90 @@ | ||
| /* | ||
| Copyright 2024 The Kubernetes Authors. | ||
|
|
||
| Licensed under the Apache License, Version 2.0 (the "License"); | ||
| you may not use this file except in compliance with the License. | ||
| You may obtain a copy of the License at | ||
|
|
||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
|
|
||
| Unless required by applicable law or agreed to in writing, software | ||
| distributed under the License is distributed on an "AS IS" BASIS, | ||
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| See the License for the specific language governing permissions and | ||
| limitations under the License. | ||
| */ | ||
|
|
||
| package tests | ||
|
|
||
| import ( | ||
| "testing" | ||
|
|
||
| "k8s.io/apimachinery/pkg/types" | ||
|
|
||
| h "sigs.k8s.io/gateway-api/conformance/utils/http" | ||
| "sigs.k8s.io/gateway-api/conformance/utils/kubernetes" | ||
| "sigs.k8s.io/gateway-api/conformance/utils/suite" | ||
| "sigs.k8s.io/gateway-api/conformance/utils/tls" | ||
| "sigs.k8s.io/gateway-api/pkg/features" | ||
| ) | ||
|
|
||
| func init() { | ||
| ConformanceTests = append(ConformanceTests, BackendTLSPolicy) | ||
| } | ||
|
|
||
| var BackendTLSPolicy = suite.ConformanceTest{ | ||
| ShortName: "BackendTLSPolicy", | ||
| Description: "A single service that is targeted by a BackendTLSPolicy must successfully complete TLS termination", | ||
| Features: []features.FeatureName{ | ||
| features.SupportGateway, | ||
| features.SupportHTTPRoute, | ||
| features.SupportBackendTLSPolicy, | ||
candita marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| }, | ||
| Manifests: []string{"tests/backendtlspolicy.yaml"}, | ||
| Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { | ||
| ns := "gateway-conformance-infra" | ||
| routeNN := types.NamespacedName{Name: "gateway-conformance-infra-test", Namespace: ns} | ||
| gwNN := types.NamespacedName{Name: "gateway-backendtlspolicy", Namespace: ns} | ||
|
|
||
| kubernetes.NamespacesMustBeReady(t, suite.Client, suite.TimeoutConfig, []string{ns}) | ||
| gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAcceptedMultipleListeners(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) | ||
| kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) | ||
|
|
||
|
There was a problem hiding this comment. I think we’re also missing an assertion here to verify that the BackendTLSPolicy has been accepted, with the appropriate ancestor references and statusconditions set. |
||
| serverStr := "abc.example.com" | ||
|
|
||
| // Verify that the response to a backend-tls-only call to /backendTLS will return the matching SNI. | ||
| t.Run("Simple HTTP request targeting BackendTLSPolicy should reach infra-backend", func(t *testing.T) { | ||
| h.MakeRequestAndExpectEventuallyConsistentResponse(t, suite.RoundTripper, suite.TimeoutConfig, gwAddr, | ||
| h.ExpectedResponse{ | ||
| Namespace: ns, | ||
| Request: h.Request{ | ||
| Host: serverStr, | ||
| Path: "/backendTLS", | ||
| SNI: serverStr, | ||
| }, | ||
| Response: h.Response{StatusCode: 200}, | ||
| }) | ||
| }) | ||
|
|
||
| // For the re-encrypt case, we need to use the cert for the frontend tls listener. | ||
| certNN := types.NamespacedName{Name: "tls-checks-certificate", Namespace: ns} | ||
| cPem, keyPem, err := GetTLSSecret(suite.Client, certNN) | ||
| if err != nil { | ||
| t.Fatalf("unexpected error finding TLS secret: %v", err) | ||
| } | ||
| // Verify that the response to a re-encrypted call to /backendTLS will return the matching SNI. | ||
| t.Run("Re-encrypt HTTPS request targeting BackendTLSPolicy should reach infra-backend", func(t *testing.T) { | ||
| tls.MakeTLSRequestAndExpectEventuallyConsistentResponse(t, suite.RoundTripper, suite.TimeoutConfig, gwAddr, cPem, keyPem, serverStr, | ||
| h.ExpectedResponse{ | ||
| Namespace: ns, | ||
| Request: h.Request{ | ||
| Host: serverStr, | ||
| Path: "/backendTLS", | ||
| SNI: serverStr, | ||
| }, | ||
| Response: h.Response{StatusCode: 200}, | ||
| }) | ||
| }) | ||
shaneutt marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| }, | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,153 @@ | ||
| apiVersion: gateway.networking.k8s.io/v1 | ||
| kind: Gateway | ||
| metadata: | ||
| name: gateway-backendtlspolicy | ||
| namespace: gateway-conformance-infra | ||
|
There was a problem hiding this comment. Is there a specific reason for using a dedicated Gateway for these tests? I’m generally in favor of reusing base resources, whenever possible, as this can help reduce the execution time of the conformance test suite and, in turn, improve the efficiency of the implementation's CICD pipelines. There was a problem hiding this comment. It's a great idea but until we have the listener merging standardized, I'm not sure I want to take the risk of adding a listener to an existing gateway and hoping for the best for each implementation to pass and for old conformance tests to continue working. There was a problem hiding this comment. Why do we need a dedicated listener for the test? Would it not be sufficient e.g., to reuse the existing HTTPS listener also used in There was a problem hiding this comment. I've updated it so that the Gateway has an HTTP listener for the backend-tls-only case, and an HTTPS listener that terminates TLS for the re-encrypt use case. There was a problem hiding this comment. I still have a strong opinion that we should reuse the base Gateway wherever possible, as this has a significant impact on the conformance test suite runtime. Therefore, I suggest using the |
||
| spec: | ||
| gatewayClassName: "{GATEWAY_CLASS_NAME}" | ||
| listeners: | ||
| - name: http | ||
| port: 80 | ||
| protocol: HTTP | ||
| hostname: "abc.example.com" | ||
| allowedRoutes: | ||
| namespaces: | ||
| from: Same | ||
| kinds: | ||
| - kind: HTTPRoute | ||
| - name: https | ||
| port: 443 | ||
| protocol: HTTPS | ||
| tls: | ||
| mode: Terminate | ||
| certificateRefs: | ||
| - group: "" | ||
| kind: Secret | ||
| name: tls-checks-certificate | ||
| hostname: "abc.example.com" | ||
| allowedRoutes: | ||
| namespaces: | ||
| from: Same | ||
| kinds: | ||
| - kind: HTTPRoute | ||
| --- | ||
| apiVersion: gateway.networking.k8s.io/v1alpha3 | ||
| kind: BackendTLSPolicy | ||
| metadata: | ||
| name: normative-test-backendtlspolicy | ||
| namespace: gateway-conformance-infra | ||
| spec: | ||
| targetRefs: | ||
| - group: "" | ||
| kind: Service | ||
| name: "backendtlspolicy-test" | ||
| sectionName: "btls" | ||
| validation: | ||
| caCertificateRefs: | ||
| - group: "" | ||
| kind: ConfigMap | ||
| # This secret is generated dynamically by the test suite. | ||
| name: "backend-tls-checks-certificate" | ||
candita marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| hostname: "abc.example.com" | ||
| --- | ||
| apiVersion: gateway.networking.k8s.io/v1 | ||
| kind: HTTPRoute | ||
| metadata: | ||
| name: gateway-conformance-infra-test | ||
| namespace: gateway-conformance-infra | ||
| spec: | ||
| parentRefs: | ||
| - name: gateway-backendtlspolicy | ||
| namespace: gateway-conformance-infra | ||
| hostnames: | ||
| - abc.example.com | ||
| rules: | ||
| - backendRefs: | ||
| - group: "" | ||
| kind: Service | ||
| name: backendtlspolicy-test | ||
| port: 443 | ||
| matches: | ||
| - path: | ||
| type: Exact | ||
| value: /backendTLS | ||
| --- | ||
| apiVersion: v1 | ||
| kind: Service | ||
| metadata: | ||
| name: backendtlspolicy-test | ||
| namespace: gateway-conformance-infra | ||
| spec: | ||
| selector: | ||
| app: backendtlspolicy-test | ||
| ports: | ||
| - name: "btls" | ||
| protocol: TCP | ||
| port: 443 | ||
| targetPort: 8443 | ||
| --- | ||
| # Deployment must not be applied until after the secret is generated. | ||
|
There was a problem hiding this comment. Same here: I think we should reuse the |
||
| apiVersion: apps/v1 | ||
| kind: Deployment | ||
| metadata: | ||
| name: backendtlspolicy-test | ||
| namespace: gateway-conformance-infra | ||
| labels: | ||
| app: backendtlspolicy-test | ||
| spec: | ||
| replicas: 1 | ||
| selector: | ||
| matchLabels: | ||
| app: backendtlspolicy-test | ||
| template: | ||
| metadata: | ||
| labels: | ||
| app: backendtlspolicy-test | ||
| spec: | ||
| containers: | ||
| - name: backendtlspolicy-test | ||
| image: gcr.io/k8s-staging-gateway-api/echo-basic:v20240412-v1.0.0-394-g40c666fd | ||
| volumeMounts: | ||
| - name: ca-volume | ||
| mountPath: /etc/ca-volume | ||
| - name: secret-volume | ||
| mountPath: /etc/secret-volume | ||
| env: | ||
| - name: POD_NAME | ||
| valueFrom: | ||
| fieldRef: | ||
| fieldPath: metadata.name | ||
| - name: NAMESPACE | ||
| valueFrom: | ||
| fieldRef: | ||
| fieldPath: metadata.namespace | ||
| - name: CA_CERT | ||
| value: /etc/ca-volume/crt | ||
| - name: CA_CERT_KEY | ||
| value: /etc/ca-volume/key | ||
| - name: TLS_SERVER_CERT | ||
| value: /etc/secret-volume/crt | ||
| - name: TLS_SERVER_PRIVKEY | ||
| value: /etc/secret-volume/key | ||
| resources: | ||
| requests: | ||
| cpu: 10m | ||
| volumes: | ||
| - name: ca-volume | ||
| configMap: | ||
| # This configMap is generated dynamically by the test suite. | ||
| name: backend-tls-checks-certificate | ||
| items: | ||
| - key: ca.crt | ||
| path: crt | ||
| - key: key.crt | ||
| path: key | ||
| - name: secret-volume | ||
| secret: | ||
| # This secret is generated dynamically by the test suite. | ||
|
There was a problem hiding this comment. I don't really care but FWIW Istio just hardcodes one and embeds it into the repo/dockerfile. A bit simpler There was a problem hiding this comment. I wanted to make it easier for later tests to be able to use different SNI and certs. |
||
| secretName: tls-checks-certificate | ||
| items: | ||
| - key: tls.crt | ||
| path: crt | ||
| - key: tls.key | ||
| path: key | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: there is newer version of this image
gcr.io/k8s-staging-gateway-api/echo-basic:v20250605-v1.3.0-25-g77baa438
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks. This version will need to be updated for all once the changes for echo-basic are accepted. Then, I will split this PR, merge the echo-basic changes, and it will give me a new version to use. Then I will change all the images to use the new version.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will not be updating echo-basic after all. We've not made any changes to echo-basic that we need to pick up, since March 2024.