Skip to content

Commit 2117b20

Browse files
authored
Terraform state backup and restore tool (#314)
* Add `restore` tool Signed-off-by: loheagn <[email protected]> * Add examples and README for `restore` tool Signed-off-by: loheagn <[email protected]> * Fix typo Signed-off-by: loheagn <[email protected]>
1 parent 971d169 commit 2117b20

File tree

9 files changed

+1568
-0
lines changed

9 files changed

+1568
-0
lines changed

hack/tool/backup_restore/README.md

+197
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,197 @@
1+
# Backup & Restore
2+
3+
This go module is a command line tool to back up and restore the configuration and the Terraform state.
4+
5+
It has two subcommands `backup` and `restore`.
6+
7+
## `restore`
8+
9+
The main usage of the `restore` subcommand is to import an "outside" Terraform instance (maybe created by the terraform command line or managed by another terraform-controller before) to the terraform-controller in the target Kubernetes without recreating the cloud resources.
10+
11+
To use the `restore` subcommand, you should:
12+
13+
1. Prepare the `configuration.yaml` file and the `state.json` file whose content is the state json of the Terraform instance which would be imported.
14+
15+
2. Confirm that the environment variables set in terraform-controller are also set in the current execution environment.
16+
17+
3. Run the `restore` command: `go run main.go restore --configuration <path/to/your/configuration.yaml> --state <path/to/your/state.json>`
18+
19+
To find more usages of the `restore` subcommand, please run `go run main.go restore -h` for help.
20+
21+
### Example
22+
23+
You can find the example in the directory [examples/oss](examples/oss).
24+
25+
Assume that we have created an oss bucket named `restore-example` by applying the `main.tf` using the Terraform command line tool.
26+
27+
```hcl
28+
provider "alicloud" {
29+
alias = "bj-prod"
30+
region = "cn-beijing"
31+
}
32+
33+
resource "alicloud_oss_bucket" "bucket-acl" {
34+
bucket = var.bucket
35+
acl = var.acl
36+
}
37+
38+
output "BUCKET_NAME" {
39+
value = "${alicloud_oss_bucket.bucket-acl.bucket}.${alicloud_oss_bucket.bucket-acl.extranet_endpoint}"
40+
}
41+
42+
variable "bucket" {
43+
description = "OSS bucket name"
44+
default = "vela-website"
45+
type = string
46+
}
47+
48+
variable "acl" {
49+
description = "OSS bucket ACL, supported 'private', 'public-read', 'public-read-write'"
50+
default = "private"
51+
type = string
52+
}
53+
```
54+
55+
```shell
56+
terraform init && terraform apply -f main.tf
57+
```
58+
59+
We can get the Terraform state from the backend using `terraform state pull` and store it in the `state.json`. The Terraform state is a json string like the flowing:
60+
61+
```json
62+
{
63+
"version": 4,
64+
"terraform_version": "1.1.9",
65+
"serial": 4,
66+
"lineage": "*******",
67+
"outputs": {
68+
"BUCKET_NAME": {
69+
"value": "restore-example.oss-cn-beijing.aliyuncs.com",
70+
"type": "string"
71+
}
72+
},
73+
"resources": [
74+
{
75+
"mode": "managed",
76+
"type": "alicloud_oss_bucket",
77+
"name": "bucket-acl",
78+
"provider": "provider[\"registry.terraform.io/hashicorp/alicloud\"]",
79+
"instances": [
80+
{
81+
"schema_version": 0,
82+
"attributes": {
83+
"acl": "private",
84+
"bucket": "restore-example",
85+
"cors_rule": [],
86+
"creation_date": "2022-05-25",
87+
"extranet_endpoint": "oss-cn-beijing.aliyuncs.com",
88+
"force_destroy": false,
89+
"id": "restore-example",
90+
"intranet_endpoint": "oss-cn-beijing-internal.aliyuncs.com",
91+
"lifecycle_rule": [],
92+
"location": "oss-cn-beijing",
93+
"logging": [],
94+
"logging_isenable": null,
95+
"owner": "*******",
96+
"policy": "",
97+
"redundancy_type": "LRS",
98+
"referer_config": [],
99+
"server_side_encryption_rule": [],
100+
"storage_class": "Standard",
101+
"tags": {},
102+
"transfer_acceleration": [],
103+
"versioning": [],
104+
"website": []
105+
},
106+
"sensitive_attributes": [],
107+
"private": "*******=="
108+
}
109+
]
110+
}
111+
]
112+
}
113+
```
114+
115+
Now, we need to restore(import) the Terraform instance into the target terraform-controller.
116+
117+
First, we need to prepare the `configuration.yaml`:
118+
119+
```yaml
120+
apiVersion: terraform.core.oam.dev/v1beta2
121+
kind: Configuration
122+
metadata:
123+
name: alibaba-oss-bucket-hcl-restore-example
124+
spec:
125+
hcl: |
126+
resource "alicloud_oss_bucket" "bucket-acl" {
127+
bucket = var.bucket
128+
acl = var.acl
129+
}
130+
131+
output "BUCKET_NAME" {
132+
value = "${alicloud_oss_bucket.bucket-acl.bucket}.${alicloud_oss_bucket.bucket-acl.extranet_endpoint}"
133+
}
134+
135+
variable "bucket" {
136+
description = "OSS bucket name"
137+
default = "loheagn-terraform-controller-2"
138+
type = string
139+
}
140+
141+
variable "acl" {
142+
description = "OSS bucket ACL, supported 'private', 'public-read', 'public-read-write'"
143+
default = "private"
144+
type = string
145+
}
146+
147+
backend:
148+
backendType: kubernetes
149+
kubernetes:
150+
secret_suffix: 'ali-oss'
151+
namespace: 'default'
152+
153+
variable:
154+
bucket: "restore-example"
155+
acl: "private"
156+
157+
writeConnectionSecretToRef:
158+
name: oss-conn
159+
namespace: default
160+
161+
```
162+
163+
Second, we need to confirm that the environment variables set in terraform-controller are also set in the current execution environment. In this case, please make sure you have set `ALICLOUD_ACCESS_KEY` and `ALICLOUD_SECRET_KEY`.
164+
165+
Third, run the restore subcommand:
166+
167+
```shell
168+
go run main.go restore --configuration examples/oss/configuration.yaml --state examples/oss/state.json
169+
```
170+
171+
Finally, you can check the status of the configuration restored just now:
172+
173+
```shell
174+
$ kubectl get configuration.terraform.core.oam.dev
175+
NAME STATE AGE
176+
alibaba-oss-bucket-hcl-restore-example Available 13m
177+
```
178+
179+
And you can check the logs of the `terraform-executor` in the pod of the "terraform apply" job:
180+
181+
```shell
182+
$ kubectl logs alibaba-oss-bucket-hcl-restore-example-apply--1-b29d6 terraform-executor
183+
alicloud_oss_bucket.bucket-acl: Refreshing state... [id=restore-example]
184+
185+
No changes. Your infrastructure matches the configuration.
186+
187+
Terraform has compared your real infrastructure against your configuration
188+
and found no differences, so no changes are needed.
189+
190+
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
191+
192+
Outputs:
193+
194+
BUCKET_NAME = "restore-example.oss-cn-beijing.aliyuncs.com"
195+
```
196+
197+
You can see the "No changes.". This shows that we did not recreate cloud resources during the restore process.
+190
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,190 @@
1+
/*
2+
Copyright 2022 The KubeVela Authors.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/
16+
17+
package cmd
18+
19+
import (
20+
"bytes"
21+
"compress/gzip"
22+
"context"
23+
"fmt"
24+
"log"
25+
"os"
26+
"os/exec"
27+
"path/filepath"
28+
29+
"github.com/oam-dev/terraform-controller/api/v1beta2"
30+
"github.com/oam-dev/terraform-controller/controllers/configuration/backend"
31+
"github.com/spf13/cobra"
32+
v1 "k8s.io/api/core/v1"
33+
kerrors "k8s.io/apimachinery/pkg/api/errors"
34+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
35+
"k8s.io/apimachinery/pkg/runtime"
36+
"k8s.io/apimachinery/pkg/runtime/serializer/json"
37+
"k8s.io/cli-runtime/pkg/genericclioptions"
38+
"sigs.k8s.io/controller-runtime/pkg/client"
39+
)
40+
41+
var (
42+
stateJSONPath string
43+
configurationPath string
44+
k8sClient client.Client
45+
)
46+
47+
// newRestoreCmd represents the restore command
48+
func newRestoreCmd(kubeFlags *genericclioptions.ConfigFlags) *cobra.Command {
49+
restoreCmd := &cobra.Command{
50+
Use: "restore",
51+
RunE: func(cmd *cobra.Command, args []string) error {
52+
var err error
53+
k8sClient, err = buildClientSet(kubeFlags)
54+
if err != nil {
55+
return err
56+
}
57+
pwd, err := os.Getwd()
58+
if err != nil {
59+
return err
60+
}
61+
stateJSONPath = filepath.Join(pwd, stateJSONPath)
62+
configurationPath = filepath.Join(pwd, configurationPath)
63+
return restore(context.Background())
64+
},
65+
}
66+
restoreCmd.Flags().StringVar(&stateJSONPath, "state", "state.json", "the path of the backed up Terraform state file")
67+
restoreCmd.Flags().StringVar(&configurationPath, "configuration", "configuration.yaml", "the path of the backed up configuration objcet yaml file")
68+
return restoreCmd
69+
}
70+
71+
func buildClientSet(kubeFlags *genericclioptions.ConfigFlags) (client.Client, error) {
72+
config, err := kubeFlags.ToRESTConfig()
73+
if err != nil {
74+
return nil, err
75+
}
76+
return client.New(config, client.Options{})
77+
}
78+
79+
func restore(ctx context.Context) error {
80+
configuration, err := getConfiguration()
81+
if err != nil {
82+
return err
83+
}
84+
backendInterface, err := backend.ParseConfigurationBackend(configuration, k8sClient)
85+
if err != nil {
86+
return err
87+
}
88+
89+
// restore the backend
90+
if err := resumeK8SBackend(ctx, backendInterface); err != nil {
91+
return err
92+
}
93+
94+
// apply the configuration yaml
95+
// FIXME (loheagn) use the restClient to do this
96+
applyCmd := exec.Command("bash", "-c", fmt.Sprintf("kubectl apply -f %s", configurationPath))
97+
if err := applyCmd.Run(); err != nil {
98+
return err
99+
}
100+
return nil
101+
}
102+
103+
func getConfiguration() (*v1beta2.Configuration, error) {
104+
configurationYamlBytes, err := os.ReadFile(configurationPath)
105+
if err != nil {
106+
return nil, err
107+
}
108+
configuration := &v1beta2.Configuration{}
109+
scheme := runtime.NewScheme()
110+
serializer := json.NewSerializerWithOptions(json.DefaultMetaFactory, scheme, scheme, json.SerializerOptions{Yaml: true})
111+
if _, _, err := serializer.Decode(configurationYamlBytes, nil, configuration); err != nil {
112+
return nil, err
113+
}
114+
return configuration, nil
115+
}
116+
117+
func resumeK8SBackend(ctx context.Context, backendInterface backend.Backend) error {
118+
k8sBackend, ok := backendInterface.(*backend.K8SBackend)
119+
if !ok {
120+
log.Println("the configuration doesn't use the kubernetes backend, no need to restore the Terraform state")
121+
return nil
122+
}
123+
124+
tfState, err := compressedTFState()
125+
if err != nil {
126+
return err
127+
}
128+
129+
var gotSecret v1.Secret
130+
131+
configureSecret := func() {
132+
if gotSecret.Annotations == nil {
133+
gotSecret.Annotations = make(map[string]string)
134+
}
135+
gotSecret.Annotations["encoding"] = "gzip"
136+
137+
if gotSecret.Labels == nil {
138+
gotSecret.Labels = make(map[string]string)
139+
}
140+
gotSecret.Labels["app.kubernetes.io/managed-by"] = "terraform"
141+
gotSecret.Labels["tfstate"] = "true"
142+
gotSecret.Labels["tfstateSecretSuffix"] = k8sBackend.SecretSuffix
143+
gotSecret.Labels["tfstateWorkspace"] = "default"
144+
145+
gotSecret.Type = v1.SecretTypeOpaque
146+
}
147+
148+
if err := k8sClient.Get(ctx, client.ObjectKey{Name: "tfstate-default-" + k8sBackend.SecretSuffix, Namespace: k8sBackend.SecretNS}, &gotSecret); err != nil {
149+
if !kerrors.IsNotFound(err) {
150+
return err
151+
}
152+
// is not found, create the secret
153+
gotSecret = v1.Secret{
154+
ObjectMeta: metav1.ObjectMeta{
155+
Name: "tfstate-default-" + k8sBackend.SecretSuffix,
156+
Namespace: k8sBackend.SecretNS,
157+
},
158+
Type: v1.SecretTypeOpaque,
159+
Data: map[string][]byte{
160+
"tfstate": tfState,
161+
},
162+
}
163+
configureSecret()
164+
if err := k8sClient.Create(ctx, &gotSecret); err != nil {
165+
return err
166+
}
167+
} else {
168+
// update the secret
169+
configureSecret()
170+
gotSecret.Data["tfstate"] = tfState
171+
if err := k8sClient.Update(ctx, &gotSecret); err != nil {
172+
return err
173+
}
174+
}
175+
return nil
176+
}
177+
178+
func compressedTFState() ([]byte, error) {
179+
srcBytes, err := os.ReadFile(stateJSONPath)
180+
var buf bytes.Buffer
181+
writer := gzip.NewWriter(&buf)
182+
_, err = writer.Write(srcBytes)
183+
if err != nil {
184+
return nil, err
185+
}
186+
if err := writer.Close(); err != nil {
187+
return nil, err
188+
}
189+
return buf.Bytes(), nil
190+
}

0 commit comments

Comments
 (0)