Image Removed

Image Added

📙 You will learn

How to backup internal ThreadFix databases running in Kubernetes.

Prerequisites

Audience: IT Professional, or End User
Difficulty: Intermediate
Time needed: Approximately 15 minutes
Tools required: If any

Backing Up and Restoring Database with Kubernetes

The following document details how to backup internal ThreadFix databases running in Kubernetes.

Vendor Links for External Database Backup and Restore

• MySQL: Dumping Data in SQL Format with mysqldump

• SQL Server: Create a Full Database Backup

• AWS RDS: Backing Up and Restoring an Amazon RDS DB Instance

• Azure: Automated backups - Azure SQL Database & SQL Managed Instance

Backing Up Internal Kubernetes Database

1. Locate the name of the ThreadFix database.

   Code Block
   TF_DB=$(kubectl get po -l app.kubernetes.io/component=database,app.kubernetes.io/part-of=threadfix -o jsonpath='{.items[].metadata.name}')

2. Create the database backup file.

   Code Block
   kubectl exec $TF_DB -- bash -c 'mysqldump -u$MYSQL_USER -p$MYSQL_PASSWORD --all-databases' > tf_db_dump.sql

3. Inspect the database dump to ensure the command completed successfully.
   

4. (Optional) gzip the database file.

   Code Block
   gzip tf_db_dump.sql

5. Copy the database dump to a secure location.

Restoring Internal Kubernetes Database

1. Get Helm instance name.

   Code Block
   helm ls

2. Get the ThreadFix repository name.

   Code Block
   helm search repo threadfix

   If installing from a local helm chart, the user’s local chart file should be used as the ThreadFix chart name.
   

3. Export Helm values (replace <tf_instance_name> with the name gathered from step 1).

   Code Block
   helm get values <tf_instance_name> > threadfix-values.yaml

4. Create db-only.yaml file with the following contents:

   Code Block
   appsec: enabled: false auth: enabled: false crud: enabled: false db: enabled: true defect: enabled: false importer: enabled: false processor: enabled: false provider: enabled: false ui: enabled: false websocket: enabled: false kong: enabled: false kafka: enabled: false zk: enabled: false

5. Restart ThreadFix in Database Only Mode (replace <tf_instance_name> with the values from step 1 and <tf_chart_name> with the value from step 2).

   Code Block
   helm upgrade <tf_instance_name> <tf_chart_name> --reuse-values -f db-only.yaml

6. Drop existing data.

   Code Block
   kubectl exec $TF_DB -- bash -c 'mysql -u$MYSQL_USER -p"$MYSQL_PASSWORD" -e "DROP DATABASE threadfix"' kubectl exec $TF_DB -- bash -c 'mysql -u$MYSQL_USER -p"$MYSQL_PASSWORD" -e "DROP DATABASE network"' kubectl exec $TF_DB -- bash -c 'mysql -u$MYSQL_USER -p"$MYSQL_PASSWORD" -e "DROP DATABASE provider"'

7. Locate the name of the ThreadFix database.

   Code Block
   TF_DB=$(kubectl get po -l app.kubernetes.io/component=database,app.kubernetes.io/part-of=threadfix -o jsonpath='{.items[].metadata.name}')

8. Restore data from the database dump file.

   Code Block
   kubectl exec $TF_DB -i -- bash -c 'cat /dev/stdin | mysql -u$MYSQL_USER -p"$MYSQL_PASSWORD"' < tf_db_dump.sql

9. Restart ThreadFix with previous settings (replace <tf_instance_name> with the values from step 1 and <tf_repo_name> with the value from step 2).

   Code Block
   helm upgrade --no-hooks <tf_instance_name> <tf_repo_name> -f threadfix-values.yaml

10. Follow the on screen instructions to access the application.

    • Use kubectl get po -w to watch the installation progress

Complete Database Wipe and Restore (Advanced)

In rare circumstances, it is not possible to restore a database as is, and the database must be completely wiped. The most common cause of this is InnoDB corruption or a corrupted disk. In such situations the user may need to completely destroy the database instance and start from scratch. Please only attempt this after all other options have been exhausted.

1. Get Helm instance name.

   Code Block
   helm ls

2. Get the ThreadFix repository name.

   Code Block
   helm search repo threadfix

3. Export Helm values (replace <tf_instance_name> with the name gathered from step 1).

   Code Block
   helm get values <tf_instance_name> > threadfix-values.yaml

4. Create db-only.yaml file with the following contents:

   Code Block
   appsec: enabled: false auth: enabled: false crud: enabled: false db: enabled: true defect: enabled: false importer: enabled: false processor: enabled: false provider: enabled: false ui: enabled: false websocket: enabled: false kong: enabled: false kafka: enabled: false zk: enabled: false

5. Find the database PersistentVolumeClaim.

   Code Block
   TF_DB_PVC=$(kubectl get pvc -l app.kubernetes.io/component=database,app.kubernetes.io/name=db,app.kubernetes.io/part-of=threadfix -o jsonpath='{.items[].metadata.name}')

6. Find the database StatefulSet.

   Code Block
   TF_DB_STS=$(kubectl get sts -l app.kubernetes.io/component=database,app.kubernetes.io/part-of=threadfix -o jsonpath='{.items[].metadata.name}')

7. Delete the database StatefulSet.

   Code Block
   kubectl delete sts $TF_DB_STS

8. Delete the database PersistentVolumeClaim.

   Code Block
   kubectl delete pvc $TF_DB_PVC

9. Restart ThreadFix in Database Only Mode (replace <tf_instance_name> with the values from step 1 and <tf_chart_name> with the value from step 2).

   Code Block
   helm upgrade <tf_instance_name> <tf_chart_name> --reuse-values -f db-only.yaml

10. Wait until the database is running (Use Ctrl-C to exit once the database status is running).

    Code Block
    kubectl get po -w -l app.kubernetes.io/component=database,app.kubernetes.io/part-of=threadfix

11. Locate the name of the ThreadFix database.

    Code Block
    TF_DB=$(kubectl get po -l app.kubernetes.io/component=database,app.kubernetes.io/part-of=threadfix -o jsonpath='{.items[].metadata.name}')

12. Restore data from the database dump file.

    Code Block
    kubectl exec $TF_DB -i -- bash -c 'cat /dev/stdin | mysql -u$MYSQL_USER -p"$MYSQL_PASSWORD"' < tf_db_dump.sql

13. Restart ThreadFix with previous settings (replace <tf_instance_name> with the values from step 1 and <tf_repo_name> with the value from step 2).

    Code Block
    helm upgrade --no-hooks <tf_instance_name> <tf_repo_name> -f threadfix-values.yaml

14. Follow the on screen instructions to access the application.

    • Use kubectl get po -w to watch the installation progress