- Created by Eric Cuevas on Oct 26, 2021
You are viewing an old version of this page. View the current version.
Compare with Current View Page History
Version 1 Next »
📙 You will learn
How to backup internal ThreadFix databases running in Kubernetes.
Prerequisites
Audience: IT Professional, or End User
Difficulty: Intermediate
Time needed: Approximately 15 minutes
Tools required: If any
Backing Up and Restoring Database with Kubernetes
The following document details how to backup internal ThreadFix databases running in Kubernetes.
To backup external databases please follow vendor instructions for creating database backups.
Vendor Links for External Database Backup and Restore
SQL Server: Create a Full Database Backup
Azure: Automated backups - Azure SQL Database & SQL Managed Instance
Backing Up Internal Kubernetes Database
The following should be performed on a bash compatible terminal with kubectl access to the Kubernetes cluster.
Locate the name of the ThreadFix database.
TF_DB=$(kubectl get po -l app.kubernetes.io/component=database,app.kubernetes.io/part-of=threadfix -o jsonpath='{.items[].metadata.name}')
Create the database backup file.
kubectl exec $TF_DB -- bash -c 'mysqldump -u$MYSQL_USER -p$MYSQL_PASSWORD --all-databases' > tf_db_dump.sql
Inspect the database dump to ensure the command completed successfully.
(Optional) gzip the database file.
gzip tf_db_dump.sql
Copy the database dump to a secure location.
Restoring Internal Kubernetes Database
Warning: This process is irreversible and will destroy all data in the ThreadFix instance.
The following should be performed on a bash compatible terminal with kubectl access to the Kubernetes cluster.
Get Helm instance name.
helm ls
Get the ThreadFix repository name.
helm search repo threadfix
If installing from a local helm chart, the user’s local chart file should be used as the ThreadFix chart name.
Export Helm values (replace
<tf_instance_name>
with the name gathered from step 1).helm get values <tf_instance_name> > threadfix-values.yaml
Create db-only.yaml file with the following contents:
appsec: enabled: false auth: enabled: false crud: enabled: false db: enabled: true defect: enabled: false importer: enabled: false processor: enabled: false provider: enabled: false ui: enabled: false websocket: enabled: false kong: enabled: false kafka: enabled: false zk: enabled: false
Restart ThreadFix in Database Only Mode (replace
<tf_instance_name>
with the values from step 1 and<tf_chart_name>
with the value from step 2).helm upgrade <tf_instance_name> <tf_chart_name> --reuse-values -f db-only.yaml
Drop existing data.
kubectl exec $TF_DB -- bash -c 'mysql -u$MYSQL_USER -p"$MYSQL_PASSWORD" -e "DROP DATABASE threadfix"' kubectl exec $TF_DB -- bash -c 'mysql -u$MYSQL_USER -p"$MYSQL_PASSWORD" -e "DROP DATABASE network"' kubectl exec $TF_DB -- bash -c 'mysql -u$MYSQL_USER -p"$MYSQL_PASSWORD" -e "DROP DATABASE provider"'
Locate the name of the ThreadFix database.
TF_DB=$(kubectl get po -l app.kubernetes.io/component=database,app.kubernetes.io/part-of=threadfix -o jsonpath='{.items[].metadata.name}')
Restore data from the database dump file.
kubectl exec $TF_DB -i -- bash -c 'cat /dev/stdin | mysql -u$MYSQL_USER -p"$MYSQL_PASSWORD"' < tf_db_dump.sql
Restart ThreadFix with previous settings (replace
<tf_instance_name>
with the values from step 1 and<tf_repo_name>
with the value from step 2).helm upgrade --no-hooks <tf_instance_name> <tf_repo_name> -f threadfix-values.yaml
Follow the on screen instructions to access the application.
Use
kubectl get po -w
to watch the installation progress
Complete Database Wipe and Restore (Advanced)
Warning: This process is irreversible and will destroy all data in the ThreadFix instance.
In rare circumstances, it is not possible to restore a database as is, and the database must be completely wiped. The most common cause of this is InnoDB corruption or a corrupted disk. In such situations the user may need to completely destroy the database instance and start from scratch. Please only attempt this after all other options have been exhausted.
The following should be performed on a bash compatible terminal with kubectl access to the Kubernetes cluster.
Get Helm instance name.
helm ls
Get the ThreadFix repository name.
helm search repo threadfix
Export Helm values (replace
<tf_instance_name>
with the name gathered from step 1).helm get values <tf_instance_name> > threadfix-values.yaml
Create db-only.yaml file with the following contents:
appsec: enabled: false auth: enabled: false crud: enabled: false db: enabled: true defect: enabled: false importer: enabled: false processor: enabled: false provider: enabled: false ui: enabled: false websocket: enabled: false kong: enabled: false kafka: enabled: false zk: enabled: false
Find the database PersistentVolumeClaim.
TF_DB_PVC=$(kubectl get pvc -l app.kubernetes.io/component=database,app.kubernetes.io/name=db,app.kubernetes.io/part-of=threadfix -o jsonpath='{.items[].metadata.name}')
Find the database StatefulSet.
TF_DB_STS=$(kubectl get sts -l app.kubernetes.io/component=database,app.kubernetes.io/part-of=threadfix -o jsonpath='{.items[].metadata.name}')
Delete the database StatefulSet.
kubectl delete sts $TF_DB_STS
Delete the database PersistentVolumeClaim.
kubectl delete pvc $TF_DB_PVC
Restart ThreadFix in Database Only Mode (replace
<tf_instance_name>
with the values from step 1 and<tf_chart_name>
with the value from step 2).helm upgrade <tf_instance_name> <tf_chart_name> --reuse-values -f db-only.yaml
Wait until the database is running (Use Ctrl-C to exit once the database status is running).
kubectl get po -w -l app.kubernetes.io/component=database,app.kubernetes.io/part-of=threadfix
Locate the name of the ThreadFix database.
TF_DB=$(kubectl get po -l app.kubernetes.io/component=database,app.kubernetes.io/part-of=threadfix -o jsonpath='{.items[].metadata.name}')
Restore data from the database dump file.
kubectl exec $TF_DB -i -- bash -c 'cat /dev/stdin | mysql -u$MYSQL_USER -p"$MYSQL_PASSWORD"' < tf_db_dump.sql
Restart ThreadFix with previous settings (replace
<tf_instance_name>
with the values from step 1 and<tf_repo_name>
with the value from step 2).helm upgrade --no-hooks <tf_instance_name> <tf_repo_name> -f threadfix-values.yaml
Follow the on screen instructions to access the application.
Use
kubectl get po -w
to watch the installation progress
Table of contents
- No labels