Data backups
Beware - complicated language ahead!
This section is intended for advanced Unifier instance owners that require an automatic backup solution to prevent Unifier data loss. If you're only a bot moderator, (Discord) server moderator, or bot user, this is not for you, but you can still keep reading if you want to!
We'll try to simplify the language here, but we can't guarantee it'll be simple enough!
Unifier allows you to store off-site backups on remote servers to prevent data loss. All backups are encrypted in-transit and at-rest.
How backups are encrypted
Both in-transit and at-rest encryptions are mandatory for Unifier backups using official source code. If both conditions cannot be met, Unifier will refuse to back up data to the webserver.
In-transit encryption
When transferring data from one system to another, a connection needs to be made to the server at the configured endpoint. Once the data transfer starts, it is now in-transit, and Unifier and the webserver will encrypt this process using TLS.
Think of this as Unifier creating a box of data, then locking that box so nobody but the webserver can see its contents. While Unifier and everyone else accessing the webserver knows how to lock the box, the webserver knows how to unlock the box, but nobody else, including Unifier, knows how to unlock the box.
At-rest encryption
Once data arrives at the server, it is now at-rest. Although the in-transit phase is encrypted, the data received by the webserver will be decrypted, which will allow the webserver to access your configuration and bot data files. As we described in the box and key example above, the webserver can unlock the box of data, as it has the private key.
To mitigate this risk, Unifier encrypts the data using AES-256-CBC symmetric encryption before data transfer, so the webserver will always receive an encrypted version of the file only you can decrypt. This is basically putting a locked box (AES-256 encrypted) inside of a locked box (TLS), but this time while the webserver knows how to unlock the latter box, only Unifier knows how to lock and unlock the former box, as only it posesses the encryption and decryption keys.
How we implement at-rest encryption
Encryption and decryption keys for at-rest encryption is derived using a user-written password, a salt (this is set to 10 on the default config, but you may, and you should, change this to a different value), and an Initialization Vector (IV).
Each file is encrypted using different encryption keys created using a cryptographically random 16-byte IV, and the IV is stored encoded in base64 in plaintext on the webserver. Through this approach, no file will share the same cipher text even if the original data for both files are identical, as the encryption key for each file is different due to different IVs.
All encryption keys are stored on-memory only and never on-disk, and are generated only when needed for encryption and decryption. Keys are deleted from memory as soon as the encryption or decryption is complete.
Configuring the webserver
To allow Unifier to securely back up data, it will first need a backup server. To make a server, you will first need a copy of the Flask application used for backups, which you can download from its GitHub repository.
You will also need a TLS certificate, as Unifier mandates that all backup servers have in-transit encryption. To do this, you can either:
Use a webserver as a reverse proxy pointing to the Flask app, then create a certificate for the webserver site, or
Make Flask use existing TLS certificates by configuring
tls_context.
After TLS has been configured for your application, you can run the application file to start your Flask server.
Activating off-site backups
First, you will need to configure your encryption password, API key and endpoint, and backup frequency. Off-site backups are disabled by default, and you can always disable off-site backups at any time by setting the backup frequency (periodic_backup_cloud) to zero.
All secrets used by the bot must be stored in a .env file in the instance's root folder. To set your encryption password, you will need to enter it under ENCRYPTION_PASSWORD. Your .env file should look like this:
You will also need to configure a salt used to create your encryption key. This is 10 by default, but you may (and should) change this to a different value. This can be changed in config.json by changing the value of cloud_backup_salt.
Keep your encryption salt and password somewhere safe, as they will be needed to decrypt your backups.
Initialization Vectors cannot be configured and are generated randomly for each file only on file encryption. No IV is reused by Unifier to encrypt your file.
To configure your API endpoint, you will need to enter the subdomain to cloud_backup_endpoint in config.json. Do note that you should not enter any protocols (https://, http://, etc) or routes (/index, /api, etc), you only need the subdomain (such as backup.example.com).
Finally, you will need to configure the backup frequency. We recommend configuring this last, so backups do not run with an incomplete configuration. As mentioned above, you will need to change periodic_backup_cloud to change the frequency. We recommend you set this to 5 minutes (300) or higher, so you do not send too many backup requests while frequently backing up your data.
If your backup system has been configured correctly, backups should silently proceed in the background. Otherwise, a warning will be raised in the console. If you are unsure what the warning shown means, refer to the Troubleshooting section.
Restoring data
If you need to restore data, disable backups by setting backup frequency to 0. Keeping backups enabled will lead to your backup being overwritten, resulting in data loss.
To restore data onto a new instance, you will need to configure your backup endpoint, salt, and password but keep the backup frequency at 0, then run u!cloud to get backup information. Below the embed, you will find a "Restore" button, which will let you restore your config.json and data.json files.
Once the restore is completed successfully, you must reboot the bot for changes to take effect. Unifier will not make any further changes to data.json until it is rebooted.
Troubleshooting
Server returns code 401
Your Flask server has been properly configured but the API key is incorrect. Copy the API key generated when the Flask app was first started. If you do not have your API key, delete the keys.json file on your server and restart the application.
Server returns code 403
Your Flask server does not allow your IP address to make requests to the backup server. Add the IP address to allowed_ips and try again.
Server returns code 50x (500, 503, etc)
If your error code is 502: Your reverse proxy is working, but your Flask application is down, or the port is incorrect.
If your error code is anything else: View the console logs, and open an issue on the GitHub repository if needed.
Could not use TLS
Your webserver has an invalid or expired certificate, or does not use TLS entirely. Reconfigure your webserver to use a valid TLS certificate.
Last updated