Skip to main content

Documentation Portal

Using Azure Blob Storage

Create your storage location using Azure Blob Provider
Prerequisite
  • You have Full Admin access on a azure subscription, allowing you to create resources needed for creating a Azure Blob Storage account.

  • You know how Azure is working and have basic knowledge to navigate and use Azure services.

Login to Azure Portal

Login to Azure (https://portal.azure.com)

  1. after login, you shall be arriving at the dashboard page, that could look like this:

    azuredashboard.png
Deploy a custom template
  1. Deploy a custom template

    Search for “deploy” in the microsoft search bar, and select “Deploy a custom template” service

    azuredeploy1.png
  2. Scroll down and search for the quick start template called “storage-blob-container”

    azuredeploy2.png
    1. Click on “Select template”

      azuredeploy3.png
    2. On the Custom Deployment screen, fill in the following information

      - Resource group: select a existing one, or create a new one (Take Note for later use)

      - Region: select the region where the files need to be saved in. This is best as close as possible to your TrendMiner installation to avoid any delays. 

      - Storage Account Name: specify a self generated name, that you can easily find back later (Take Note for later use)

      - Container Name: Suggest you are using the name of the server you are migrating here. But this field is also open to pick a generic name. (Take Note for later use)

      If done, click Review + Create

      azuredeploy4.png

      On the last screen (review changes), Click “Create”

    3. Wait until all actions are done

      azuredeploy5.png
Get your Subscription ID and Access Keys
  1. Click open the ‘Deployments details’ and find back the Storage Account. 

    Click the link left from “Storage account” Type

    In this case “storageacctm01”

    getaccesskeys.png
  2. On the overview page, make note of the “Subscription ID”

    getsubscriptionid.png
  3. Open Security + Network menu

    In the menu at the left of the screen, click on “Security + Networking” and click on “Access Keys”

    accesskeysdetails.png

    Retrieve your Access Key by clicking on Show button next to the “Key” Field.

    Note

    If you plan on rotating your access keys, do not forget to update the TrendMiner Installation with the new access key

Information you need to save from this procedure

The information from this procedure you need to save to be able to configure backup/restore in TrendMiner.

  • Subscription ID

  • Resource group

  • Storage account

  • Container name

  • access key

Configure your Azure Blob Storage provider in Edge Manager
Prerequisite
  • you have created the Azure Blob Storage Account according to our procedure.

  • You are running a TrendMiner version later then or equal to R2

Login
  1. Login to TrendMiner EdgeManager 

    https://url/edge or http://url:8080 (in case no valid certificate is installed)

    em-config-awss3-storage-login.png

    Click “Sign In”

  2. Navigate to “BACKUP AND RESTORE”->Remote

    backuprestore-menu

    Click on Full

    em-config-awss3-storage-backupandrestore.png

    Click on Storage Settings

Configure the Storage Provider

In this procedure we use Azure Blob Storage, after selecting this, fill in all information required

You should have captured the data from the procedure on how to create a Azure Blob Storage account, except the backup Path.

em-config-azure-storage-configscreen.png

Click Save

Note

  • You can choose a backup path yourself. E.G. name of the server that your are backing up.

  • One storage account, can be used for multiple TrendMiner Installations, however, it is advised to use at least another backup path to be able to use the same storage account.

    • There are 4 reasons for this recommendation:

      • It simplifies backup management for each environment.

      • It ensures that backups for each environment remain separate, minimising the risk of accidentally restoring from one environment with backups from another.

      • It streamlines the process of deleting backups specific to an environment.

      • It prevents scheduled backups from interfering with each other if they are executed simultaneously.

  • When restoring to a new server, make sure to use the same backup path on the destination server 

Click Save

Validate the Configuration

Wait until storage location is validated

em-config-awss3-storage-valavailable.png

you can click on Storage Settings tab, to refresh the page

The status should say Available before continuing with the next steps.

em-config-awss3-storage-valpending.png

In case the status is as following

em-config-awss3-storage-valfailed.png

Tip

In case something is going wrong (Status stays on Unavailable), these are some tips you can try to fix

  • make sure the date and time are correct on the server. Also NTP server need to be in place. Not setting it, will result in authentication failure for the backend tooling used to setup the remote storage

  • Doublecheck all parameters provided.

  • Validate you can properly communicate to Azure Blob from the TrendMiner Host.

  • In case all parameters are properly set, and it still doesn't work, please try to use a Azure Blob client (E.g. FilezillePro https://filezillapro.com/)  that you can run on your Laptop/Desktop to try to access the Storage Account, to fully validate the parameters.

  • In case all above is not helping, please contact TrendMiner Support.

Create backup on the old server

Important

It is advised to inform your users and/or run backup creations during off peek hours as it could result in performance degradation. Optionally the scheduling functionality can be used for more flexible backup creation.

  1. Login to TrendMiner Edge Manager

    em-config-awss3-storage-login.png

    Click “Sign In”

  2. Navigate to “BACKUP AND RESTORE” -> Remote

    em-config-awss3-storage-gotomenu.png
  3. Click on “Create new backup now”

    createbackup_1.png

    A notification popup in the bottom right corner will shown that a backup creation has started.

    The backup will also be shown in the backup overview table:

    createbackup_backupCreated.png

    Important

    Creating a backup will take some time, especially the initial backup, as this will be a full backup. Any backup taken after the initial backup will be increments of the previous backup and will be more risk free related to potential performance degradations (depending on the data differences)

    As the full backup takes long, it makes sense to execute that a week before the real migration is happening. 

    Full manual backups time out after 48 hours. Successive incremental backups time out after 4 hours. To prepare for the OS migration make sure to manually trigger the first backup to ensure it does not time out.

    Right before the migration it is advised to do a incremental backup as this is way faster, and will save all differences from the last backup.

    Depending on the size of the backup and the network upload speed the time it takes can differ. A simple calculator to calculate the upload time can be found here: Free Download/Upload Time Calculator | Site24x7 Tools.

    Backup creation and upload time can vary based on the following factors:

    • The available upload speed (bandwidth) which is often shared with other servers on the network.

    • Disk performance.

    • Other overhead on the network like meta data traffic from the Storage Provider.

  4. Wait until the backup creation has completed (changing from 'in progress' to 'completed')

    createbackup-inprogress.png
    createbackup-backupfinished.png

    Click the refresh button to update the latest status (status does not auto refresh).

    refreshicon.png

    When the backup creation has finished the entry in the table will also show how much time the creation took.

    In case something went wrong an error “Partially Failed” will be shown. In case this happens, please reach out to TrendMiner Support.

Restore backup on the new server

For the OS migration scenario we assume the new server is a clean installation and no user data is present on the server yet. Despite no user data can get lost we still advise to take a VM snapshot of the clean server installation for easy recovery in case something should go wrong during the backup restore.

Important

It is required to shutdown the old server just before you start your restore on the news server, to avoid any issues with the backup/restore functionality. Its also adviced to keep this server shutdown.

  1. Configure and validate remote storage settings. To access the remote backup(s) from the old server first configure your remote storage location, exactly the same as it was configured on the old server (AWS S3 or Azure Blob Storage).

    The storage configuration page should look similar to this one, and the validation status should be “Available”.

    restore-remotestorageconfig.png

    When the configuration of the remote storage location is successful the remote backups should show up.

    restore-backupandrestorescreen.png
  2. To initiate the restore, click on the 3 dots on the line of the backup you want to restore and then click on 'Restore'.

    Note

    Make sure you choose the latest backup you have created on the old server.

    restore-clickrestore.png

    A popup window will show, warning you about the impact of this restore.

    restore-confirmrestore.png

    Click on 'Restore' to start the restore process.

    Danger

    The restore will wipe any existing data and configurations present on the server and restore the data and config from the selected backup.

  3. A banner will be shown on the top of the Edge Manager page to indicate that a restore is in progress.

    restore-restoreinprogress.png

    Warning

    From this point on do not interact anymore with the TrendMiner server until the restore process has finished.

    The restore procedure will download all the files from the configured remote storage and restore its setup and configuration. Depending on the size of the backup and the network download speed the time it takes can differ. A simple calculator to calculate the download time can be found here: Free Download/Upload Time Calculator | Site24x7 Tools.

    Part of the restore process is also the booting of all TrendMiner services which on its own will take some time (around 15-30 minutes depending on the performance of the server).

    Note

    In case of doubt about the progress of the restore, please contact TrendMiner support before rebooting or interacting with the server to avoid breaking the system.

    After the restore process has finished (yellow banner gone) some processes (playbooks) are still running to boot up and configure all TrendMiner services. Wait until these playbooks have finished (status 'success') as shown in the 'Recent actions' on the right side in Edge Manager.

  4. Validate the successful restore.

    Once the Restore completes you will see all the services on the services screen are up and running again, as in the screenshot below:

    appliance_services_2024_R2_0.png

    Validate that TrendMiner is working as expected. If all is working fine, you can trigger a new backup on the new server if you might need that for rollback scenario's later. After this steps the old centos backups shall not be used anymore. 

Reconfigure SAML/SSO (optional)

In case you had configured a SAML IDP on your old server and you likely need to update the configuration (metadata.xml) again in your IDP (Entra AD/OKTA).

It should be sufficient to download the metadata.xml file again from ConfigHub and upload it again in your SAML IDP as documented here: SAML/SSO