Object storage

Toolbox Enterprise offers you the flexibility to choose between object storage options: S3-compatible or Azure Blob Storage. The selected storage type depends on the value of the tbe.storage.type property in your application.yaml: s3 or azure.

        tbe:
            storage:
                type: s3
    

AWS S3

To configure the AWS S3 object storage, follow the example and add the necessary configuration to your application.yaml:

            tbe:
                storage:
                    type: s3
                minio:
                    url: "https://s3.${tbe.minio.s3-region}.amazonaws.com"
                    s3-region: eu-west-1
                    bucket: "<BUCKET NAME>"
                    accessKey: "<AWS_ACCESS_KEY_ID>"
                    secretKey: "<AWS_SECRET_ACCESS_KEY>"
        

S3 Automatic configuration

You can configure access to your AWS S3 object storage by implicitly inferring the credentials. Before start, make sure your S3 bucket is up and running.

In your application.yaml, do the following:
1. Set the value of the useS3AutoConfiguration boolean option to true.
2. Remove the accessKey, region, and secretKey properties. If Toolbox Enterprise detects explicitly declared credentials, it will use them over autoconfiguration.
The example of the configuration is as follows:
tbe: minio: useS3AutoConfiguration: true bucket: "<BUCKET_NAME>"
In your AWS Management Console (or alternative tool of your choice), create an IAM role with the following permissions:
s3:GetBucketLocation, s3:ListBucket, s3:GetObject, s3:DeleteObject, s3:PutObject
Deploy the server with the IAM role. Once the server is up, you will be able to execute any bucket-related actions with no need to provide the password explicitly.

For more details, see the description of configuration parameters.

Google Cloud Storage

To configure Google Cloud Storage, you need to create HMAC Keys as outlined in the Google Cloud documentation.

Before creating HMAC keys, ensure that your service account has the following roles assigned:

Storage Object Creator
Storage Object Viewer
Storage Transfer Agent
Storage Transfer User
Storage Transfer Viewer

Update your application.yaml following the example:

            tbe:
                storage:
                    type: s3
                minio:
                    bucket: "<BUCKET NAME>"
                    url: "https://storage.googleapis.com"
                    access-key: "<HMAC_ACCESS_KEY>"
                    secret-key: "<HMAC_SECRET_KEY>"

        

Azure Blob Storage

You can use one of the two methods to configure connection to Azure in your application.yaml:

Providing a connection string
tbe: storage: type: azure azure: connection-string: "DefaultEndpointsProtocol=http;AccountName=<ACCOUNT_NAME>;AccountKey=<ACCOUNT_KEY>;BlobEndpoint=http://127.0.0.1:10000/<ACCOUNT_NAME>;" container: "<CONTAINER_NAME>"
Providing raw credentials
tbe: storage: type: azure azure: container: "<CONTAINER_NAME>" account-key: "<ACCOUNT_KEY>" account-name: "<ACCOUNT_NAME>" endpoint: "http://127.0.0.1:10000/<ACCOUNT_NAME>"
For more details, see the description of configuration parameters.

Last modified: 16 April 2024