Object storage

IDE Services offers you the flexibility to choose between object storage options: S3-compatible or Azure Blob Storage. The selected storage type depends on the value of the storage.type property in your server configuration file: s3 or azure.

AWS

To configure the AWS S3 object storage, follow the example and add the necessary configuration to your application.yaml:

            tbe:
                storage:
                    type: s3
                minio:
                    url: "https://s3.${tbe.minio.s3-region}.amazonaws.com"
                    s3-region: eu-west-1
                    bucket: "<BUCKET NAME>"
                    accessKey: "<AWS_ACCESS_KEY_ID>"
                    secretKey: "<AWS_SECRET_ACCESS_KEY>"
        

S3 Automatic configuration

You can configure access to your AWS S3 object storage by implicitly inferring the credentials. Before start, make sure your S3 bucket is up and running.

In your application.yaml, do the following:
1. Set the value of the use-s3-auto-configuration parameter to true.
2. Remove the url, access-key, region, and secret-key parameters. If IDE Services detects explicitly declared credentials, it will use them over autoconfiguration.
The example of the configuration is as follows:
tbe: minio: use-s3-auto-configuration: true bucket: "<BUCKET_NAME>"
In your AWS Management Console (or an alternative tool of your choice), create an IAM role with the following permissions:
s3:GetBucketLocation, s3:ListBucket, s3:GetObject, s3:DeleteObject, s3:PutObject
Deploy the server with the IAM role. Once the server is up, you will be able to execute any bucket-related actions with no need to provide the password explicitly.

For more details, see the description of configuration parameters.

Google Cloud Storage

To configure Google Cloud Storage, you need to create HMAC Keys as outlined in the Google Cloud documentation.

Before creating HMAC keys, ensure that your service account has the following roles assigned:

Storage Object Creator
Storage Object Viewer
Storage Transfer Agent
Storage Transfer User
Storage Transfer Viewer

Update your application.yaml following the example:

            tbe:
                storage:
                    type: s3
                minio:
                    bucket: "<BUCKET NAME>"
                    url: "https://storage.googleapis.com"
                    access-key: "<HMAC_ACCESS_KEY>"
                    secret-key: "<HMAC_SECRET_KEY>"

        

Azure Blob Storage

You can use one of the two methods to configure connection to Azure in your application.yaml:

Providing a connection string
tbe: storage: type: azure azure: connection-string: "DefaultEndpointsProtocol=http;AccountName=<ACCOUNT_NAME>;AccountKey=<ACCOUNT_KEY>;BlobEndpoint=http://127.0.0.1:10000/<ACCOUNT_NAME>;" container: "<CONTAINER_NAME>"
Providing raw credentials
tbe: storage: type: azure azure: container: "<CONTAINER_NAME>" account-key: "<ACCOUNT_KEY>" account-name: "<ACCOUNT_NAME>" endpoint: "http://127.0.0.1:10000/<ACCOUNT_NAME>"
For more details, see the description of configuration parameters.

What is stored in object storage

See what data IDE Services products may store in S3-compatible or Azure Blob storage.

IDE Services Core and IDE Provisioner	Binaries of plugins uploaded to a private plugin repository, if such a repository has been set up for your IDE Services instance. JSON-formatted plugin download logs that show what machine (userAgent) downloaded which plugin and when, as illustrated in this example: { "events" : [ { "pluginId" : "org.jetbrains.toolbox-enterprise-client", "pluginName" : "Toolbox Enterprise", "pluginVersion" : "2024.3.0.1622-ij233", "timestamp" : "2024-03-20T00:00:00Z", "userAgent" : "Toolbox/2.2.3.20090 (Mac OS X)" } ] } JSON files with raw user data uploaded through the API during a batch user import, stored for audit purposes. Logs that record the history of significant entity changes and their authors, including modifications to plugins, profiles, users, groups, and automation tokens. IDE and plugin binaries along with their metadata files, can be stored if the administrator chooses so for the offline mode.
Code With Me Enterprise	Personalized installers.
AI Enterprise	AI plugin
License Vault	Doesn't store any data.

Last modified: 20 September 2024