Python

Prerequisites

You have a project written in Python.
(Optional) The project has unit tests.
(Optional) Packaging is configured in the project: It's possible to build a package on your local machine.
If you want to publish packages to Space Packages, make sure the project has a Python package index.

Eligible images

An image with Python and the required tooling, for example pytest for running tests and twine for publishing packages. This example implies creating a custom image based on official Python images.

Build, run tests, and publish in a Python project

Currently, Automation does not provide any API for working with Python projects. So, the only way to build, test, and publish Python projects is to use the corresponding command-line tools. In our example, we will use pytest to run tests in the project, python to build a package, and twine to publish this package to Space packages. In order Automation could use these tools in build steps, we must prepare a custom Docker image that contains them. We will use Automation to prepare the image.

To prepare an image for your Python project

In Space, open the project's Packages page and create a new container registry. Let's suppose the URL of your registry is mycompany.registry.jetbrains.space/p/pythonProject/docker/python_custom_img.
On your local machine, make sure the environment is configured to work with the project: run tests, build packages, and so on.
Open the directory containing the project and record the list of required packages to requirements.txt by running:
pip freeze > requirements.txt
In the project root, create the Dockerfile file:
# We use Python 3 in our project FROM python:3 WORKDIR /usr/src/app COPY requirements.txt ./ RUN pip install --no-cache-dir -r requirements.txt
In the project root, create the .space.kts file:
job("Prepare Docker image") { // do not run on git push startOn { gitPush { enabled = false } } kaniko { build { file = "./Dockerfile" labels["vendor"] = "mycompany" } push("mycompany.registry.jetbrains.space/p/pythonProject/docker/python_custom_img") { tags{ +"0.0.1" } } } }
In Space, go to project's Job page and run the Prepare Docker image job.
Make sure the job is finished correctly and the image is uploaded to the project's container registry.

Now, you can create an Automation job that will do the rest: run tests, build a project package and publish it to Space Packages.

To run tests, build and publish a Python package

If your project doesn't have a Python package index, open the project's Packages page and create a new index. Let's suppose the URL of your index is https://packages.jetbrains.space/pypi/p/my-python-project/mypypi/simple.
Add a job to the project's .space.kts file:
job("Run tests, build, publish") { container(image = "mycompany.registry.jetbrains.space/p/pythonProject/docker/python_custom_img:0.0.1") { // specify URL of the package index using env var env["URL"] = "https://packages.jetbrains.space/pypi/p/my-python-project/mypypi/legacy" // We suppose that your project has default build configuration - // the built package is saved to the ./dist directory shellScript { content = """ echo Run tests... pytest ./tests/ echo Build package... python -m build echo Upload package... twine upload --repository-url ${'$'}URL -u ${'$'}JB_SPACE_CLIENT_ID -p ${'$'}JB_SPACE_CLIENT_SECRET dist/* """ } } }
Here $JB_SPACE_CLIENT_ID and $JB_SPACE_CLIENT_SECRET are environment variables that authenticate the Automation service in Space Packages.
Note that the built package will have the version specified in the setup.py or setup.cfg file. If you want to dynamically change the package version, you can do this by using the JB_SPACE_EXECUTION_NUMBER environment variable in the setup.py configuration file. Also check the Python official documentation about single-sourcing the package version.

Last modified: 15 December 2023