How to Upload Files to DigitalOcean Spaces with Python

Sometimes I need to upload something to DigitalOcean Spaces. Here is my snippet on how to upload files to DigitalOcean Spaces with boto3 and python. First you'll need to install dependencies.

pip install boto3

Upload local file

From boto3 library we'll need upload_file method. For some content type magic we'll use mimetypes. More on this later. Create a file, let's say digitalocean.py.

import boto3
import mimetypes


def get_spaces_client(**kwargs):
    """
    :param kwargs:
    :return:
    """
    region_name = kwargs.get("region_name")
    endpoint_url = kwargs.get("endpoint_url")
    key_id = kwargs.get("key_id")
    secret_access_key = kwargs.get("secret_access_key")

    session = boto3.session.Session()

    return session.client(
        's3',
        region_name=region_name,
        endpoint_url=endpoint_url,
        aws_access_key_id=key_id,
        aws_secret_access_key=secret_access_key
    )


def upload_file_to_space(spaces_client, space_name, file_src, save_as, **kwargs):
    """
    :param spaces_client: Your DigitalOcean Spaces client from get_spaces_client()
    :param space_name: Unique name of your space. Can be found at your digitalocean panel
    :param file_src: File location on your disk
    :param save_as: Where to save your file in the space
    :param kwargs
    :return:
    """

    is_public = kwargs.get("is_public", False)
    content_type = kwargs.get("content_type")
    meta = kwargs.get("meta")

    if not content_type:
        file_type_guess = mimetypes.guess_type(file_src)

        if not file_type_guess[0]:
            raise Exception("We can't identify content type. Please specify directly via content_type arg.")

        content_type = file_type_guess[0]

    extra_args = {
        'ACL': "public-read" if is_public else "private",
        'ContentType': content_type
    }

    if isinstance(meta, dict):
        extra_args["Metadata"] = meta

    return spaces_client.upload_file(
        file_src,
        space_name,
        save_as,

        # boto3.s3.transfer.S3Transfer.ALLOWED_UPLOAD_ARGS
        ExtraArgs=extra_args
    )

Now, when you want to upload file from your local disk to digitalocean spaces simply import your python module.

import digitalocean


client = digitalocean.get_spaces_client(
    region_name="nyc3",
    endpoint_url="https://nyc3.digitaloceanspaces.com",
    key_id="my-spaces-key",
    secret_access_key="my-spaces-secret-access-key"
)

digitalocean.upload_file_to_space(client, "space-name", "local/my_photo.jpg", "place/here/my_photo.jpg")

How to Set DigitalOcean Spaces File Permissions to Public?

By default, file will be uploaded in private mode. Which means, you can't share this file url with others. If you want to change it, then use is_public=True.

In boto3 this is implemented via ACL property, but Spaces only support "public-read" and "private" values. That's why i reduced configuration to is_public argument.

digitalocean.upload_file_to_space(
    client,
    "space-name",
    "local/my_photo.jpg",
    "place/here/my_photo.jpg",
    is_public=True
)

Uploaded files content type. binary/octet-stream error

If you uploaded files with boto3 before, then you might've stumbled upon a mime-type issue. I mean, "binary/octet-stream" or "octet/binary" mime-types in S3 or DigitalOcean Spaces. I resolved this problem with mimetypes.guess_type, but if you want to specify content type directly, then use content_type argument.

digitalocean.upload_file_to_space(
    client,
    "space-name",
    "local/my_styles.css",
    "place/my_styles.css",
    content_type="text/css"
)

Set File Metadata in upload_file_to_space

digitalocean.upload_file_to_space(
    client,
    "space-name",
    "local/my_styles.css",
    "space/path/my_styles.css",
    meta={
      "x-amz-meta-my-key": "your-value"
    }
)

Upload Byte Array File with put_object

First, add this function to digitalocean.py module.

def upload_bytes_array_to_space(spaces_client, space_name, file_body, save_as, **kwargs):
    """
    :param spaces_client: Your DigitalOcean Spaces client from get_spaces_client()
    :param space_name: Unique name of your space. Can be found at your digitalocean panel
    :param file_body: Byte Array File
    :param save_as: Where to save your file in the space
    :param kwargs:
    :return:
    """

    is_public = kwargs.get("is_public", False)
    content_type = kwargs.get("content_type")
    meta = kwargs.get("meta")

    args = {
        "Bucket": space_name,
        "Body": file_body,
        "Key": save_as,
        "ACL": "public-read" if is_public else "private"
    }

    if content_type:
        args["ContentType"] = content_type

    if isinstance(meta, dict):
        args["Metadata"] = meta

    return spaces_client.put_object(**args)

Use it in your module as described previously. Open your file in binary mode and send it to space. I'm not doing here any mime type detection, so it's up to you to set it directly via content_type.

with open("scripts/test/test.jpg", "rb") as my_file:
    digitalocean.upload_bytes_array_to_space(
        client,
        "space-name",
        my_file,
        "space/path/test6.jpg",
        is_public=True
    )

How to Set File Metadata?

If you want to pass additional metadata to your file, simply add meta key to upload_bytes_array_to_space.

with open("scripts/test/test.jpg", "rb") as my_file:
    digitalocean.upload_bytes_array_to_space(
        client,
        "space-name",
        my_file,
        "space/path/test6.jpg",
        is_public=True,
        meta={
            "x-amz-meta-my-key": "your-value"
        }
    )

How to Create Spaces key_id and secret_access_key?

Visit official documentation from DigitalOcean.