Table of contents
Sometimes I need to upload something to DigitalOcean Spaces. Here is my snippet on how to upload files to DigitalOcean Spaces with boto3 and python. First you'll need to install dependencies.
pip install boto3
From boto3 library we'll need upload_file method. For some content type magic we'll use mimetypes. More on this later. Create a file, let's say digitalocean.py.
import boto3
import mimetypes
def get_spaces_client(**kwargs):
"""
:param kwargs:
:return:
"""
region_name = kwargs.get("region_name")
endpoint_url = kwargs.get("endpoint_url")
key_id = kwargs.get("key_id")
secret_access_key = kwargs.get("secret_access_key")
session = boto3.session.Session()
return session.client(
's3',
region_name=region_name,
endpoint_url=endpoint_url,
aws_access_key_id=key_id,
aws_secret_access_key=secret_access_key
)
def upload_file_to_space(spaces_client, space_name, file_src, save_as, **kwargs):
"""
:param spaces_client: Your DigitalOcean Spaces client from get_spaces_client()
:param space_name: Unique name of your space. Can be found at your digitalocean panel
:param file_src: File location on your disk
:param save_as: Where to save your file in the space
:param kwargs
:return:
"""
is_public = kwargs.get("is_public", False)
content_type = kwargs.get("content_type")
meta = kwargs.get("meta")
if not content_type:
file_type_guess = mimetypes.guess_type(file_src)
if not file_type_guess[0]:
raise Exception("We can't identify content type. Please specify directly via content_type arg.")
content_type = file_type_guess[0]
extra_args = {
'ACL': "public-read" if is_public else "private",
'ContentType': content_type
}
if isinstance(meta, dict):
extra_args["Metadata"] = meta
return spaces_client.upload_file(
file_src,
space_name,
save_as,
# boto3.s3.transfer.S3Transfer.ALLOWED_UPLOAD_ARGS
ExtraArgs=extra_args
)
Now, when you want to upload file from your local disk to digitalocean spaces simply import your python module.
import digitalocean
client = digitalocean.get_spaces_client(
region_name="nyc3",
endpoint_url="https://nyc3.digitaloceanspaces.com",
key_id="my-spaces-key",
secret_access_key="my-spaces-secret-access-key"
)
digitalocean.upload_file_to_space(client, "space-name", "local/my_photo.jpg", "place/here/my_photo.jpg")
By default, file will be uploaded in private mode. Which means, you can't share this file url with others. If you want to change it, then use is_public=True.
In boto3 this is implemented via ACL property, but Spaces only support "public-read" and "private" values. That's why i reduced configuration to is_public argument.
digitalocean.upload_file_to_space(
client,
"space-name",
"local/my_photo.jpg",
"place/here/my_photo.jpg",
is_public=True
)
If you uploaded files with boto3 before, then you might've stumbled upon a mime-type issue. I mean, "binary/octet-stream" or "octet/binary" mime-types in S3 or DigitalOcean Spaces. I resolved this problem with mimetypes.guess_type, but if you want to specify content type directly, then use content_type argument.
digitalocean.upload_file_to_space(
client,
"space-name",
"local/my_styles.css",
"place/my_styles.css",
content_type="text/css"
)
digitalocean.upload_file_to_space(
client,
"space-name",
"local/my_styles.css",
"space/path/my_styles.css",
meta={
"x-amz-meta-my-key": "your-value"
}
)
Universal code-snippet to know when HTML is ready for modern browsers.
First, add this function to digitalocean.py module.
def upload_bytes_array_to_space(spaces_client, space_name, file_body, save_as, **kwargs):
"""
:param spaces_client: Your DigitalOcean Spaces client from get_spaces_client()
:param space_name: Unique name of your space. Can be found at your digitalocean panel
:param file_body: Byte Array File
:param save_as: Where to save your file in the space
:param kwargs:
:return:
"""
is_public = kwargs.get("is_public", False)
content_type = kwargs.get("content_type")
meta = kwargs.get("meta")
args = {
"Bucket": space_name,
"Body": file_body,
"Key": save_as,
"ACL": "public-read" if is_public else "private"
}
if content_type:
args["ContentType"] = content_type
if isinstance(meta, dict):
args["Metadata"] = meta
return spaces_client.put_object(**args)
Use it in your module as described previously. Open your file in binary mode and send it to space. I'm not doing here any mime type detection, so it's up to you to set it directly via content_type.
with open("scripts/test/test.jpg", "rb") as my_file:
digitalocean.upload_bytes_array_to_space(
client,
"space-name",
my_file,
"space/path/test6.jpg",
is_public=True
)
If you want to pass additional metadata to your file, simply add meta key to upload_bytes_array_to_space.
with open("scripts/test/test.jpg", "rb") as my_file:
digitalocean.upload_bytes_array_to_space(
client,
"space-name",
my_file,
"space/path/test6.jpg",
is_public=True,
meta={
"x-amz-meta-my-key": "your-value"
}
)
Visit official documentation from DigitalOcean.