I have used S3 and S3-compatible storage for a long time now, and have used both s3cmd and the AWS CLI tooling to sync, either between buckets or with a local filesystem. It generally "just works".

Except that when it doesn't, it's really hard to debug.

The problem

The issue I had recently was that I was creating a "folder" in a bucket for namespacing some specific files. In my case, I was doing this on DigitalOcean Spaces, but over the course of trying to isolate the issue, I also tried on AWS S3, and had the same issue.

What I did was create a folder in the bucket in the web interface. This seems reasonable; both services have a button for this and allow you to do that and start adding files to that folder.

I was then configuring a container based on elementar/s3-volume to provide a persistent volume for my application, and it was going to sync to this folder.

But I kept getting a failure, saying it couldn't download the file due to a cross-device link. And doing a web search on that was not giving me any help.

I tried digging into debug logs, and just kept coming up with an exception that also gave no helpful results when searching.

Until I came across a random comment in a random issue on the AWS CLI tooling where somebody noted that their sync only failed when they created a folder via the web interface. They discovered that doing this created a 0 byte, inaccessible virtual object in the bucket, and that the AWS CLI tooling was unable to sync this object, leading to the error.

The Solution

So the solution was that I needed to:

1. Delete the folder via the web interface.

2. Locally create the folder with a file under it.

3. Sync the folder and file to the bucket:

   aws s3 sync .placeholder s3://s3-bucket-name/destination_folder/.placeholder
   

This approach creates the folder with the file, and prevents creation of that virtual object, which then allows syncing to happen normally.