--- 
canonical: 'https://mwop.net/blog/2024-04-06-aws-s3-sync-folder.html'
title: 'Addressing AWS S3 Sync Folder Issues'
author: "[Matthew Weier O'Phinney](https://mwop.net)"
created: '2024-04-06T08:37:35-05:00'
updated: '2024-04-08T07:51:35-05:00'
tags:
  - aws
  - s3

---
I have used S3 and S3-compatible storage for a long time now, and have used both s3cmd and the AWS CLI tooling to sync, either between buckets or with a local filesystem.
It generally "just works".

Except that when it doesn't, it's really hard to debug.





### The problem

The issue I had recently was that I was creating a "folder" in a bucket for namespacing some specific files.
In my case, I was doing this on DigitalOcean Spaces, but over the course of trying to isolate the issue, I also tried on AWS S3, and had the same issue.

What I did was create a folder in the bucket _in the web interface_.
This seems reasonable; both services have a button for this and allow you to do that and start adding files to that folder.

I was then configuring a container based on [elementar/s3-volume](https://github.com/elementar/docker-s3-volume) to provide a persistent volume for my application, and it was going to sync to this folder.

But I kept getting a failure, saying it couldn't download the file due to a cross-device link.
And doing a web search on that was not giving me any help.

I tried digging into debug logs, and just kept coming up with an exception that also gave no helpful results when searching.

Until I came across a random comment in a random issue on the AWS CLI tooling where somebody noted that their sync only failed when they created a folder _via the web interface_.
They discovered that doing this created a 0 byte, inaccessible virtual object in the bucket, and that the AWS CLI tooling was unable to sync this object, leading to the error.

### The Solution

So the solution was that I needed to:

1. Delete the folder via the web interface.
2. Locally create the folder _with a file under it_.
3. Sync the folder _and file_ to the bucket:

   ```bash
   aws s3 sync .placeholder s3://s3-bucket-name/destination_folder/.placeholder
   ```

This approach creates the folder with the file, and prevents creation of that virtual object, which then allows syncing to happen normally.