← Back to TIL

Moving files up one folder level in GCP buckets

gcp

If you have a bucket with a hierarchy like:

  • bucket
    • folder
      • subfolder
        • 1.txt
        • 2.txt

And you run gcloud storage mv 'gs://bucket/folder/subfolder/*' 'gs://bucket/folder/' you will get the error:

ERROR: (gcloud.storage.mv) Destination URL gs://bucket/folder/subfolder/ already exists.

This is because the wildcard * matches both the contents of subfolder and also subfolder itself. This isn't clear in the documentation of how wildcards work, but is at least partially due to the fact that buckets don't really have "folders", just blobs that have slashes in the filenames.

The docs however do point to a workaround. By using the ? single character match token, you can ensure that only the contents of the folder are matched:

gcloud storage mv 'gs://bucket/folder/subfolder/?*' 'gs://bucket/folder/'

Should work as expected.