Skip to content

Commit

Permalink
Fix cf invalidation (#24)
Browse files Browse the repository at this point in the history
* Fixed the bug with cloudfront invalidation paths

* Adding flag to delete stale files from s3
  • Loading branch information
sanchitbapat authored Nov 24, 2022
1 parent 63f1bc8 commit ce39dfc
Show file tree
Hide file tree
Showing 2 changed files with 30 additions and 9 deletions.
9 changes: 5 additions & 4 deletions push-to-s3-and-invalidate-cloudfront/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Files are pushed to the S3 bucket depending on the user inputs supplied when the
Following examples describe how to use the action.

## Uploading to S3 based on user input
This example will **upload** the specified folder (`build-directory`) to a folder named `blog` on the S3 bucket(`myS3Bucket`). The action will not exclude any files while uploading them. The action will also invalidate the cloudfront cache. The action will attempt a max of `5` (based on `aws-retry-attempts` value) retries for any AWS operations before erroring out.
This example will **upload** the specified folder (`build-directory`) to a folder named `blog` on the S3 bucket(`myS3Bucket`). The action will not exclude any files while uploading them. The action will also invalidate the cloudfront cache. The action will attempt a max of `5` (based on `aws-retry-attempts` value) retries for any AWS operations before erroring out. This example will also delete files from S3 that do not exist in the source.

```yaml
- uses: XanaduAI/cloud-actions/push-to-s3-and-invalidate-cloudfront@main
Expand All @@ -19,10 +19,11 @@ This example will **upload** the specified folder (`build-directory`) to a folde
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
s3-bucket: myS3Bucket
s3-action: "upload"
s3-dir-to-upload-to: "blog"
s3-directory: "blog"
s3-files-to-exclude: ""
invalidate-cloudfront-cache: "true"
aws-retry-attempts: 5
s3-delete-stale-files: "true"
```
## Deleting from S3 based on user input
Expand All @@ -40,13 +41,13 @@ This example will **delete** the specified folder (`build-directory`) from the S
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
s3-bucket: myS3Bucket
s3-action: "delete"
s3-dir-to-delete-from: "blog"
s3-directory: "blog"
invalidate-cloudfront-cache: "true"
aws-retry-attempts: 5
```

## Uploading/Deleting to/from S3 with minimal user unput
If this action is called when a `pull_request` is `opened` or `syncronized`, then the example will **upload** the specified folder (`build-directory`) to a folder named `pr-previews/PR-123` on the S3 bucket (`myS3Bucket`). The action will exclude any files that start with the default `pr-previews` prefix. The action will not invalidate the cloudfront cache.
If this action is called when a `pull_request` is `opened` or `syncronized`, then the example will **upload** the specified folder (`build-directory`) to a folder named `pr-previews/PR-123` on the S3 bucket (`myS3Bucket`). The action will exclude any files that start with the default `pr-previews` prefix. The action will not invalidate the cloudfront cache. This example will not delete files from S3 that do not exist in the source.

If this action is called when a `pull_request` is `closed`, then the example will **delete** the folder named `pr-previews/PR-123` on the S3 bucket (`myS3Bucket`).
The action will attempt a max of `2` (default) retries for any AWS operations before erroring out.
Expand Down
30 changes: 25 additions & 5 deletions push-to-s3-and-invalidate-cloudfront/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,12 @@ inputs:
Refer to https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-retries.html for the usage of retry retry attempts
required: false
default: "2"
s3-delete-stale-files:
description: |
A flag to indicate if the aws sync command should be run with the `--delete` flag. This will delete the files that exist in the destination but not in the source.
More information on this can be found at https://docs.aws.amazon.com/cli/latest/reference/s3/sync.html
required: false
default: "false"

outputs:
dir_name:
Expand All @@ -64,6 +70,9 @@ outputs:
files_to_exlude:
description: Files to exclude from S3 when using the sync command.
value: ${{ steps.exclude.files_to_exclude }}
delete_stale_files:
description: Compute if the aws s3 sync command should run with the `--delete` flag based on user input.
value: ${{ steps.stale.delete_stale_files }}

runs:
using: composite
Expand All @@ -81,12 +90,12 @@ runs:
run: |
if [[ "${{ inputs.s3-directory }}" == "pr-previews" ]]
then
dir_name="${{ inputs.s3-directory }}/PR-${{ inputs.pull-request-number }}"
dir_name="${{ inputs.s3-directory }}/PR-${{ inputs.pull-request-number }}/"
elif [[ "${{ inputs.s3-directory }}" == "/" ]]
then
dir_name=""
else
dir_name="${{ inputs.s3-directory }}"
dir_name="${{ inputs.s3-directory }}/"
fi
echo "dir_name=$dir_name" >> $GITHUB_OUTPUT
Expand All @@ -108,6 +117,17 @@ runs:
files_to_exlude="--exclude \"${{ inputs.s3-files-to-exclude }}\""
fi
echo "files_to_exlude=$files_to_exlude" >> $GITHUB_OUTPUT
- name: Compute if stale files should be deleted
shell: bash
id: stale
run: |
delete_stale_files=""
if [[ "${{ inputs.s3-delete-stale-files }}" == "true" ]]
then
delete_stale_files="--delete"
fi
echo "delete_stale_files=$delete_stale_files" >> $GITHUB_OUTPUT
- name: Upload the files to S3
shell: bash
Expand All @@ -118,7 +138,7 @@ runs:
AWS_SECRET_ACCESS_KEY: ${{ inputs.aws-secret-access-key }}
AWS_MAX_ATTEMPTS: ${{ inputs.aws-retry-attempts }}
run: |
aws s3 sync ${{ inputs.build-directory }}/ s3://${{ inputs.s3-bucket }}/${{ steps.directory.outputs.dir_name }} ${{ steps.exclude.outputs.files_to_exlude }}
aws s3 sync ${{ inputs.build-directory }}/ s3://${{ inputs.s3-bucket }}/${{ steps.directory.outputs.dir_name }} ${{ steps.exclude.outputs.files_to_exlude }} ${{ steps.stale.outputs.delete_stale_files }}
- name: Delete the files on S3
shell: bash
Expand All @@ -128,7 +148,7 @@ runs:
AWS_ACCESS_KEY_ID: ${{ inputs.aws-access-key-id }}
AWS_SECRET_ACCESS_KEY: ${{ inputs.aws-secret-access-key }}
AWS_MAX_ATTEMPTS: ${{ inputs.aws-retry-attempts }}
run: aws s3 rm s3://${{ inputs.s3-bucket }}/${{ steps.directory.outputs.dir_name }}/ --recursive ${{ steps.exclude.outputs.files_to_exlude }}
run: aws s3 rm s3://${{ inputs.s3-bucket }}/${{ steps.directory.outputs.dir_name }} --recursive ${{ steps.exclude.outputs.files_to_exlude }}

- name: Invalidate the cloudfront cache
shell: bash
Expand All @@ -139,4 +159,4 @@ runs:
AWS_SECRET_ACCESS_KEY: ${{ inputs.aws-secret-access-key }}
AWS_MAX_ATTEMPTS: ${{ inputs.aws-retry-attempts }}
run: |
aws cloudfront create-invalidation --distribution-id ${{ inputs.aws-cloudfront-distribution-id }} --paths "/${{ steps.directory.outputs.dir_name }}/*"
aws cloudfront create-invalidation --distribution-id ${{ inputs.aws-cloudfront-distribution-id }} --paths "/${{ steps.directory.outputs.dir_name }}*"

0 comments on commit ce39dfc

Please sign in to comment.