Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move test resources to new bucket #548

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 19 additions & 2 deletions .github/workflows/build-main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,13 @@ on:
# Run every Monday at 18:00:00 UTC (Monday at 10:00:00 PST)
- cron: "0 18 * * 1"

env:
BUCKET_NAME : "bioio-dev-test-resources"
AWS_REGION : "us-west-2"
permissions:
id-token: write # This is required for requesting the JWT
contents: read # This is required for actions/checkout

jobs:
test-core-lib:
runs-on: ${{ matrix.os }}
Expand All @@ -27,6 +34,11 @@ jobs:
- uses: actions/checkout@v4
with:
submodules: true
- uses: aws-actions/configure-aws-credentials@v3
with:
role-to-assume: arn:aws:iam::978220035532:role/bioio_github
role-session-name: bioio-aicsimageio-${{ github.sha }}
aws-region: ${{ env.AWS_REGION }}
- name: Setup Python
uses: actions/setup-python@v4
with:
Expand All @@ -35,7 +47,7 @@ jobs:
run: |
python -m pip install --upgrade pip
pip install .[test]
- uses: actions/cache@v4
- uses: actions/cache@v3
id: cache
with:
path: aicsimageio/tests/resources
Expand Down Expand Up @@ -76,6 +88,11 @@ jobs:
- uses: actions/checkout@v4
with:
submodules: true
- uses: aws-actions/configure-aws-credentials@v3
with:
role-to-assume: arn:aws:iam::978220035532:role/bioio_github
role-session-name: bioio-aicsimageio-${{ github.sha }}
aws-region: ${{ env.AWS_REGION }}
- name: Setup Python
uses: actions/setup-python@v4
with:
Expand All @@ -88,7 +105,7 @@ jobs:
run: |
python -m pip install --upgrade pip
pip install .[test]
- uses: actions/cache@v4
- uses: actions/cache@v3
id: cache
with:
path: aicsimageio/tests/resources
Expand Down
16 changes: 16 additions & 0 deletions .github/workflows/test-and-lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,12 @@ on: pull_request
concurrency:
group: ${{ github.head_ref }}
cancel-in-progress: true
env:
BUCKET_NAME : "bioio-dev-test-resources"
AWS_REGION : "us-west-2"
permissions:
id-token: write # This is required for requesting the JWT
contents: read # This is required for actions/checkout

jobs:
test-core-lib:
Expand All @@ -23,6 +29,11 @@ jobs:
- uses: actions/checkout@v2
with:
submodules: true
- uses: aws-actions/configure-aws-credentials@v3
with:
role-to-assume: arn:aws:iam::978220035532:role/bioio_github
role-session-name: bioio-aicsimageio-${{ github.sha }}
aws-region: ${{ env.AWS_REGION }}
- name: Setup Python
uses: actions/setup-python@v2
with:
Expand Down Expand Up @@ -72,6 +83,11 @@ jobs:
- uses: actions/checkout@v2
with:
submodules: true
- uses: aws-actions/configure-aws-credentials@v3
with:
role-to-assume: arn:aws:iam::978220035532:role/bioio_github
role-session-name: bioio-aicsimageio-${{ github.sha }}
aws-region: ${{ env.AWS_REGION }}
- name: Setup Python
uses: actions/setup-python@v2
with:
Expand Down
20 changes: 13 additions & 7 deletions .github/workflows/test-upstreams.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,13 @@ on:
# Run every Mon,Wed,Thurs at 19:00:00 UTC (Monday at 11:00:00 PST)
- cron: "0 19 * * 1,3,4"

env:
BUCKET_NAME : "bioio-dev-test-resources"
AWS_REGION : "us-west-2"
permissions:
id-token: write # This is required for requesting the JWT
contents: read # This is required for actions/checkout

jobs:
test-core-lib:
runs-on: ${{ matrix.os }}
Expand All @@ -30,21 +37,20 @@ jobs:
- uses: actions/checkout@v4
with:
submodules: true
- uses: aws-actions/configure-aws-credentials@v3
with:
role-to-assume: arn:aws:iam::978220035532:role/bioio_github
role-session-name: bioio-aicsimageio-${{ github.sha }}
aws-region: ${{ env.AWS_REGION }}
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v3
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-west-2
- name: Install Dependencies
run: |
python -m pip install --upgrade pip
pip install .[test]
- uses: actions/cache@v4
- uses: actions/cache@v3
id: cache
with:
path: aicsimageio/tests/resources
Expand Down
6 changes: 4 additions & 2 deletions aicsimageio/tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,12 @@
REMOTE = "REMOTE"

LOCAL_RESOURCES_DIR = Path(__file__).parent / "resources"
REMOTE_RESOURCES_DIR = "s3://aics-modeling-packages-test-resources/aicsimageio/test_resources/resources" # noqa: E501
REMOTE_RESOURCES_DIR = (
"s3://bioio-dev-test-resources/aicsimageio/test_resources/resources" # noqa: E501
)

LOCAL_RESOURCES_WRITE_DIR = Path(__file__).parent / "writer_products"
REMOTE_RESOURCES_WRITER_DIR = "s3://aics-modeling-packages-test-resources/fake/dir"
REMOTE_RESOURCES_WRITER_DIR = "s3://bioio-dev-test-resources/fake/dir"


def get_resource_full_path(filename: str, host: str) -> Union[str, Path]:
Expand Down
2 changes: 1 addition & 1 deletion scripts/TEST_RESOURCES_HASH.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
4e3100e86a7fb0a6d1d6f38d1bd04b98165dbcea2323a8130bf16525a532d65d
1e7fef4fba5979e91927fd2ba6f0f830b7cdc97f8ad0e37c256204c8722e3319
2 changes: 1 addition & 1 deletion scripts/download_test_resources.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ def download_test_resources(args: Args):
# Get quilt package
package = Package.browse(
"aicsimageio/test_resources",
"s3://aics-modeling-packages-test-resources",
"s3://bioio-dev-test-resources",
top_hash=top_hash,
)

Expand Down
2 changes: 1 addition & 1 deletion scripts/upload_test_resources.py
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ def upload_test_resources(args: Args):
if confirmation:
pushed = package.push(
package_name,
"s3://aics-modeling-packages-test-resources",
"s3://bioio-dev-test-resources",
message=f"Test resources for `aicsimageio` version: {__version__}.",
)

Expand Down
Loading