Hooks in Amplify v.10.0.0 runs only once(first time when branch is connected) in CI/CD Pipeline. Locally - as expected. #11021

hackmajoris · 2022-09-17T15:09:38Z

Before opening, please confirm:

I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
I have searched for duplicate or closed issues.
I have read the guide for submitting bug reports.
I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
I have removed any sensitive information from my code snippets and submission.

How did you install the Amplify CLI?

remote config

If applicable, what version of Node.js are you using?

16.15.9

Amplify CLI Version

10.0.0

What operating system are you using?

Linux

Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.

Depends

Amplify Categories

hosting

Amplify Commands

push

Describe the bug

Starting with Amplify v.10.0.0, the hooks should be checked in to the source control.
I have a post-push.js hook which is needed to be triggered.
After connecting the branch in the CI/CD Pipeline in the Amplify Console for the first time, the hook is executed with success.
If another commit/build is triggered, the build runs without executing the hook.

Locally, every time the amplify push is executed, the hook is also triggered.

Expected behavior

The hook is triggered at every CI/CD build trigger.

Reproduction steps

Have a post-push.js in the amplify/hooks folder.
Connect a branch in CI/CD.
Wait for the first build.
Check if hook is executed(is should).
Push some change with a new commit to the branch.
Wait for the second build.
Check if the hook is executed.

GraphQL schema(s)

# Put schemas below this line
//NOT_RELEVANT

Project Identifier

a88954c5b3c47e78dce85b3e0fd819c8

Log output

First build

2022-09-17T14:10:58.046Z [INFO]: �[0mGraphQL transformer version: 2�[0m
2022-09-17T14:10:58.072Z [INFO]: 
2022-09-17T14:10:58.132Z [INFO]: ✅ Initialized your environment successfully.
2022-09-17T14:10:58.132Z [INFO]: �[32mYour project has been successfully initialized and connected to the cloud!�[39m
                                 �[32mSome next steps:�[39m
                                 �[0m"amplify status" will show you what you've added already and if it's locally configured or deployed�[0m
                                 �[0m"amplify add <category>" will allow you to add features like user login or a backend API�[0m
                                 �[0m"amplify push" will build all your local backend resources and provision it in the cloud�[0m
                                 �[0m"amplify console" to open the Amplify Console and view your project status�[0m
                                 �[0m"amplify publish" will build all your local backend and frontend resources (if you have hosting category added) and provision it in the cloud�[0m
                                 �[32mPro tip:�[39m
                                 �[0mTry "amplify add api" to create a backend API and then "amplify push" to deploy everything�[0m
2022-09-17T14:10:58.144Z [INFO]: 
2022-09-17T14:10:58.144Z [INFO]: ----- 🪝 post-push execution start -----
2022-09-17T14:10:58.939Z [INFO]: Running post push hooks scripts...
2022-09-17T14:10:59.055Z [INFO]: Setting cognito triggers...
2022-09-17T14:11:01.382Z [INFO]: Finished configuring triggers

Second Build

2022-09-17T14:36:59.793Z [INFO]: �[0mGraphQL transformer version: 2�[0m
2022-09-17T14:36:59.808Z [INFO]: 
2022-09-17T14:36:59.865Z [INFO]: ✅ Initialized your environment successfully.
2022-09-17T14:36:59.865Z [INFO]: �[32mYour project has been successfully initialized and connected to the cloud!�[39m
                                 �[32mSome next steps:�[39m
                                 �[0m"amplify status" will show you what you've added already and if it's locally configured or deployed�[0m
                                 �[0m"amplify add <category>" will allow you to add features like user login or a backend API�[0m
                                 �[0m"amplify push" will build all your local backend resources and provision it in the cloud�[0m
                                 �[0m"amplify console" to open the Amplify Console and view your project status�[0m
                                 �[0m"amplify publish" will build all your local backend and frontend resources (if you have hosting category added) and provision it in the cloud�[0m
                                 �[32mPro tip:�[39m
                                 �[0mTry "amplify add api" to create a backend API and then "amplify push" to deploy everything�[0m
                                 ⚠️ The maximum number of apps that you can create with Amplify in this region has likely been reached:
                                 For more information on Amplify Service Quotas, see:
                                 https://docs.aws.amazon.com/general/latest/gr/amplify.html#service-quotas-amplify
2022-09-17T14:37:02.291Z [INFO]: # Completed phase: build

Additional information

No workaround yet.

But, if you check the second build logs, there is that error:
"The maximum number of apps that you can create with Amplify in this region has likely been reached" related to aws-amplify/amplify-hosting#2797
Could be that it breaks the execution of the hooks.

Worth to mention that the script: https://github.com/aws-amplify/amplify-hosting/blob/main/scripts/amplifyPush.sh
is used for remote deployment. Maybe it should be updated also.

Update:
The issue is definitely in the https://github.com/aws-amplify/amplify-hosting/blob/main/scripts/amplifyPush.sh.
I did run the deployment without --simple flag, and at least it executes the hooks:

Update:
Dropping the --simple flag and have a branch name without special characters (see: #979), then the hooks are executed as expected, at each build, but it fails because it tries to re-create the stack.

Conclusion:

After the first build, the stack is pulled from the cache, and it doesn't have the hooks because they are not stack related.

The text was updated successfully, but these errors were encountered:

renanfferreira · 2022-09-19T19:49:11Z

I'm facing this problem also

ykethan · 2022-09-20T14:50:19Z

hey @hackmajoris, thank you for reaching out. I was able to replicate the behaviour in my Amplify application.

observed the amplify push does not get invoked when a resource has been updated. observed the following output at the end of the build.

marking this as a bug for further investigation.

removing amplifyPush --simple and utilizing amplifyPush works as expected.

hackmajoris · 2022-10-10T17:41:32Z

@ykethan any updates on this issue? Don't know how you prioritised it, but it's kind of a blocker, since if updating to version >=10.0.0 there is no any workaround to get the hooks working. If there is no any prioritisation then perhaps it's better to rollback the hooks refactorization done in v.10.

ykethan · 2022-10-10T18:21:16Z

hey @hackmajoris, thank you for checking in. Currently, the Amplify team is investigating into this behavior and will provide any updates on this issue here.
The current workaround is to remove the --simple flag to invoke hooks on each backend build.

hackmajoris · 2022-10-11T08:18:54Z

@ykethan thank you for your fast response. Just removing the --simple flag is not a solution. By removing the flag, a new environment is created at every commit. Indeed, the hooks works, but having a new environment for each commit doesn't bring any value.

If re-using the same name for the stack, as for the original issue description: "Dropping the --simple flag and have a branch name without special characters (see: #979), then the hooks are executed as expected, at each build, but it fails [after first build] because it tries to re-create the stack."

ykethan · 2022-10-18T20:53:43Z

Hey @hackmajoris, thank you for providing us the information, and great callout regarding the --simple behavior. The issue has been brought up as part of our team meetings and has been prioritized. Currently, the team is working on a fix for this here: #11179. Apologies for the inconvenience!

hackmajoris · 2022-10-28T10:28:34Z

Closing this issue, as the fix was done in #11179 and released with Amplify v10.3.2.
Good job!

hackmajoris · 2022-10-28T11:09:41Z

Unfortunately, I have to re-open it. After more testing, the hooks are still running only once - at the first build.

More findings.

How to reproduce the issue:
1 build:

  amplify.yml
    backend:
            build:
              commands:
                - ls amplify/hooks # it will show the files from git             
                - amplifyPush --simple # it will generate a new env at the first build. The amplify/hooks folder will not be replaced.          
                - ls amplify/hooks # the files are there

2 build:

  amplify.yml
    backend:
            build:
              commands:
                - ls amplify/hooks # it will show the files from git             
                - amplifyPush --simple # it will pull the existing env. The amplify/hooks folder WILL be replaced. Because they are not present on S3, the folder will be empty.         
                - ls amplify/hooks # the files are not there, the hooks are not executed.

aws-eddy · 2022-11-03T15:48:19Z

What version of Amplify are you running? @hackmajoris

aws-eddy · 2022-11-03T17:20:15Z

This seems to be fixed in 10.4.1 rc cc: @ykethan

hackmajoris · 2022-11-03T22:10:56Z

@evcodes this is a good news. I run on 9.2.1

aws-eddy · 2022-11-07T20:47:39Z

Can you please try this out with latest? 10.4.1

hackmajoris · 2022-11-11T23:50:51Z

@evcodes @ykethan Still no hooks run after first build on 10.4.1.

Info: amplifyPush --simple is used for CI

Using the command amplify push -y will run the hooks, but on Amplify Hosting that amplifyPush script does some aditional magic like checking/downloading existing environment. It looks that when it pulls the existing environment, the local hooks folder(from CI container) is cleared and it has nothing to run(as explained in above comment).

Proof:

List hooks dir before running amplifyPush --simple:

List hooks dir after running amplifyPush --simple:

Again, the changes which were done here: https://github.com/aws-amplify/amplify-cli/pull/11179/files#diff-e4044365569b0d6894bc87f33a263088f2ff5145bb90c64b8744595bdc129212 (PR closed in respect for hooks fix) are not in the 10.4.1 release:
https://github.com/aws-amplify/amplify-cli/blob/v10.4.1/packages/amplify-cli/src/attach-backend.ts

But they are on the dev branch: https://github.com/aws-amplify/amplify-cli/blob/dev/packages/amplify-cli/src/attach-backend.ts

Realease a new release with the changes made in the #11179 and let's test that one.

aws-eddy · 2022-11-17T18:23:45Z

Hey @hackmajoris, sorry for the delay. You are right. Here is the dif between 10.4.1 and 10.5

v10.4.1...v10.5.0#diff-e4044365569b0d6894bc87f33a263088f2ff5145bb90c64b8744595bdc129212L100-L123

Please try with 10.5.0 and this should solve the hooks issue.

hackmajoris · 2022-11-17T23:02:26Z

@evcodes Thanks for the updates.

I did test. It works with the 10.5.0!

One more thing:
If using AMPLIFY_DIFF_BACKEND: true at the Console Environment Variables, and if there aren't any backend changes, the hooks are not executed. Is it intended to have this behavior?

I'm asking because, locally, when running amplify push and if there aren't any backend changes which has to be pushed, the hooks still run.

aws-eddy · 2022-11-18T00:46:31Z

I believe that this is something for Hosting team to answer. I am going to close this issue as the original issue is resolved

aws-eddy · 2022-11-18T01:28:34Z

Just followed up with Hosting. This is in fact the expected behavior.

AMPLIFY_DIFF_BACKEND checks to see if there are any code changes - If there's no change since the previous commit, the build is skipped

hackmajoris changed the title ~~Hooks in Amplify v.10.0.0 runs only once(first time when branch is connected) in CI/CD Pipeline~~ Hooks in Amplify v.10.0.0 runs only once(first time when branch is connected) in CI/CD Pipeline. Locally - as expected. Sep 17, 2022

This was referenced Sep 17, 2022

post-push.js is not executing anymore aws-amplify/amplify-hosting#2981

Closed

fix: raise incorrect post event data #10954

Merged

josefaidt added platform Issues tied to the general CLI platform pending-triage Issue is pending triage labels Sep 19, 2022

ykethan added bug Something isn't working hosting Issues tied to hosting category p2 and removed pending-triage Issue is pending triage labels Sep 20, 2022

josefaidt added p1 and removed p2 labels Oct 10, 2022

hackmajoris mentioned this issue Oct 17, 2022

fix: keeps hooks intact when attaching backend #11179

Merged

6 tasks

hackmajoris mentioned this issue Oct 27, 2022

Fetching updates to backend environment: takes a lot of time aws-amplify/amplify-hosting#3055

Closed

hackmajoris closed this as completed Oct 28, 2022

hackmajoris reopened this Oct 28, 2022

swaminator closed this as completed Nov 4, 2022

swaminator reopened this Nov 4, 2022

aws-eddy closed this as completed Nov 18, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Hooks in Amplify v.10.0.0 runs only once(first time when branch is connected) in CI/CD Pipeline. Locally - as expected. #11021

Hooks in Amplify v.10.0.0 runs only once(first time when branch is connected) in CI/CD Pipeline. Locally - as expected. #11021

hackmajoris commented Sep 17, 2022 •

edited

Loading

First build

Second Build

renanfferreira commented Sep 19, 2022

ykethan commented Sep 20, 2022 •

edited

Loading

hackmajoris commented Oct 10, 2022 •

edited

Loading

ykethan commented Oct 10, 2022

hackmajoris commented Oct 11, 2022 •

edited

Loading

ykethan commented Oct 18, 2022

hackmajoris commented Oct 28, 2022

hackmajoris commented Oct 28, 2022 •

edited

Loading

aws-eddy commented Nov 3, 2022

aws-eddy commented Nov 3, 2022

hackmajoris commented Nov 3, 2022

aws-eddy commented Nov 7, 2022 •

edited

Loading

hackmajoris commented Nov 11, 2022 •

edited

Loading

aws-eddy commented Nov 17, 2022

hackmajoris commented Nov 17, 2022 •

edited

Loading

aws-eddy commented Nov 18, 2022

aws-eddy commented Nov 18, 2022

Hooks in Amplify v.10.0.0 runs only once(first time when branch is connected) in CI/CD Pipeline. Locally - as expected. #11021

Hooks in Amplify v.10.0.0 runs only once(first time when branch is connected) in CI/CD Pipeline. Locally - as expected. #11021

Comments

hackmajoris commented Sep 17, 2022 • edited Loading

Before opening, please confirm:

How did you install the Amplify CLI?

If applicable, what version of Node.js are you using?

Amplify CLI Version

What operating system are you using?

Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.

Amplify Categories

Amplify Commands

Describe the bug

Expected behavior

Reproduction steps

GraphQL schema(s)

Project Identifier

Log output

First build

Second Build

Additional information

renanfferreira commented Sep 19, 2022

ykethan commented Sep 20, 2022 • edited Loading

hackmajoris commented Oct 10, 2022 • edited Loading

ykethan commented Oct 10, 2022

hackmajoris commented Oct 11, 2022 • edited Loading

ykethan commented Oct 18, 2022

hackmajoris commented Oct 28, 2022

hackmajoris commented Oct 28, 2022 • edited Loading

aws-eddy commented Nov 3, 2022

aws-eddy commented Nov 3, 2022

hackmajoris commented Nov 3, 2022

aws-eddy commented Nov 7, 2022 • edited Loading

hackmajoris commented Nov 11, 2022 • edited Loading

aws-eddy commented Nov 17, 2022

hackmajoris commented Nov 17, 2022 • edited Loading

aws-eddy commented Nov 18, 2022

aws-eddy commented Nov 18, 2022

hackmajoris commented Sep 17, 2022 •

edited

Loading

ykethan commented Sep 20, 2022 •

edited

Loading

hackmajoris commented Oct 10, 2022 •

edited

Loading

hackmajoris commented Oct 11, 2022 •

edited

Loading

hackmajoris commented Oct 28, 2022 •

edited

Loading

aws-eddy commented Nov 7, 2022 •

edited

Loading

hackmajoris commented Nov 11, 2022 •

edited

Loading

hackmajoris commented Nov 17, 2022 •

edited

Loading