-
Notifications
You must be signed in to change notification settings - Fork 825
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RFC: Importing Existing AWS Resources to Amplify Project using the Amplify CLI #3977
Comments
How would this play out across environments? If I add a bucket Though I hopefully won't need it, I'm a big fan of this RFC. |
Will you be able to add resources in other regions or will it be limited as same data center as the amplify env? |
It would be amazing if we can use same cognito user pool for two different application like for example one app for back office management and other app for front office users. In this way security is not compromised or accidental release happen |
We are currently already using AWS Amplify with existing resources (e.g. Cognito User Pool) by adding manually created config files which are included by our CI/CD pipeline depending on the relevant stage. It works but I would really prefer a way which is natively supported by AWS Amplify CLI. Furthermore I would really love to include existing AppSync apis with support for Amplify DataStore even though this might be a bigger task. |
I’m not in favour of this feature, and I’d rather see this supported through solid documentation and exposing users to a bit of plumbing. Main reasons:
|
I am really happy that you guys are considering this feature 🔥. In short, my opinion is that this feature can increase the number of projects and developers which adopt AWS Amplify. My reasons are:
The CLI will be probably more complex. Also, the CLI will need to manage existing and new services at the same time. But it is a "downside" that it worth to assume. |
This would be perfect, as long as it's robust enough to never delete the existing Auth if there's a reference to it from another project. We're planning to add several applications around a single Cognito User Pool, which serves as SSO, and being able to attach the existing Auth is exactly what we need. I second the question how would this work across different envs? |
Yea, this will be very useful for me, and will ease the stress of manually setting the config file to use existing resources. 👍 |
Wonderful news!It took me quite some time in January to include an existing UserPool in a project in my company. The cloud is managed by terraform and setup manually. Every environment is in a different account and I do not have permissions to use I started with creating the UserPool Client for web and the identity pool in terraform, but soon realized that this was too much work to transform all the cloudformation templates to another format. With every Amplify update, I would also have to check if something changed, that would be a nightmare! Note:Before I could do anything with the Amplify CLI, I had to configure the CLI with an external AWS-Account where I had all permissions to. It turned out to be a good testing ground for experiments without committing the changes to git. This gave me the /amplify folder I needed to check into version control. My solution so far:
In build pipelines:
The Resulting ProjectThere are probably several steps I did not mention, but I ended up with Amplify CLI managing everything except the UserPool, including a new UserPoolClientForWeb and IdentityPool. Working with this kind of setupEvery change to the auth part is a huge pain! If I want to add IAM autentication to the AppSync API for example, I would need to try it out in another project and compare the CFN templates to see what changed. Amplify is under heavy development and is adding features and fixing bugs all the time. The generated templates change quite heavily sometimes, which is a good thing, they are auto generated, but for me I just want to use Amplify because I do not want to write CloudFormation or TerraForm! Create the initial Amplify App with CFN?I think if I would start over, I would explore to create the Amplify App with either a CloudFormation template, the aws-cli (not amplify-cli) or some other InfrastructureAsCode tool and see how to progress from there... Protect existing resourcesI'm concerned about having the CLI modify or accidentally delete the existing UserPool. Even changing the triggers might overwrite some existing trigger and cause some grieve for the developer responsible for the change. So maybe start with only adding resources like the UserPoolClient and IdentityPool and wire them to the other Amplify categories, but leave the UserPool alone? I think the people in charge wouldn't mind seeing UserPoolClients created for every new Amplify project as long as the existing parts are protected. Managing triggers by Amplify is a very good feature though! Export resourcesThere is also some additional complexity when Amplify also needs to export created resources. One scenario I'm facing is to give the AppSync API URL some pretty company sub-domain name. There is an existing TerraForm module that makes it easy, but I would need to figure out how to best fetch the URL and create a new TerraForm plan that is run after the Amplify deploy process. (The first plan only imports existing resources) TAGSAlso, please add the option to specify tags for the root stack, so that every resource created by the Amplify CLI gets them automatically applied! Please! Parting wordsAnyways, I got into quite some details, so you could hopefully understand one developers needs when it comes to integrating an Amplify App into an existing AWS infrastructure. So every progress in that area is highly anticipated by me! |
Hello guys! Had a mid size project on amplify pushing to dynamo DB tables. Had to play in my graphql schema to change connections between types and this introduced a secondary index error on a table upon doing amplify push... tried reverting back by pushing the schema I had prior to my connections modifications instead and now I have big rollback errors many tables in dynamo db. Hard to know what is the problem and impossible to debug. If you guys have other ways right now to revert changes made to my dynamo db to get past the errors on Amplify push I'm all ears! |
You likely have a GSIs created on dynamo tables from the first push failure. When a GSI change fails in DynamoDB, CloudFormation will not roll it back for you, it must be done manually. You need to use the API or console to remove the created GSIs so that you tables match your last-known good schema. Then be careful and make sure each push only changes 1 GSI per table going forward. This feature being discussed is not likely to be a good long-term solution to your situation. |
@vdumouchel I would suggest to always only experiment in a different environment, but make sure to only incrementally merge the changes to the main branch. Renaming trickOne 'trick' I use when I get into this situation and don't want to create a new environment is to simply rename all those I haven't tried the backup part yet, so I cannot help you with that. |
Hmmm... This is a dope idea, i will give it a try also |
I love this RFC, I'm sure I'll have additional thoughts, but the first two things that spring to mind are:
|
I want to collect emails with Hugo static website. |
Maybe there is a comment above that elaborates, but based on the heading I am worried "support for USING existing resources" (created by whatever process, including another Cloudformation stack) might get mixed in with "import existing resources" which implies leveraging Cloudformation's recent support for importing resources that were not created by a CF stack. If a resource was created by another CF stack, I don't want to modify that resource. Instead I think the CF templates and CLI interview should allow users to provide existing resources to be USED (not only IMPORTED). I made a feature request for "support for existing resources" over here. I think it's notable that for some cases (mine) it is currently supported! And I don't want it be be broken. :) |
Please add this please! Thank you, great idea |
This would be amazing for working with multiple apps within the same company. It would be great to add functions directed to an existing table and/or AppSync API. |
I believe I managed to solve as I described it in this thread: |
Also adding my support for this feature. Similar to @cyrfer my use case is around being able to use / link Amplify to existing resources for easy and seamless integration rather than importing them for control by the Amplify App. I'm most interested in Auth, allowing a Cognito instance to be used by multiple apps but only one in control of it (or perhaps not at all where managed externally). |
Apologies if I am duplicating advice written above. I have not had the chance to read through all of it. It would appear that creating an entirely new resource (Lambda function, for example) through the CLI alongside an already existing reference (i.e. an AppSync API) and executing Of course, you can go ahead and delete the Lambda or temporary resource after. Cheers 🍰 |
We have a largish (9000 lines of cloudformation for the appsync not including the lambdas that it calls) AppSync API that we'd like to be able to develop/debug locally. At the moment we have to deploy all changes to AWS before we can test them which means we have a slow feedback loop (especially since we do the deployments via a pipeline we've built, so it requires a git commit and a minute or two's wait plus the ~5 minutes for the AppSync CloudFormation to deploy). We'd love to be able to develop locally so if Amplify could import existing cloudformation templates for AppSync and Lambdas that would be great. A bit more info about our stack: we have a React web app and an AppSync API backed by lambdas and also some direct AppSync->DynamoDB access. The VTL in our AppSync API is quite complex (possibly too complex, but simplifying it would involve compromises/downsides). |
Various CLI components currently take dependency on this "shape" of the Cognito resource format. This might be a requirement that we'll drop in the future. Good call out. For now, you'll just need to add a second app client in order to conform to the Amplify requirements. |
@renebrandel Is the "imported" resource specific to each environment? I use a different Cognito User Pool for each of my environments. |
@renebrandel something I didn't see in the original RFC was extending Auth, API and Storage to use existing Lambdas. I really like AWS CDK and I want to create my Lambdas using CDK and Typescript. I want to be able to use these Lambdas (some are step functions, another feature amplify cli doesn't currently have) as resolvers for mutations or Auth Lambda triggers or DynamoDB triggers. |
I'm interested in making some modifications to an existing Amplify-managed Cognito User Pool (for example, enabling username case-insensitivity). Would it be possible for us to create a new self-managed Cognito User Pool, import users from current Amplify-managed Cognito, and then Edit: I'm aware of this solution to import using a user migration lambda trigger. Would that be the best solution? |
You’ll be asked to either import a different Cognito resource or maintain the same Cognito resource for your app’s auth category. If you want to have Amplify manage your auth resources in a new environment, run amplify remove auth to unlink the imported Cognito resource and amplify add auth to create new Amplify-managed auth resources in the new environment. https://docs.amplify.aws/cli/auth/import#multi-environment-support |
That's a good call out. I think we should look into that for next year. |
I haven't tried it myself but if you are able to transfer the users out and want to fully manage the Cognito instance yourself, you could rereference that new Cognito instance through |
Also, just to provide an update, we've now enabled the ability to import S3 and DynamoDB tables to your Amplify project: Read the blog post on how to import S3 buckets here: https://aws.amazon.com/blogs/mobile/use-an-existing-s3-bucket-for-your-amplify-project/ |
Hello everyone, |
What are they pre-requisites/criteria needed to be able to import
With
But, looking at my Identity Pool: ... anyone else in the same boat 🛶? ❌ ultimately, I'm trying to |
@renebrandel Worked like a charm ! thank you and to your team. in the mean time, are there any way of adding existing APIs from API gateway to Amplify? Manually? Been stuck at this for days. |
having same issue when trying to import Cognito User Pool and Identity Pool: |
I was able to solve this problem by adding the "Web App Client" and the "Native app Client" to the Identity Pool as shown in this page: https://aws.amazon.com/blogs/mobile/use-existing-cognito-resources-for-your-amplify-api-storage-and-more/ |
How soon are we looking for this? I'm looking to add Admin Queries to an existing auth I've set up. Or if anyone has a resource to a workaround so I can pull in a list of users in the current group that would be ideal. I assume a lambda function is the way to go for this |
Well, it's quite easy: amplify auth update You can add Admin Queries or Triggers there. |
@BeaveArony you cant use the amplify update auth to add the admin queries on an imported cognito user pool. That is the issue I am also trying to solve. |
With API imports (hopefully) on the distant horizon, what is the current recommended way to add an existing AWS REST API to amplifyconfiguration.json (referring to this guide) so that everyone else in the team automatically has this config too? My current solution is splitting the amplifyconfig string after |
There is a lot of confusion in conflicting information for a person newly introduced to Cognito and Amplify. I'd like to bring in Auth to my existing React application, however, it is challenging to determine which is the correct path to follow. I believe most beneficial would be good documentation that outlines not only what needs to be done, but where and why. I'd love to see a schema for the auth section - it still confuses me. I think I understand what to call, but sometimes things appear in my sign up/sign in dialogs that I don't know how they got there (e.g. Sign in with AWS). There are multitudes of times where I cannot get the federated signin working. And the libraries are so exacting, but the documentation leaves some things up to interpretation and a bit of hope. I spent days trying to understand why something wasn't working and it turns out that I needed a trailing slash at the end of the URL, associated with the redirect query param. I was surprised to see my formatted signup dialog appear when I added that slash. A reference implementation would go hand-in-hand with the documentation. If you could deliver those two things, it would be a huge win for this library! Things change so often. It took me a while to realize aws-amplify-react was the old library, even though there was plenty of information about if during the past few years. I'm not sure how you factor in ensuring newer projects are started with the latest and greatest, but that would also be helpful. I don't want to use the amplify push/pull/update cli framework. I want to use my own, but leverage the great content in the associated libraries. |
Any progress on using Amplify CLI to import existing REST API ECS endpoints? |
Is there a way to increase the visibility/priority of this Amplify CLI issue. |
I tried your way, add awsAPIPlugin (called ie. 'peopleAPI') but, i have no idea how configure react to recognize that API? amplify api push/update doesnt work. |
We have some existing lambdas built using CDK, and we would love to import them into our amplify project, and then have amplify build an API Gateway around them so we can use the amplify authentication to call those lambdas built with CDK. |
Would love to see this added. Also would like to see the amplify CLI support creation of python 3.10 lambdas instead of just python 3.8 |
Have an existing Lambda I manually build and would love to import it directly into Amplify. |
Want this. |
Currently, the CLI provisions new AWS resources based on the categories that you add to your Amplify project.
This RFC is to gauge community interest and hear more thoughts around the CLI being able to import existing resources into an Amplify project.
As an MVP, we’re planning to support importing of resources in the following categories:
We’ll be relying heavily on the new CloudFormation import functionality - https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/resource-import-existing-stack.html for this feature
Auth
Storage
API
GraphQL API
REST API
Please comment on this thread if you have some thoughts or suggestions on this feature or if you think we’re missing any story points which you would love to see as a part of this feature.
The text was updated successfully, but these errors were encountered: