Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: Handler does not properly fail request if the canary's status has a 'StateReason' #18

Open
tylerburdsall opened this issue Sep 1, 2020 · 0 comments

Comments

@tylerburdsall
Copy link

During stabilization, the handler needs to be checking if StateReason is being returned from the request. The presence of this value indicates that there was an error and that the canary did not successfully update as intended. Currently, the handler is still treating this as a successful update. This can cause CloudFormation to register a bad deployment as a successful update.

As an example, here is a canary that it not in a stable state. This should be causing CloudFormation updates to fail, but it is not:

% aws synthetics get-canary --name <REDACTED>
{
    "Canary": {
        "Id": <REDACTED>,
        "Name": <REDACTED>,
        "Code": {
            "SourceLocationArn": <REDACTED>,
            "Handler": <REDACTED>
        },
        "ExecutionRoleArn": <REDACTED>,
        "Schedule": {
            "Expression": "rate(5 minutes)",
            "DurationInSeconds": 0
        },
        "RunConfig": {
            "TimeoutInSeconds": 300,
            "MemoryInMB": 960
        },
        "SuccessRetentionPeriodInDays": 31,
        "FailureRetentionPeriodInDays": 31,
        "Status": {
            "State": "RUNNING",
            "StateReason": "Error occurred while DescribeSecurityGroups. EC2 Error Code: InvalidGroup.NotFound. EC2 Error Message: The security group <REDACTED> does not exist (Service: AWSLambda; Status Code: 400; Error Code: InvalidParameterValueException; Request ID :<REDACTED>; Proxy: null)"
        },
        "Timeline": {
            "Created": 1593634958.304,
            "LastModified": 1598909334.057,
            "LastStarted": 1598909344.0
        },
        "ArtifactS3Location": <REDACTED>,
        "EngineArn": <REDACTED>,
        "RuntimeVersion": "syn-1.0",
        "Tags": {}
    }
}

It looks like the model JSON also needs to be updated to be able to support this, see here where it only has State.

When performing the Update (and most likely in the Create handler as well), the handler needs to check that there is no StateReason returned before considering the request successful, see here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant