Integration testing of Appsody Serverless function in the Context of Kabanero 0.3.0.

[stephenkinder]

We need to add additional Appsody tests to ensure that the Serverless deploy capability properly behaves in the context of Kabanero 0.3.0.

This integration test is a kick-the-tires test to make sure that an Appsody application that intended to be deployed as serverless, can be wired through a github webhook to our build and deploy pipeline, hosted in a kabanero instance.

The application should be verified to be able to scale to zero, and scale up when a request hits the route.

Note: The appsody operator team should already have test projects that have been verified in Kubernetes, the intention of this integration is to ensure that the same behavior correctly works in a Kabanero deployment in OCP 4.2.

[marikaj123]

Dan is looking after automation.
Vijai run the pipeline
Arthur needs a Servless project to run the Appsody pipeline and delpoy properly

[marikaj123]

@dacleyra - Is the work for the issue on track to complete by Nov-15?

[dacleyra]

@kaczyns @stephenkinder

Looking for an opinion here

Tekton drives their tests with a manual pipelinerun using bash shell scripting
https://github.com/tektoncd/dashboard/tree/master/test

Appsody drives their tests using go
https://github.com/appsody/appsody-operator/tree/master/test/e2e

What I believe we need to do to achive an e2e is

1. Create an appsody init application of each collection type, and commit them each to a repo in the kabanero-io org
2. Set the appsody deploy yaml for each application to serverless
3. Decide whether to script in bash shell script, or write the test in go
4. Execute the tests using manual pipelinerun

A webhook run test I don't think currently exists for Tekton, answer pending.

[kaczyns]

Just my opinion here... I don't think that either approach would be bad. The things I have been thinking about are:

1. Is the test going to install Kabanero to a cluster as well. If so, it will probably want to leverage the install shell script, and so having the tests themselves being scripted in bash would not be too much additional work.
2. Writing in go lets us leverage the built-in testing framework which the kab operator already wants to use. I don't know what frameworks exist in bash, I'm sure there is something, but go feels like a more natural fit. But I think to be able to do this, the test has to be provided with a cluster that already has Kabanero operator and its prerequisites installed on it.
3. The clients and go mappings of the various CRDs would make it easier to query/scrape things like a pipeline run to see how it is progressing. I don't know how easy it is to extract debug information using Go though (I am thinking that if there is a failure, or maybe even if there is not, we're going to want to get container logs and things like that and save them with the test results). that process is pretty straightforward in bash.

[kvijai82]

Just wanted to mention that @smcclem started working on a script to manually drive all pipelines for all the active collections and validate the app deployed. Now this is not serverless, but I guess we could modify the app-deploy.yaml to make it.

@StringCheeseCR was looking into picking this up and figuring out the infrastructure to drive this and run this as our daily integration test.

[dacleyra]

Serverless deployment is blocked by
appsody/appsody-operator#183

Some progress has been delivered
kabanero-io/kabanero-operator#295

Can continue to make progress on adding the tests for remaining stacks, but attempting to test for app state, and endpoint will fail.

[dacleyra]

Test completion blocked by

Impacts java- collections
kabanero-io/collections#174

Impacts nodejs-loopback collection
kabanero-io/collections#177

[marikaj123]

According to Daniel: About 90% done with writing the automation/tests. Blocked by the collection issues, 174 and 177, with the fix to test the app endpoint.
The fix kabanero-io/collections#174 should go into the next collection v0.3.0-RCx
kabanero-io/collections#177 needs to fix the Loopback collection.

[dacleyra]

code is thought to be complete, but the blockering bugs for test completion still need to be resolved

[dacleyra]

Impacted by
kabanero-io/kabanero-pipelines#126

[marikaj123]

DANIEL CLEYRAT:
the tests don't pass because of the other issues that cause the tests to fail - they are noted in the issue

[dacleyra]

Tests complete successfully with 0.3.0-rc.5