Integrates CI Pipeline With the Public Repo (#76)

This change integrates the CI pipeline introduced in e2220e5 to the public repo.

To achieve this, the CI rig in the aforementioned commit has been updated with the following changes:

    * Use GitHub secret token to validate all requests coming to CI rig through webhook
    * Encrypt GitHub secret token at rest.  Decrypt secret token when initializing server to use for request validation
    * Requests must come from GitHub IPs
    * If request comes from OpenNetVM repo, then the user must be in a whitelist
    * Adds audit logs to identify unauthorized requests

Commit log:

* Add security checks, extend CI to public repo

* Added authorized users

* Authorized user log

* Add secret encryption

* Add logging

* Better logging, optimized encryption

* Fix unused var

ci/.gitignore

@@ -1,4 +1,9 @@
+access_log
 config
+webhook-config.json
+encrypted_secret.bin
+private.pem
+public.pem
 githubcreds
 mykeyfile
 repository

ci/README.md

@@ -3,12 +3,12 @@
 ### Setting up CI
 Run a Flask server that listens for new events from github, will get triggered when a new PR is created or when keyword `@onvm` is mentioned.  
 ```sh
-python3 webhook-receiver.py 0.0.0.0 8080 @onvm
+python3 webhook-receiver.py 0.0.0.0 8080 @onvm webhook-config.json
 ```
 
 To run CI tests manually, requires a config file, the github PR ID, request message and a response message.  
 ```sh
-./manager.sh <config file> <pr ID> <request msg>
+./manager.sh <config file> <pr ID> <repo name> <request msg>
 ```  
 
 ### Usage
@@ -34,15 +34,23 @@ The CI process can be broken into multiple steps:
     WORKER_LIST=("WORKER_1_IP WORKER_1_KEY", "WORKER_2_IP WORKER_2_KEY", ...)
     GITHUB_CREDS=path_to_creditential_file
     REPO_OWNER="OWNER_STRING"
-    REPO_NAME="NAME_STRING"
     ```
 
     Config file example:
     ```
     WORKER_LIST=("nimbnode42 nn42_key")
     GITHUB_CREDS=githubcreds
     REPO_OWNER="sdnfv"
-    REPO_NAME="openNetVM-dev"
+    ```
+
+    Webhook json config example
+    ```
+    {
+        "secret-file": "very_special_encrypted_secret_file.bin",
+        "private-key-file": "private_key.pem", 
+        "log-successful-attempts": true,
+        "authorized-users": ["puffin", "penguin", "pcoach"]
+    }
     ```
 
     GITHUB_CREDS file example:

ci/ci_busy.sh

@@ -29,43 +29,45 @@ fi
 
 if [[ -z "$3" ]]
 then
-    echo "ERROR: Missing third argument, Request body!"
+    echo "ERROR: Missing third argument, Repo name!"
     exit 1
 else
-    REQUEST=$3
+    REPO_NAME=$3
 fi
 
 if [[ -z "$4" ]]
 then
-    echo "ERROR: Missing fourth argument, POST_MSG!"
+    echo "ERROR: Missing fourth argument, Request body!"
     exit 1
 else
-    POST_MSG=$4
+    REQUEST=$4
+fi
+
+if [[ -z "$5" ]]
+then
+    echo "ERROR: Missing fifth argument, POST_MSG!"
+    exit 1
+else
+    POST_MSG=$5
 fi
 
 . $1 # source the variables from config file
 
 print_header "Checking Required Variables"
 
 
- if [[ -z "$GITHUB_CREDS" ]]
+if [[ -z "$GITHUB_CREDS" ]]
 then
     echo "ERROR: GITHUB_CREDS not provided"
     exit 1
 fi
 
- if [[ -z "$REPO_OWNER" ]]
+if [[ -z "$REPO_OWNER" ]]
 then
     echo "ERROR: REPO_OWNER not provided"
     exit 1
 fi
 
- if [[ -z "$REPO_NAME" ]]
-then
-    echo "ERROR: REPO_NAME not provided"
-    exit 1
-fi
-
 print_header "Posting Message in Comments on GitHub"
 python3 post-msg.py $GITHUB_CREDS "{\"id\": $PR_ID,\"request\":\"$REQUEST\"}" $REPO_OWNER $REPO_NAME "$POST_MSG"
 check_exit_code "ERROR: Failed to post results to GitHub"

ci/clone-and-checkout-pr.py

@@ -41,10 +41,13 @@
 cmd = "git clone " + str(repo.clone_url) + " repository"
 
 child = pexpect.spawn(cmd)
-child.expect("Username.*")
-child.sendline(username + "\n")
-child.expect("Password.*")
-child.sendline(password + "\n")
+
+if '-dev' in REPO_NAME:
+    child.expect("Username.*")
+    child.sendline(username + "\n")
+    child.expect("Password.*")
+    child.sendline(password + "\n")
+
 child.interact()
 
 print(pexpect.run("git checkout " + branch_name, cwd="./repository"))

ci/manager.sh

@@ -31,10 +31,18 @@ fi
 
 if [[ -z "$3" ]]
 then
-    echo "ERROR: Missing third argument, Request body!"
+    echo "ERROR: Missing third argument, Repo name!"
     exit 1
 else
-    REQUEST=$3
+    REPO_NAME=$3
+fi
+
+if [[ -z "$4" ]]
+then
+    echo "ERROR: Missing fourth argument, Request body!"
+    exit 1
+else
+    REQUEST=$4
 fi
 
 . $1 # source the variables from config file
@@ -60,12 +68,6 @@ then
     exit 1
 fi
 
-if [[ -z "$REPO_NAME" ]]
-then
-    echo "ERROR: REPO_NAME not provided"
-    exit 1
-fi
-
 print_header "Cleaning up Old Results"
 
 sudo rm -f *.txt