Quorum Network Manager (QNM)

Preparation for setting up new Quorum Raft network:

The VM setup for this document assumes VM nx01 is the Coordinator node and MAS Regulator node.

You can follow the ubin VMs IP config here.

1. Pre-requisites to be executed on all the nodes

$ cd ~/ubin-quorum-setup/binaries/setup
$ ./cleanup_process.sh
$ ps –alef | grep geth # you should not see any geth instances running
$ ps –alef | grep constellation # you should not see any constellation instances running
$ cd ~/ubin-quorum-setup/binaries/QuorumNetworkManager

# if you are doing a re-setup and you will need to reattach to the existing screen instance
$ screen -r

# if you are doing a setup for the first time or if the screen process has been killed, execute the below
$ screen node index.js

Do note that usage of "screen" is not mandatory. The alternative is to open more ssh sessions.

2. Configure coordinating node

1. Identify the coordinator node, this is the node that all the participating nodes will join. This is the MAS Regulator node for Project Ubin - in this guide it will be named " nx01".

Example 1:

# Quorum Network Manager open, proceed only if ip address available.
# Type in the server name for the nodeName in the four character, this is only used for reference
# format nx01 - nx15
prompt: localIpAddress: (1.2.3.4)
prompt: nodeName: nx01

Example 2:

prompt: localIpAddress: (210.240.162.43) 192.168.56.3
prompt: nodeName: nx01

2. Close any running instances of geth and constellation by selecting option 5.

prompt: nodeName: nx01
Please select an option:
1) Raft
2) QuorumChain
5) Kill all geth and constellation
prompt: option: 5

3. In the menu select option 1, "Raft"

prompt: localIpAddress: (1.2.3.4)
prompt: nodeName: nx01
Please select an option:
1) Raft
2) QuorumChain
5) Kill all geth and constellation
prompt: option: 1

4. Copy the IP address (this will be used when we configure the other non-coordinating nodes) Select 1, "Start a node as the setup coordinator"

Please select an option below:
----- Option 1 and 2 are for the initial setup of a raft network -----
1) Start a node as the setup coordinator [Ideally there should only be one coordinator]
2) Start a node as a non-coordinator
----- Option 3 is for joining a raft network post initial setup -----
3) Join a raft network if you were not part of the initial setup
4) TODO: Start whisper services and attach to already running node
5) killall geth constellation-node
0) Quit
prompt: option: 1

5. Select 1, "Allow anyone to connect"

Please select an option below:
1) Allow anyone to connect
2) [TODO] Allow only people with pre-auth tokens to connect
prompt: option: 1

6. Select 1, "Clear all files..." or "Keep old files..."

Please note that if this is the first time you are setting up, or if you want to erase previous and re-do the setup, you should select "Clear all files". If you are re-running from previous setup, you MUST select "Keep old files". Failing this will result in all your nodes configuration information being erased!

Please select an option below:
1) Clear all files/configuration and start from scratch[WARNING: this clears everything]
2) Keep old files/configuration intact and start the node + whisper services
prompt: option: 1
[*] Starting new node...
Generating node key
enode: enode://b05e6feb06c36196bba57ca5e324f888b2c12f63e00503fbcc247d56f42cea2c8017cda9c90fca6381f8936582b1a5bcf3148c91891a22762665c6f33c8f0e16@52.187.61.155:20000
[*] Starting communication node...
[*] RPC connection established, Node started
Please wait for others to join. Hit any key + enter once done.
prompt: done:

7. A message will appear that it is waiting for new nodes to join; at this point, participating nodes can join in. Go to Section 3 (Steps for Participating nodes (non-coordinator)) and execute the steps for each node you want to add. Return to this section and continue with the next step once you have finished adding all the new nodes.

Only do the next step once all the nodes have joined in

8. Once all nodes have joined, press any key and then Enter to create the network config files. Wait until all nodes have responded then press Ctrl+A+D to detach from the screen. This example only shows a network with only 3 nodes joining (including coordinating node):

prompt: done:  nx02 has joined the network
nx03 has joined the network
<any key> and <enter>
Adding the following addresses to the genesis block: [ '0x6398dd29d801211d2492e9b6062c33b8aac74599',
  '0x000000000000000000000000000000005a534c01',
  '0x000000000000000000000000000000005a534c02',
  '0x000000000000000000000000000000005a534c03',
  '0x000000000000000000000000000000005a534c04',
  '0x000000000000000000000000000000005a534c05',
  '0x4a96d31bfe8d4e797ba60cb9994fb18220354263',
  '0x49c40d934caf14114210161ab4bd6d59d12b53e3' ]
[*] Creating genesis config...
[*] Starting raft node...
[*] RPC connection established, Node started
[*] Done

9.\ Save a copy of network configuration file (networkNodeInfo.json) immediately after the above step 8. This file is updated by step 8 and is found in QuorumNetworkManager directory. It will be used for the Deployment section later.

3. Steps for Participating nodes (non-coordinator)

Ensure Section 1: Pre-requisites are executed For each participating (non-coordinator) node, perform below steps:

1. Give the node a name (e.g. "nx02"), this name has to be unique.

Example 1:

prompt: localIpAddress: (1.2.3.5)
prompt: nodeName: nx02

Example 2:

prompt: localIpAddress: (210.240.162.43) 192.168.56.4
prompt: nodeName: nx02

2. Select 1, "Raft"

prompt: localIpAddress: (1.2.3.5)
prompt: nodeName: nx02
Please select an option:
1) Raft
2) QuorumChain
5) Kill all geth and constellation
prompt: option: 1

3. Select 2, "Start a node as a non-coordinator"

Please select an option below:
----- Option 1 and 2 are for the initial setup of a raft network -----
1) Start a node as the setup coordinator [Ideally there should only be one coordinator]
2) Start a node as a non-coordinator
----- Option 3 is for joining a raft network post initial setup -----
3) Join a raft network if you were not part of the initial setup
4) TODO: Start whisper services and attach to already running node
5) killall geth constellation-node
0) Quit
prompt: option: 2

4. Select 1, "Clear all files..." or "Keep old files..."

Again, please note that if this is the first time you are setting up, or if you want to erase previous and re-do the setup, you should select "Clear all files". If you are re-running from previous setup, you MUST select "Keep old files". Failing this will result in all your nodes configuration information being erased!

Please select an option below:
1) Clear all files/configuration and start from scratch[WARNING: this clears everything]
2) Keep old files/configuration intact and start the node + whisper services
prompt: option: 1

5. Enter the IP address of the coordinating node.

Example 1:

In order to join the network, please enter the ip address of the coordinating node
prompt: ipAddress: 1.2.3.4

Example 2:

prompt: ipAddress: 192.168.56.3

6. After a short moment, a message should appear that the node has joined.

[*] Starting new node...
Account: 0xed0b370fc5d85293ed6278bfd2da15dae23f9879
Generating node key
enode: enode://0795cf843077067f740142548098dc7a01046fe9ac958984ee6ca923816e7c14ec32d98eb7816c55fcda7cd914b70bae3dca6c62063884900d7490acf40dcd7d@52.187.57.112:20000
[*] Joining communication network...
[*] RPC connection established, Node started
true
[*] Communication network joined
[*] Requesting network membership. This will block until the other node responds
[*] Network membership: ACCEPTED
[*] Requesting genesis block config. This will block until the other node is online

7. The node setup is complete. Repeat the steps above for each node that will be part of the Quorum network. Detach from the screen sessions using key combos Ctrl+A+D.

8. Confirm network setup by connecting to the geth console from within the same directory and examining the number of peers the node is connected to. The number should be total number of nodes - 1 to exclude the current node.

# if not already in this directory
$ cd ~/ubin-quorum-setup/binaries/QuorumNetworkManager
# execute script to attach to the local geth console
$ ./attachToLocalQuorumNode.sh

In the resulting geth console, execute command admin.peers.length to see the number of connected peers.

Welcome to the Geth JavaScript console!

instance: Geth/v1.5.0-unstable-f4adbc2e/linux/go1.7.3
coinbase: 0x517424387f40b6e7406ee33be7ac31700012d762
at block: 3656 (Fri, 11 Feb 47846936856 07:04:31 UTC)
 datadir: /home/azureuser/ubin-quorum-setup/binaries/QuorumNetworkManager/Blockchain
 modules: admin:1.0 debug:1.0 eth:1.0 net:1.0 personal:1.0 quorum:1.0 raft:1.0 rpc:1.0 shh:1.0 txpool:1.0 web3:1.0 zsl:1.0

> admin.peers.length
13
> 

9. When all nodes have been added, return to Section 2, Step 8 to complete the network setup from the coordinator VM.

If you are using "screen" - Once the coordinator steps have been completed and you have detacted from screen (Ctrl+A+D) in each of the VMs, the Raft Quorum network setup is complete.