BLS Aggregated Committed Seals + ParentCommittedSeal (0xPolygon#649)

* Refactor the functions/methods for IBFT extra * Add ParentCommittedSeal in IBFT Extra * Fix lint error * Add unit test for ParentCommittedSeal in IBFT Extra * Fix comments * Rename function name * Fix typo * Remove return values from initIbftExtra * Shallow nesting * Add BLS in IBFT (WIP) * Add ECDSA & BLS Signer in IBFT * Update go.mod * Remove debug file * Add BLS mode in secrets init and genesis command * Fix build error * Update go.mod * Fix failed test * Fix lint error * Fix lint error * Split Signer struct to remove redundant codes in ECDSA & BLS Signer * Fix failing ParentCommittedSeal * Move validators parser from genesis package to validators package * Move the function to convert keys to validator set into validators package * Add new error for empty key type in secrets init command * Move GetBLSPubkeyFromValidatorKey func from secrets/init package to crypto * Remove debug code * Rename ECDSAToBLSPubkey to ECDSAToMarshalledBLSPubkey * Fix typo * Continue instead of error when failing collected committed seal * Uncommented required code * Fix code style * Uncomment extra test * Fix failed test * Abstract validator set - WIP * Change minter type in header to bytes - WIP * Revert Minter type in header * Fix build error - WIP * Fix failed test * Fix faild test * Add params in ibft switch command to change validation type - WIP * Switch IBFT signer dynamically * Fix failing ParentCommittedSeals verificateion issue * Add ForkManager * Add validator set * Fix predeploy staking SC * Fix empty Miner issue * Fix go.mod * Fix type of visited * Fix voting * Add ParseBLSPublicKey * Fix BLS validator voting issue * Fix predeploy script for the SC that holds BLS Public Keys * Ignore validator whose BLS Key is not set in fetch * Fix secret init command's output * Fix update validator set * Fix issue of ParentCommittedSeals verification * Fix update predeploy staking contract script * Fix lint error * Add IsGenesis helper method in Header * Fix build error in tests * Fix CreatedCommittedSeal issue in BLS * Remove unused function, NewValidatorSetFromType * Remove unused function, AddressToBytes * Removed unused function ParseSourceType * Fix Copy method in Vote in order to prevent memory leak issue * Refactor getNextCandidate * Add err check in FetchBLSValidators * Fix comment in ParseIBFTType * Fix comment in Validators * Add constants for map keys * Add ErrUndefinedIBFTConfig * Removed unused function, ParseEDDSAValidators * Removed unused function, ParseBLSValidators * Add comments in validators helper * Add comments for ECDSAValidator and BLSValidator * Refactor SnapshotSet's methods * Revert comment * Remove old comment * Extract common processes in query.go * Add error handling for os.Remove * Define ibftTypesToSourceType instead of function * Refactor loading Snapshot and Metadata * Add error constant in ibft/fork/hooks.go * Remove unused params * Add comments in manager.go * Remove unused error * Correct error message * Rename the fields in IBFTExtra * Fix comment * Add comment in signer package * Fix comment * Call Bytes() instead of getting slice * Remove error from return values, which is never returned * Correct command flag description * Add error handling in Quorum * Add helper method, extractParentCommittedSeals * Removed commented methods * Add unit tests in validators package * Add unit tests for contract validator set * Add unit tests for Snapshot validator set * Fix missing update of submodules after block insertion * Fix failed tests * Skip parent committed seals if it doesn't exist * Fix lint error * Fix wrong unmarshalling of BLSValidator * Rename ValidatorSet to ValidatorStore * Add tests in extra * Fix the existince check of ParentCommittedSeals * Fix lint error * Add unit tests in KeyManager * Make DecodeBLSPublicKeys unexported * Add error log after PostInsertBlock * Add unit tests of BLS Key Manager * Rename Sealer to Seals * Rename BLSSeal to AggregatedSeal * Fix json output of BLS Public Key * Add Set in validators package and remove ECDSAValidators and BLSValidators * Fix lint error * Move interface of Hooks to ibft package * Move interface of ForkManager to ibft package * Fix comment * Add comment in SnapshotValidatorStore * Remove unused function * Add LRU cache in ContractValidatorStore * Fix lint error * Fix unstable tests in BLS * Return Math.Int32 instead of 0 in Quorum * Revert arg name from proposal to proposalHash * Correct comment * Fix comment for function ParseValidatorType * Remove unrelated comment and unused method from hook package * Fix UpdateValidatorSet in PoA * Add mission method from previous commit * Move HeaderModifier and HeaderProcessor to fork package that calls store * Move Updatable to fork package that calls store * Move Votable to ibft package that calls SnapshotStore * Add comment * Add wrapper of validator store in fork package * Add comment * Fix lint error * Fix lint error * Fix lint error around math/rand * Fix lint error around math/rand * Revert "Fix lint error around math/rand" This reverts commit b4d58e9. Revert "Fix lint error around math/rand" This reverts commit 85ad56b. Revert "Fix lint error" This reverts commit 3769e5c. Revert "Fix lint error" This reverts commit 82e896c. * Cleanup Hooks Register Process * Fix wrong height calculation for contract store * Fix unmarshal parent committed seals issue in Signer * Fix linter * Make bls default * Make switch IBFTValidatorTypeFlag default bls * Make ECDSA default for e2e tests * Make bls public key requred for voting in bls * Removed unused fields * Remove unused cast * Add omitempty * Fix typo * Merge two if conditions * Add logging of IBFT validation type * Add unit tests for BLS * Move update submodules for IBFT because of concurrent access by syncer * Fix failed test * Fix failed test * Enable log in e2e for debug * Add timeout in e2e * Fix typo Co-authored-by: AleksaOpacic <[email protected]>
jhkimqd · Sep 5, 2022 · c39d854 · c39d854
1 parent 8e9099a
commit c39d854
Show file tree

Hide file tree

Showing 285 changed files with 78,044 additions and 4,385 deletions.
diff --git a/blockchain/blockchain.go b/blockchain/blockchain.go
@@ -87,7 +87,7 @@ type Verifier interface {
 	VerifyHeader(header *types.Header) error
 	ProcessHeaders(headers []*types.Header) error
 	GetBlockCreator(header *types.Header) (types.Address, error)
-	PreStateCommit(header *types.Header, txn *state.Transition) error
+	PreCommitState(header *types.Header, txn *state.Transition) error
 }
 
 type Executor interface {
@@ -843,7 +843,7 @@ func (b *Blockchain) executeBlockTransactions(block *types.Block) (*BlockResult,
 		return nil, err
 	}
 
-	if err := b.consensus.PreStateCommit(header, txn); err != nil {
+	if err := b.consensus.PreCommitState(header, txn); err != nil {
 		return nil, err
 	}
 

diff --git a/blockchain/testing.go b/blockchain/testing.go
@@ -262,22 +262,22 @@ func (m *MockVerifier) GetBlockCreator(header *types.Header) (types.Address, err
 		return m.getBlockCreatorFn(header)
 	}
 
-	return header.Miner, nil
+	return types.BytesToAddress(header.Miner), nil
 }
 
 func (m *MockVerifier) HookGetBlockCreator(fn getBlockCreatorDelegate) {
 	m.getBlockCreatorFn = fn
 }
 
-func (m *MockVerifier) PreStateCommit(header *types.Header, txn *state.Transition) error {
+func (m *MockVerifier) PreCommitState(header *types.Header, txn *state.Transition) error {
 	if m.preStateCommitFn != nil {
 		return m.preStateCommitFn(header, txn)
 	}
 
 	return nil
 }
 
-func (m *MockVerifier) HookPreStateCommit(fn preStateCommitDelegate) {
+func (m *MockVerifier) HookPreCommitState(fn preStateCommitDelegate) {
 	m.preStateCommitFn = fn
 }
 

diff --git a/chain/chain.go b/chain/chain.go
@@ -68,7 +68,7 @@ func (g *Genesis) GenesisHeader() *types.Header {
 		GasUsed:      g.GasUsed,
 		Difficulty:   g.Difficulty,
 		MixHash:      g.Mixhash,
-		Miner:        g.Coinbase,
+		Miner:        g.Coinbase.Bytes(),
 		StateRoot:    stateRoot,
 		Sha3Uncles:   types.EmptyUncleHash,
 		ReceiptsRoot: types.EmptyRootHash,

diff --git a/command/common.go b/command/common.go
@@ -2,15 +2,27 @@ package command
 
 import (
 	"errors"
+	"io/ioutil"
+	"strings"
 
+	"github.com/0xPolygon/polygon-edge/crypto"
 	"github.com/0xPolygon/polygon-edge/helper/common"
+	"github.com/0xPolygon/polygon-edge/secrets"
+	"github.com/0xPolygon/polygon-edge/secrets/local"
+	"github.com/0xPolygon/polygon-edge/types"
+	"github.com/0xPolygon/polygon-edge/validators"
+	"github.com/hashicorp/go-hclog"
 )
 
 const (
 	ConsensusFlag  = "consensus"
 	NoDiscoverFlag = "no-discover"
 	BootnodeFlag   = "bootnode"
 	LogLevelFlag   = "log-level"
+
+	IBFTValidatorTypeFlag   = "ibft-validator-type"
+	IBFTValidatorFlag       = "ibft-validator"
+	IBFTValidatorPrefixFlag = "ibft-validators-prefix-path"
 )
 
 var (
@@ -19,6 +31,10 @@ var (
 	errInvalidMinNumValidators = errors.New("minimum number of validators must be greater than 0")
 	errInvalidMaxNumValidators = errors.New("maximum number of validators must be lower or equal " +
 		"than MaxSafeJSInt (2^53 - 2)")
+
+	ErrValidatorNumberExceedsMax = errors.New("validator number exceeds max validator number")
+	ErrECDSAKeyNotFound          = errors.New("ECDSA key not found in given path")
+	ErrBLSKeyNotFound            = errors.New("BLS key not found in given path")
 )
 
 func ValidateMinMaxValidatorsNumber(minValidatorCount uint64, maxValidatorCount uint64) error {
@@ -36,3 +52,113 @@ func ValidateMinMaxValidatorsNumber(minValidatorCount uint64, maxValidatorCount
 
 	return nil
 }
+
+// GetValidatorsFromPrefixPath extracts the addresses of the validators based on the directory
+// prefix. It scans the directories for validator private keys and compiles a list of addresses
+func GetValidatorsFromPrefixPath(
+	prefix string,
+	validatorType validators.ValidatorType,
+) (validators.Validators, error) {
+	files, err := ioutil.ReadDir(".")
+	if err != nil {
+		return nil, err
+	}
+
+	validatorSet := validators.NewValidatorSetFromType(validatorType)
+
+	for _, file := range files {
+		path := file.Name()
+
+		if !file.IsDir() || !strings.HasPrefix(path, prefix) {
+			continue
+		}
+
+		localSecretsManager, err := local.SecretsManagerFactory(
+			nil,
+			&secrets.SecretsManagerParams{
+				Logger: hclog.NewNullLogger(),
+				Extra: map[string]interface{}{
+					secrets.Path: path,
+				},
+			},
+		)
+		if err != nil {
+			return nil, err
+		}
+
+		address, err := getValidatorAddressFromSecretManager(localSecretsManager)
+		if err != nil {
+			return nil, err
+		}
+
+		switch validatorType {
+		case validators.ECDSAValidatorType:
+			if err := validatorSet.Add(&validators.ECDSAValidator{
+				Address: address,
+			}); err != nil {
+				return nil, err
+			}
+
+		case validators.BLSValidatorType:
+			blsPublicKey, err := getBLSPublicKeyBytesFromSecretManager(localSecretsManager)
+			if err != nil {
+				return nil, err
+			}
+
+			if err := validatorSet.Add(&validators.BLSValidator{
+				Address:      address,
+				BLSPublicKey: blsPublicKey,
+			}); err != nil {
+				return nil, err
+			}
+		}
+	}
+
+	return validatorSet, nil
+}
+
+func getValidatorAddressFromSecretManager(manager secrets.SecretsManager) (types.Address, error) {
+	if !manager.HasSecret(secrets.ValidatorKey) {
+		return types.ZeroAddress, ErrECDSAKeyNotFound
+	}
+
+	keyBytes, err := manager.GetSecret(secrets.ValidatorKey)
+	if err != nil {
+		return types.ZeroAddress, err
+	}
+
+	privKey, err := crypto.BytesToECDSAPrivateKey(keyBytes)
+	if err != nil {
+		return types.ZeroAddress, err
+	}
+
+	return crypto.PubKeyToAddress(&privKey.PublicKey), nil
+}
+
+func getBLSPublicKeyBytesFromSecretManager(manager secrets.SecretsManager) ([]byte, error) {
+	if !manager.HasSecret(secrets.ValidatorBLSKey) {
+		return nil, ErrBLSKeyNotFound
+	}
+
+	keyBytes, err := manager.GetSecret(secrets.ValidatorBLSKey)
+	if err != nil {
+		return nil, err
+	}
+
+	secretKey, err := crypto.BytesToBLSSecretKey(keyBytes)
+	if err != nil {
+		return nil, err
+	}
+
+	pubKey, err := secretKey.GetPublicKey()
+	if err != nil {
+		return nil, err
+	}
+
+	pubKeyBytes, err := pubKey.MarshalBinary()
+	if err != nil {
+		return nil, err
+	}
+
+	return pubKeyBytes, nil
+}
diff --git a/command/genesis/genesis.go b/command/genesis/genesis.go
@@ -7,6 +7,7 @@ import (
 	"github.com/0xPolygon/polygon-edge/command/helper"
 	"github.com/0xPolygon/polygon-edge/consensus/ibft"
 	"github.com/0xPolygon/polygon-edge/helper/common"
+	"github.com/0xPolygon/polygon-edge/validators"
 	"github.com/spf13/cobra"
 )
 
@@ -90,24 +91,31 @@ func setFlags(cmd *cobra.Command) {
 
 	// IBFT Validators
 	{
+		cmd.Flags().StringVar(
+			&params.rawIBFTValidatorType,
+			command.IBFTValidatorTypeFlag,
+			string(validators.BLSValidatorType),
+			"the type of validators in IBFT",
+		)
+
 		cmd.Flags().StringVar(
 			&params.validatorPrefixPath,
-			ibftValidatorPrefixFlag,
+			command.IBFTValidatorPrefixFlag,
 			"",
 			"prefix path for validator folder directory. "+
 				"Needs to be present if ibft-validator is omitted",
 		)
 
 		cmd.Flags().StringArrayVar(
 			&params.ibftValidatorsRaw,
-			ibftValidatorFlag,
+			command.IBFTValidatorFlag,
 			[]string{},
 			"addresses to be used as IBFT validators, can be used multiple times. "+
 				"Needs to be present if ibft-validators-prefix-path is omitted",
 		)
 
 		// --ibft-validator-prefix-path & --ibft-validator can't be given at same time
-		cmd.MarkFlagsMutuallyExclusive(ibftValidatorPrefixFlag, ibftValidatorFlag)
+		cmd.MarkFlagsMutuallyExclusive(command.IBFTValidatorPrefixFlag, command.IBFTValidatorFlag)
 	}
 
 	// PoS