Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
blockchain: Convert to full block index in mem.
This reworks the block index code such that it loads all of the headers in the main chain at startup and constructs the full block index accordingly. Since the full index from the current best tip all the way back to the genesis block is now guaranteed to be in memory, this also removes all code related to dynamically loading the nodes and updates some of the logic to take advantage of the fact traversing the block index can no longer potentially fail. There are also many more optimizations and simplifications that can be made in the future as a result of this. Due to removing all of the extra overhead of tracking the dynamic state, and ensuring the block node structs are aligned to eliminate extra padding, the end result of a fully populated block index now takes quite a bit less memory than the previous dynamically loaded version. It also speeds up the initial startup process by roughly 2x since it is faster to bulk load the nodes in order as opposed to dynamically loading only the nodes near the tip in backwards order. For example, here is some startup timing information before and after this commit on a node that contains roughly 238,000 blocks: 7200 RPM HDD: ------------- Startup time before this commit: ~7.71s Startup time after this commit: ~3.47s SSD: ---- Startup time before this commit: ~6.34s Startup time after this commit: ~3.51s Some additional benefits are: - Since every block node is in memory, the code which reconstructs headers from block nodes means that all headers can always be served from memory which will be important since the network will be moving to header-based semantics - Several of the error paths can be removed since they are no longer necessary - It is no longer expensive to calculate CSV sequence locks or median times of blocks way in the past - It is much less expensive to calculate the initial states for the various intervals such as the stake and voter version - It will be possible to create much more efficient iteration and simplified views of the overall index An overview of the logic changes are as follows: - Move AncestorNode from blockIndex to blockNode and greatly simplify since it no longer has to deal with the possibility of dynamically loading nodes and related failures - Replace nodeAtHeightFromTopNode from BlockChain with RelativeAncestor on blockNode and define it in terms of AncestorNode - Move CalcPastMedianTime from blockIndex to blockNode and remove no longer necessary test for nil - Remove findNode and replace all of its uses with direct queries of the block index - Remove blockExists and replace all of its uses with direct queries of the block index - Remove all functions and fields related to dynamically loading nodes - children and parentHash fields from blockNode - depNodes from blockIndex - loadBlockNode from blockIndex - PrevNodeFromBlock from blockIndex - {p,P}revNodeFromNode from blockIndex - RemoveNode - Replace all instances of iterating backwards through nodes to directly access the parent now that nodes don't potentially need to be dynamically loaded - Introduce a lookupNode function on blockIndex which allows the initialization code to locklessly query the index - No longer take the chain lock when only access to the block index, which has its own lock, is needed - Removed the error paths from several functions that can no longer fail - getReorganizeNodes - findPrevTestNetDifficulty - sumPurchasedTickets - findStakeVersionPriorNode - Removed all error paths related to node iteration that can no longer fail - Modify FetchUtxoView to return an empty view for the genesis block