From 2e01f95f372f1807d0302853fdf8ea77b2fee262 Mon Sep 17 00:00:00 2001 From: Calvin Kim Date: Mon, 24 Feb 2020 00:11:38 +0900 Subject: [PATCH] utreexo/cmd/readme.md && utreexo/readme.md: Update readme to reflect the code changes --- cmd/readme.md | 44 +++++++++++++++++++++++++++++++------------- readme.md | 21 +++++++++++++-------- 2 files changed, 44 insertions(+), 21 deletions(-) diff --git a/cmd/readme.md b/cmd/readme.md index 953f7929..eb7630ae 100644 --- a/cmd/readme.md +++ b/cmd/readme.md @@ -1,29 +1,47 @@ # cmd -folders with executables +Everything in cmd/ is related to the actual node implementation of Utreexo. -General flow to get data on utreexo performance: +## csn -1. run blockparser to get txo list and ttl data. +Implements the Utreexo Compact State Node. The CSN is the node that keeps only +the Utreexo tree tops. For caching purposes, some TXOs may be kept. However, when +flushing to disk, the cache data isn't flushed. This feature will come in the future. -2. run txottl to merge the ttl data into the txo file +## bridgenode -3. run ibdsim -genroofs to build a db of proofs +Since a Bitcoin Core node cannot serve Utreexo proofs, a bridge node is needed. +The bridge node currently does: -4. run ibdsim to run through the IBD process with those proofs and see how much data & time it takes +1. Generate an index from the provided blk*.dat files. +2. Generate TXO proofs. +3. Do the Utreexo accumulator operations and maintain an Utreexo Forest. +The bridge node currently does not: -## blockparser +1. Verify headers +2. Verify blocks +3. Verify signatures -goes through bitcoind's blocks folder and reads all the transaction data. Creates 2 things: a txos text file, and a ttl leveldb folder. +The general idea for a bridge node is outlined in Section 4.5 in the Utreexo paper. +https://github.com/mit-dci/utreexo/blob/master/utreexo.pdf -## txottl +## ttl -txottl parses a text list of transactions and builds a database of how long txos last. It can then append the txo durations to the transaction file. +Time To Live is the representation of how long each transaction "lives" until it is +spent. +For example, a transaction that is created in block #200 and spent at block #400 has +a ttl value of 200. -## ibdsim +This is needed for caching outlined in Section 5.3 and 5.4 in the Utreexo paper. +https://github.com/mit-dci/utreexo/blob/master/utreexo.pdf -Performs the accumulator operations of initial block download (IBD) to measure performance. -Note that this doesn't do any signature or transaction verification, only the accumulator operations. +## util +Various reused functions, constants, and paths used in all the packages. + +## clair + +An attempt at implementing Bélády's clairvoyent algorithm. Currently, it does not build. +This code is kept to be used in the future. diff --git a/readme.md b/readme.md index 3b57355f..521630ef 100644 --- a/readme.md +++ b/readme.md @@ -12,7 +12,7 @@ Logs for freenode are [here](https://github.com/utreexo-log/utreexo-irc-log) ### cmd -subfolders with executable code +subfolders with implementation ### utreexo @@ -44,11 +44,16 @@ $ bitcoin-cli stop $ go get github.com/mit-dci/utreexo ``` -* build utreexo/simcmd/sim.go +* build utreexo -This will give you simcmd binary. +``` +$ cd ~/go/src/github.com/mit-dci/utreexo/cmd/ +$ go build +``` + +This will give you ``cmd``` binary -simcmd contains various commands that go from organizing the .dat files from Bitcoin Core to actually doing a Utreexo simulation. To view all the available commands and flags, just run './simcmd' by itself. +cmd contains various commands that go from indexing the blk*.dat files from Bitcoin Core to building the Bridge Node and the Compact State Node. To view all the available commands and flags, just run './cmd' by itself. First we need to organize the blocks in .dat file, build a proof file and a db keeping record how long each transaction lasts until it is spent. @@ -56,10 +61,10 @@ First, the "genproofs" command builds all the block proofs for the blockchain an ``` $ cd ~/.bitcoin/testnet3/blocks -$ ./simcmd genproofs -testnet=1 // -testnet=1 flag needed for testnet. Leave empty for mainnet +$ ./cmd genproofs -testnet=1 // -testnet=1 flag needed for testnet. Leave empty for mainnet [... takes time and builds block proofs] [genproofs is able to resume from where it left off. Use `ctrl + c` to stop it.] -[To resume, just do `./simcmd genproofs -testnet=1 again`] +[To resume, just do `./cmd genproofs -testnet=1 again`] ``` * "genproofs" should take a few hours as it does two things. First, it goes through the blockchain, maintains the full merkle forest, and saves proofs for each block to disk. Second, it saves each TXO and height with leveldb to make a TXO time to live (bascially how long each txo lasts until it is spent) for caching purposes. This is what the bridge node and archive node would do in a real node. Next, you can run 'simcmd ibdsim -testnet=1'; it will perform IBD as a compact node which maintains only a reduced state, and accepts proofs (which are created in the proof.dat file during the previous step) @@ -67,10 +72,10 @@ $ ./simcmd genproofs -testnet=1 // -testnet=1 flag needed for testnet. Leave emp ``` $ cd ~/.bitcoin/testnet3/blocks -$ ./simcmd ibdsim -testnet=1 // -testnet=1 flag needed fro testnet. Leave empty for mainnet +$ ./cmd ibdsim -testnet=1 // -testnet=1 flag needed fro testnet. Leave empty for mainnet [... takes time and does utreexo sync simulation] [ibdsim is able to resume from where it left off. Use `ctrl + c` to stop it.] -[To resume, just do `./simcmd ibdsim -testnet=1 again`] +[To resume, just do `./cmd ibdsim -testnet=1 again`] ``` Note that your folders or filenames might be different, but this should give you the idea and work on default linux / golang setups. If you've tried this and it doesn't work and you'd like to help out, you can either fix the code / documentation so that it does work and make a pull request, or open an issue describing what doesn't work.