Skip to content

Commit

Permalink
utreexo/cmd/readme.md && utreexo/readme.md: Update readme to reflect …
Browse files Browse the repository at this point in the history
…the code changes
  • Loading branch information
kcalvinalvin committed Feb 23, 2020
1 parent 19ca2a3 commit 2e01f95
Show file tree
Hide file tree
Showing 2 changed files with 44 additions and 21 deletions.
44 changes: 31 additions & 13 deletions cmd/readme.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,47 @@
# cmd

folders with executables
Everything in cmd/ is related to the actual node implementation of Utreexo.

General flow to get data on utreexo performance:
## csn

1. run blockparser to get txo list and ttl data.
Implements the Utreexo Compact State Node. The CSN is the node that keeps only
the Utreexo tree tops. For caching purposes, some TXOs may be kept. However, when
flushing to disk, the cache data isn't flushed. This feature will come in the future.

2. run txottl to merge the ttl data into the txo file
## bridgenode

3. run ibdsim -genroofs to build a db of proofs
Since a Bitcoin Core node cannot serve Utreexo proofs, a bridge node is needed.
The bridge node currently does:

4. run ibdsim to run through the IBD process with those proofs and see how much data & time it takes
1. Generate an index from the provided blk*.dat files.
2. Generate TXO proofs.
3. Do the Utreexo accumulator operations and maintain an Utreexo Forest.

The bridge node currently does not:

## blockparser
1. Verify headers
2. Verify blocks
3. Verify signatures

goes through bitcoind's blocks folder and reads all the transaction data. Creates 2 things: a txos text file, and a ttl leveldb folder.
The general idea for a bridge node is outlined in Section 4.5 in the Utreexo paper.
https://github.com/mit-dci/utreexo/blob/master/utreexo.pdf

## txottl
## ttl

txottl parses a text list of transactions and builds a database of how long txos last. It can then append the txo durations to the transaction file.
Time To Live is the representation of how long each transaction "lives" until it is
spent.

For example, a transaction that is created in block #200 and spent at block #400 has
a ttl value of 200.

## ibdsim
This is needed for caching outlined in Section 5.3 and 5.4 in the Utreexo paper.
https://github.com/mit-dci/utreexo/blob/master/utreexo.pdf

Performs the accumulator operations of initial block download (IBD) to measure performance.
Note that this doesn't do any signature or transaction verification, only the accumulator operations.
## util

Various reused functions, constants, and paths used in all the packages.

## clair

An attempt at implementing Bélády's clairvoyent algorithm. Currently, it does not build.
This code is kept to be used in the future.
21 changes: 13 additions & 8 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Logs for freenode are [here](https://github.com/utreexo-log/utreexo-irc-log)

### cmd

subfolders with executable code
subfolders with implementation

### utreexo

Expand Down Expand Up @@ -44,33 +44,38 @@ $ bitcoin-cli stop
$ go get github.com/mit-dci/utreexo
```

* build utreexo/simcmd/sim.go
* build utreexo

This will give you simcmd binary.
```
$ cd ~/go/src/github.com/mit-dci/utreexo/cmd/
$ go build
```

This will give you ``cmd``` binary

simcmd contains various commands that go from organizing the .dat files from Bitcoin Core to actually doing a Utreexo simulation. To view all the available commands and flags, just run './simcmd' by itself.
cmd contains various commands that go from indexing the blk*.dat files from Bitcoin Core to building the Bridge Node and the Compact State Node. To view all the available commands and flags, just run './cmd' by itself.

First we need to organize the blocks in .dat file, build a proof file and a db keeping record how long each transaction lasts until it is spent.

First, the "genproofs" command builds all the block proofs for the blockchain and the db for keeping how long a transaction lasts.

```
$ cd ~/.bitcoin/testnet3/blocks
$ ./simcmd genproofs -testnet=1 // -testnet=1 flag needed for testnet. Leave empty for mainnet
$ ./cmd genproofs -testnet=1 // -testnet=1 flag needed for testnet. Leave empty for mainnet
[... takes time and builds block proofs]
[genproofs is able to resume from where it left off. Use `ctrl + c` to stop it.]
[To resume, just do `./simcmd genproofs -testnet=1 again`]
[To resume, just do `./cmd genproofs -testnet=1 again`]
```

* "genproofs" should take a few hours as it does two things. First, it goes through the blockchain, maintains the full merkle forest, and saves proofs for each block to disk. Second, it saves each TXO and height with leveldb to make a TXO time to live (bascially how long each txo lasts until it is spent) for caching purposes. This is what the bridge node and archive node would do in a real node. Next, you can run 'simcmd ibdsim -testnet=1'; it will perform IBD as a compact node which maintains only a reduced state, and accepts proofs (which are created in the proof.dat file during the previous step)


```
$ cd ~/.bitcoin/testnet3/blocks
$ ./simcmd ibdsim -testnet=1 // -testnet=1 flag needed fro testnet. Leave empty for mainnet
$ ./cmd ibdsim -testnet=1 // -testnet=1 flag needed fro testnet. Leave empty for mainnet
[... takes time and does utreexo sync simulation]
[ibdsim is able to resume from where it left off. Use `ctrl + c` to stop it.]
[To resume, just do `./simcmd ibdsim -testnet=1 again`]
[To resume, just do `./cmd ibdsim -testnet=1 again`]
```

Note that your folders or filenames might be different, but this should give you the idea and work on default linux / golang setups. If you've tried this and it doesn't work and you'd like to help out, you can either fix the code / documentation so that it does work and make a pull request, or open an issue describing what doesn't work.

0 comments on commit 2e01f95

Please sign in to comment.