Goals for upcoming May Utreexo Workshop

[kcalvinalvin]

This is a list of goals for the upcoming May Utreexo workshop. Having a Bridge Node and a well documented code would help with newcomers to the project.

I think the core libraries are stable enough to go make a bridge node now. I thought of some things the bridge node should do. Please correct/add to the list as necessary.

Nodes

1. Make Standalone Bridge Node

Network

A. Query the default bootstrapping Classic Bitcoin nodes and receive blocks received from them.

B. Serve Classic Bitcoin Nodes as well as Utreexo Compact State Nodes as stated in the paper.

1. Keep blk*.dat files and rev*.dat files like Bitcoin Core.
2. Keep Proofs and the Compact state separately in a different directory.

C. Contain a mempool.

1. Generate Proof for TX from a Classic Bitcoin Node.
2. Serve both the Classic Bitcoin Node as well as Utreexo Compact State Nodes.

Storage

A. Index the locations of a block in a blk*.dat file. We have three options:

1. Use our offsetfile method for indexing. It would just do them on the fly instead of reading from a given local blk*.dat file.
2. Use Leveldb.
3. Receive the blocks sequentially. (Would be slower)

Open to thoughts here.

2. Make Standalone Utreexo Compact State Node

Network

A. Set up Bridge Node(s) for Utreexo Compact State Nodes to query.

Make a Bridge Node with MIT's resources?

B. Set up Utreexo Compact State Node(s)

Should be easy.

C. Make it able to flush caches.

3. Utreexo API

A list of calls to provide so anyone can fork and start using the library. Would also help in seeing how Utreexo could fit into Bitcoin Core. This is an attempt to list what there should be. Please suggest additions/subtractions. Does fit into the Bridge Node and the Utreexo Compact Node construction

Documentation

Make godoc like btcd.

1. README.md and doc.go for every directory

There are old comments that need cleanup and there are also functions with outdated/no comments.

2. Clean up old comments from all directories

A. cmd/* needs cleanup

B. utreexo/* needs cleanup

3. A medium article or something similar would be nice.

[adiabat]

Cool - generally agree; some comments:

1. Make Standalone Bridge Node - this is pretty much done. Not too much left here, and we don't need it to support normal bitcoin full nodes, or even do any validation itself. Initially it can be just a server that feeds data do CSNs (compact state nodes)

I also think that mempool can come later, after we get block proofs & validation on both sides. There are some optimizations & tricky parts about dealing with mempool that we can leave till later. Blocks-only mode is a thing, and gives plenty of functionality for proving the concept.

2. CSN - this part is the harder one because of the need to do block validation, without a database. The function we need is

func ValidateBlock(block wire.MsgBlock, utxos []simutil.CTxOut) bool {}

(maybe not CTxOut but something like that). It can return false on an error.

Once we have ValidateBlock, try changing btcd code internally to use that ValidateBlock() function, while still keeping everything in place (levelDB utxo set). At that point, everything should still work. Maybe this seems a bit useless, but I think it's helpful to have; we can be sure that the shuffling of code hasn't broken things, so that when we drop in utreexo we can see if things still work. Then pull out levelDB and have utreexo be the one giving the CTxOuts to the ValidateBlock function.

[kcalvinalvin]

Cool. So just need to do some network stuff and make CSN validate.

Could you add them to projects?

[ysangkok]

bc-2.jp now says "TBD" where the date used to be, so i think it may not be in May?

[kcalvinalvin]

Yeah.. it got delayed because of the virus. I guess we have more time now? heh