Initial version #3

mike-petrov · 2025-01-16T18:25:50Z

closes #1

Signed-off-by: Mikhail Petrov [email protected]

mike-petrov · 2025-01-16T18:26:40Z

mike-petrov · 2025-01-16T18:27:58Z

@roman-khimov first working version above, now I'm doing streaming file downloads in the background with no memory load.

Signed-off-by: Mikhail Petrov <[email protected]>

Follows send.fs.neo.org mostly. Signed-off-by: Mikhail Petrov <[email protected]>

mike-petrov · 2025-01-23T16:27:58Z

roman-khimov

Min/max do not read well, we need something like "the latest available block is XXX" dependent on the network.

Downloading the full set should be like opening up the page and hitting one button, that is 0 and the latest should be pre-filled (on network switch)

Browser limitations should be clearly explained, IIRC Firefox can't do this because it doesn't support appropriate API.

Some "About" can be useful as well explaining how it works similar to how send.fs.neo.org has it.

CHANGELOG.md

src/Home.tsx

Follows send.fs.neo.org mostly. Signed-off-by: Mikhail Petrov <[email protected]>

Signed-off-by: Mikhail Petrov <[email protected]>

mike-petrov · 2025-01-31T21:54:29Z

public/img/cover.png

should be updated when we're done with the content.

public/index.html

mike-petrov · 2025-01-31T21:56:10Z

src/About.tsx

+						>
+							<Heading weight="semibold" subtitle style={{ textAlign: 'center' }}>About Service</Heading>
+							<Content>
+								<p>Archive.NeoFS is a web application that allows users to create blockchain archives of any span (from block 0 to the current block or a custom range) directly in the browser. It operates fully client-side, leveraging streaming techniques to efficiently fetch and store blockchain data without requiring additional backend processing.</p>


I tried to describe it well, but @roman-khimov do it better.

Archive.NeoFS is a web application that allows users to create blockchain archives of any span (from block 0 to the current block or a custom range) directly in the browser. It operates fully client-side, leveraging streaming techniques to efficiently fetch and store blockchain data without requiring additional backend processing.

Archive.NeoFS is a web application that allows users to create blockchain archives of any span (from block 0 to the current block or a custom range) directly in the browser. It operates fully client-side, leveraging standard NeoFS REST gateway APIs and in-browser streaming techniques to efficiently fetch and store blocks without requiring additional backend processing.

src/Home.tsx

mike-petrov · 2025-01-31T22:01:01Z

roman-khimov · 2025-02-03T15:49:38Z

src/About.tsx

+						>
+							<Heading weight="semibold" subtitle style={{ textAlign: 'center' }}>About Service</Heading>
+							<Content>
+								<p>Archive.NeoFS is a web application that allows users to create blockchain archives of any span (from block 0 to the current block or a custom range) directly in the browser. It operates fully client-side, leveraging streaming techniques to efficiently fetch and store blockchain data without requiring additional backend processing.</p>


Archive.NeoFS is a web application that allows users to create blockchain archives of any span (from block 0 to the current block or a custom range) directly in the browser. It operates fully client-side, leveraging streaming techniques to efficiently fetch and store blockchain data without requiring additional backend processing.

Archive.NeoFS is a web application that allows users to create blockchain archives of any span (from block 0 to the current block or a custom range) directly in the browser. It operates fully client-side, leveraging standard NeoFS REST gateway APIs and in-browser streaming techniques to efficiently fetch and store blocks without requiring additional backend processing.

roman-khimov · 2025-02-03T15:50:57Z

src/About.tsx

+							<Heading weight="semibold" subtitle style={{ textAlign: 'center' }}>About Service</Heading>
+							<Content>
+								<p>Archive.NeoFS is a web application that allows users to create blockchain archives of any span (from block 0 to the current block or a custom range) directly in the browser. It operates fully client-side, leveraging streaming techniques to efficiently fetch and store blockchain data without requiring additional backend processing.</p>
+								<p>The service supports four networks: mainnet, testnet, NeoFS mainnet, and NeoFS testnet. It interacts with the <a href="https://github.com/nspcc-dev/neofs-rest-gw/" target="_blank" rel="noopener noreferrer">NeoFS REST gateway</a> to retrieve blockchain data stored in NeoFS objects and assembles them into a structured archive format (.acc), optimized for chain data storage.</p>


s/optimized for chain data storage/that is compatible with both C# Neo node and NeoGo/

roman-khimov · 2025-02-03T15:52:15Z

src/About.tsx

+							<Content>
+								<p>Archive.NeoFS is a web application that allows users to create blockchain archives of any span (from block 0 to the current block or a custom range) directly in the browser. It operates fully client-side, leveraging streaming techniques to efficiently fetch and store blockchain data without requiring additional backend processing.</p>
+								<p>The service supports four networks: mainnet, testnet, NeoFS mainnet, and NeoFS testnet. It interacts with the <a href="https://github.com/nspcc-dev/neofs-rest-gw/" target="_blank" rel="noopener noreferrer">NeoFS REST gateway</a> to retrieve blockchain data stored in NeoFS objects and assembles them into a structured archive format (.acc), optimized for chain data storage.</p>
+								<p><a href="https://github.com/nspcc-dev/send-fs-neo-org" target="_blank" rel="noopener noreferrer">Frontend part</a> first determines the latest available block in the selected network. Blockchain data is stored in NeoFS objects, indexed in segments of 128,000 blocks. The service sequentially retrieves these index files, extracts block references, and downloads the corresponding data. The entire process runs in the browser, streaming blocks directly into an archive file, ensuring efficient storage and real-time access.</p>


determines the latest available block

How?

indexed in segments of 128,000 blocks

Indexes and objects are separate things, this should be described. In fact it's one object per one block, but index objects are needed to map indexes to OIDs.

roman-khimov · 2025-02-03T15:54:28Z

src/About.tsx

+								<p>Archive.NeoFS is a web application that allows users to create blockchain archives of any span (from block 0 to the current block or a custom range) directly in the browser. It operates fully client-side, leveraging streaming techniques to efficiently fetch and store blockchain data without requiring additional backend processing.</p>
+								<p>The service supports four networks: mainnet, testnet, NeoFS mainnet, and NeoFS testnet. It interacts with the <a href="https://github.com/nspcc-dev/neofs-rest-gw/" target="_blank" rel="noopener noreferrer">NeoFS REST gateway</a> to retrieve blockchain data stored in NeoFS objects and assembles them into a structured archive format (.acc), optimized for chain data storage.</p>
+								<p><a href="https://github.com/nspcc-dev/send-fs-neo-org" target="_blank" rel="noopener noreferrer">Frontend part</a> first determines the latest available block in the selected network. Blockchain data is stored in NeoFS objects, indexed in segments of 128,000 blocks. The service sequentially retrieves these index files, extracts block references, and downloads the corresponding data. The entire process runs in the browser, streaming blocks directly into an archive file, ensuring efficient storage and real-time access.</p>
+							</Content>


Nothing is written about in-browser dynamics of it, it is important to mention specific streaming APIs (and tell why some browsers are not supported) and how they're leveraged.

roman-khimov · 2025-02-03T15:55:18Z

src/Home.tsx

+
+			await writableStream.close();
+		} catch (error: any) {
+			if (error.message.indexOf('showSaveFilePicker is not a function') !== -1) {


It should be possible to see about from a "bad" browser.

roman-khimov · 2025-02-03T15:55:53Z

src/Home.tsx

+
+		try {
+			const fileHandle = await window.showSaveFilePicker({
+				suggestedName: `chain_${formData.spanStart}.acc`,


IINM it's chain.X.acc, not chain_X.acc.

roman-khimov · 2025-02-03T15:56:11Z

src/Home.tsx

+						>
+							<Heading weight="semibold" subtitle style={{ textAlign: 'center' }}>Archive.NeoFS – Download Blockchain Data Snapshot</Heading>
+							<Content>
+								<p>Easily download an offline package of blockchain data up to a specific block height.</p>


s/blockchain data/blocks/

roman-khimov · 2025-02-03T15:57:16Z

src/Home.tsx

+							renderAs={Notification}
+							color={"gray"}
+						>
+							<Heading weight="semibold" subtitle style={{ textAlign: 'center' }}>Archive.NeoFS – Download Blockchain Data Snapshot</Heading>


It's not a blockchain data snapshot, this can be understood as "state snapshot". It is an offline synchronization package, block dump.

mike-petrov self-assigned this Jan 16, 2025

mike-petrov requested a review from roman-khimov as a code owner January 16, 2025 18:25

mike-petrov added 2 commits January 16, 2025 21:29

github: add build workflow

c2192cf

Signed-off-by: Mikhail Petrov <[email protected]>

dockerignore: add one

04c0f17

Follows send.fs.neo.org mostly. Signed-off-by: Mikhail Petrov <[email protected]>

mike-petrov force-pushed the feature/1-initial_version branch 2 times, most recently from 44cafcd to 422263c Compare January 23, 2025 16:22

mike-petrov force-pushed the feature/1-initial_version branch from ad9a26d to e5b9728 Compare January 23, 2025 16:41

roman-khimov reviewed Jan 26, 2025

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

src/Home.tsx Outdated Show resolved Hide resolved

mike-petrov force-pushed the feature/1-initial_version branch from e408862 to 6693f0f Compare January 31, 2025 21:45

mike-petrov added 3 commits February 1, 2025 00:50

makefile: add one

f354003

Follows send.fs.neo.org mostly. Signed-off-by: Mikhail Petrov <[email protected]>

readme: add one with logo

b89e19a

Signed-off-by: Mikhail Petrov <[email protected]>

web: Init first version with downloading snapshots, closes #1

979cde4

Signed-off-by: Mikhail Petrov <[email protected]>

mike-petrov force-pushed the feature/1-initial_version branch from 6693f0f to 979cde4 Compare January 31, 2025 21:52

mike-petrov requested a review from roman-khimov January 31, 2025 21:52

mike-petrov commented Jan 31, 2025

View reviewed changes

public/img/cover.png

Copy link

Author

mike-petrov Jan 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be updated when we're done with the content.

mike-petrov commented Jan 31, 2025

View reviewed changes

public/index.html Show resolved Hide resolved

mike-petrov commented Jan 31, 2025

View reviewed changes

src/Home.tsx Show resolved Hide resolved

roman-khimov reviewed Feb 3, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initial version #3

Initial version #3

mike-petrov commented Jan 16, 2025

mike-petrov commented Jan 16, 2025

mike-petrov commented Jan 16, 2025 •

edited

Loading

mike-petrov commented Jan 23, 2025

roman-khimov left a comment

mike-petrov Jan 31, 2025

mike-petrov Jan 31, 2025

roman-khimov Feb 3, 2025

mike-petrov commented Jan 31, 2025

roman-khimov Feb 3, 2025

roman-khimov Feb 3, 2025

roman-khimov Feb 3, 2025

roman-khimov Feb 3, 2025

roman-khimov Feb 3, 2025

roman-khimov Feb 3, 2025

roman-khimov Feb 3, 2025

roman-khimov Feb 3, 2025

Initial version #3

Are you sure you want to change the base?

Initial version #3

Conversation

mike-petrov commented Jan 16, 2025

mike-petrov commented Jan 16, 2025

mike-petrov commented Jan 16, 2025 • edited Loading

mike-petrov commented Jan 23, 2025

roman-khimov left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mike-petrov commented Jan 31, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mike-petrov commented Jan 16, 2025 •

edited

Loading