v1.4.5

• Updated metering (bumped versions of dmetering, dauth, and firehose libraries.)
• Fixed firehose service healthcheck on shutdown
• Fixed panic on download-blocks-from-firehose tool

v1.4.4

Operators

• When upgrading a substreams server to this version, you should delete all existing module caches to benefit from deterministic output

Substreams changes

• Switch default engine from wasmtime to wazero
• Prevent reusing memory between blocks in wasm engine to fix determinism
• Switch our store operations from bigdecimal to fixed point decimal to fix determinism
• Sort the store deltas from DeletePrefixes() to fix determinism
• Implement staged module execution within a single block.
• "Fail fast" on repeating requests with deterministic failures for a "blacklist period", preventing waste of resources
• SessionInit protobuf message now includes resolvedStartBlock and MaxWorkers, sent back to the client

v1.4.3

Highlights

• This release brings an update to substreams to v1.1.4 which includes the following:
  • Changes the module hash computation implementation to allow reusing caches accross substreams that 'import' other substreams as a dependency.
  • Faster shutdown of requests that fail deterministically
  • Fixed memory leak in RPC calls

Note for Operators

Note This upgrade procedure applies to you if your Substreams deployment topology includes both tier1 and tier2 processes. If you have defined somewhere the config value substreams-tier2: true, then this applies to you, otherwise, if you can ignore the upgrade procedure.

• The components should be deployed simultaneously to tier1 and tier2, or users will end up with backend error(s) saying that some partial file are not found. These errors will be resolved when both tiers are upgraded.

Added

• Added Substreams scheduler tracing support. Enable tracing by setting the ENV variables SF_TRACING to one of the following:
  • stdout://
  • cloudtrace://[host:port]?project_id=<project_id>&ratio=<0.25>
  • jaeger://[host:port]?scheme=<http|https>
  • zipkin://[host:port]?scheme=<http|https>
  • otelcol://[host:port]

v1.4.2

Highlights

• This release brings an update to substreams to v1.1.3 which includes the following:
  • Fixes an important bug that could have generated corrupted store state files. This is important for developers and operators.
  • Fixes for race conditions that would return a failure when multiple identical requests are backprocessing.
  • Fixes and speed/scaling improvements around the engine.

Note for Operators

Note This upgrade procedure is applies if your Substreams deployment topology includes both tier1 and tier2 processes. If you have defined somewhere the config value substreams-tier2: true, then this applies to you, otherwise, if you can ignore the upgrade procedure.

This release includes a small change in the internal RPC layer between tier1 processes and tier2 processes. This change requires an ordered upgrade of the processes to avoid errors.

The components should be deployed in this order:

1. Deploy and roll out tier1 processes first
2. Deploy and roll out tier2 processes in second

If you upgrade in the wrong order or if somehow tier2 processes start using the new protocol without tier1 being aware, user will end up with backend error(s) saying that some partial file are not found. Those will be resolved only when tier1 processes have been upgraded successfully.

v1.4.1

Fixed

• Substreams running without a specific tier2 substreams-client-endpoint will now expose tier2 service sf.substreams.internal.v2.Substreams so it can be used internally.

Warning
If you don't use dedicated tier2 nodes, make sure that you don't expose sf.substreams.internal.v2.Substreams to the public (from your load-balancer or using a firewall)

Breaking changes

• flag substreams-partial-mode-enabled renamed to substreams-tier2
• flag substreams-client-endpoint now defaults to empty string, which means it is its own client-endpoint (as it was before the change to protocol V2)

v1.4.0

Substreams RPC protocol V2

Substreams protocol changed from sf.substreams.v1.Stream/Blocks to sf.substreams.rpc.v2.Stream/Blocks for client-facing service. This changes the way that substreams clients are notified of chain reorgs.
All substreams clients need to be upgraded to support this new protocol.

See https://github.com/streamingfast/substreams/releases/tag/v1.1.1 for details.

Added

• firehose-client tool now accepts --limit flag to only send that number of blocks. Get the latest block like this: fireeth tools firehose-client <endpoint> --limit=1 -- -1 0

v1.3.8

Highlights

This is a bug fix release for node operators that are about to upgrade to Shanghai release. The Firehose instrumented geth compatible with Shanghai release introduced a new message CANCEL_BLOCK. It seems in some circumstances, we had a bug in the console reader that was actually panicking but the message was received but no block was actively being assembled.

This release fix this bogus behavior by simply ignoring CANCEL_BLOCK message when there is no active block which is harmless. Every node operators that upgrade to https://github.com/streamingfast/go-ethereum/releases/tag/geth-v1.11.5-fh2.2 should upgrade to this version.

Note There is no need to update the Firehose instrumented geth binary, only fireeth needs to be bumped if you already are at the latest geth version.

Fixed

• Fixed a bug on console reader when seeing CANCEL_BLOCK on certain circumstances.

Changed

• Now using Golang 1.20 for building releases.

• Changed default value of flag substreams-sub-request-block-range-size from 1000 to 10000.

v1.3.7

Fixed

• Fixed a bug in data normalization for Polygon chain which would cause panics on certain blocks

Added

• support for gcp 'archive' types of snapshots

v1.3.6

Highlights

• This release implements the new CANCEL_BLOCK instruction from Firehose protocol 2.2 (fh2.2), to reject blocks that failed post-validation.
• This release fixes polygon "StateSync" transactions by grouping the calls inside an artificial transaction.

If you had previous blocks from a Polygon chain (bor), you will need to reprocess all your blocks from the node because some StateSync transactions may be missing on some blocks.

Operators

This release now supports the new Firehose node exchange format 2.2 which introduced a new exchanged message CANCEL_BLOCK. This has an implication on the Firehose instrumented Geth binary you can use with the release.

• If you use Firehose instrumented Geth binary tagged fh2.2 (like geth-v1.11.4-fh2.2-1), you must use firehose-ethereum version >= 1.3.6
• If you use Firehose instrumented Geth binary tagged fh2.1 (like geth-v1.11.3-fh2.1), you can use firehose-ethereum version >= 1.0.0

New releases of Firehose instrumented Geth binary for all chain will soon all be tagged fh2.2, so upgrade to >= 1.3.6 of firehose-ethereum will be required.

v1.3.5

Highlights

This release is required if you run on Goerli and is mostly about supporting the upcoming Shangai fork that has been activated on Goerli on March 14th.

Changed

• Added support for withdrawal balance change reason in block model, this is required for running on most recent Goerli Shangai hard fork.
• Added support for withdrawals_root on Header in the block model, this will be populated only if the chain has activated Shangai hard fork.
• --substreams-max-fuel-per-block-module will limit the number of wasmtime instructions for a single module in a single block.