v2.3.4

• Fix JSON decoding in the client tools (firehose-client, print merged-blocks, etc.).

v2.3.3

Known issues

• The block decoding to JSON is broken in the CLI tools (firehose-client, print merged-blocks, etc.). Use version v2.3.1 for those tools

Hotfix

• Fix block poller panic on v2.3.2

v2.3.2

Auth and metering

• Add missing metering events for sf.firehose.v2.Fetch/Block responses.
• Changed default polling interval in 'continuous authentication' from 10s to 60s, added 'interval' query param to URL.

Substreams

• Fixed bug in scheduler ramp-up function sometimes waiting before raising the number of workers
• Fixed load-balancing from tier1 to tier2 when using dns:/// (round-robin policy was not set correctly)
• Added trace_id in grpc authentication calls
• Bumped connect-go library to new "connectrpc.com/connect" location

v2.3.1

Operators

• Firehose blocks that were produced using the RPC Poller will have to be extracted again to fix the Transaction Status and the potential missing receipt (ex: arb-one pre-nitro, Avalanche, Optimism ...)

Fixes

• Fix race condition in RPC Poller which would cause some missing transaction receipts
• Fix conversion of transaction status from RPC Poller: failed transactions would show up as "status unknown" in firehose blocks.

Added

• Added the support the FORCE_FINALITY_AFTER_BLOCKS environment variable: setting it to a value like '200' will make the 'reader' mark blocks as final after a maximum of 200 block confirmations, even if the chain implements finality via a beacon that lags behind.

v2.3.0

• Reduce logging and logging "payload".

• Tools printing Firehose Block model to JSON now have --proto-paths take higher precedence over well-known types and even the chain itself, the order is --proto-paths > chain > well-known (so well-known is lookup last).

• The tools print one-block now works correctly on blocks generated by omni-chain firecore binary.

• The various health endpoint now sets Content-Type: application/json header prior sending back their response to the client.

• The firehose, substreams-tier1 and substream-tier2 health endpoint now respects the common-system-shutdown-signal-delay configuration value meaning that the health endpoint will return false now if SIGINT has been received but we are still in the shutdown unready period defined by the config value. If you use some sort of load balancer, you should make sure they are configured to use the health endpoint and you should common-system-shutdown-signal-delay to something like 15s.

• Changed reader logger back to reader-node to fit with the app's name which is reader-node.

• Fix tools compare-blocks that would fail on new format.

• Fix substreams to correctly delete .partial files when serving a request that is not on a boundary

v2.2.2

The Cancun hard fork happened on Goerli and after further review, we decided to change the Protobuf definition for the new BlockHeader, Transaction and TransactionReceipt fields that are related to blob transaction.

We made explicit that those fields are optional in the Protobuf definition which will render them in your language of choice using the appropriate "null" mechanism. For example on Golang, those fields are generated as BlobGasUsed *uint64 and ExcessBlobGas *uint64 which will make it clear that those fields are not populated at all.

The affected fields are:

• BlockHeader.blob_gas_used, now optional uint64.
• BlockHeader.excess_blob_gas, now optional uint64.
• TransactionTrace.blob_gas, now optional uint64.
• TransactionTrace.blob_gas_fee_cap, now optional BigInt.
• TransactionReceipt.blob_gas_used, now optional uint64.
• TransactionReceipt.blob_gas_price, now optional BigInt.

This is technically a breaking change for those that could have consumed those fields already but we think he impact is so minimal that it's better to make the change right now.

Operators

You will need to reprocess a small Goerli range. You should update to new version to produce the newer version and the reprocess from block 10377700 up to when you upgraded to v2.2.2.

The block 10377700 was chosen since it is the block at the time of the first release we did supporting Cancun where we introduced those new field. If you know when you deploy either v2.2.0 or v2.2.1, you should reprocess from that point.

An alternative to reprocessing is updating your blocks by having a StreamingFast API Token and using fireeth tools download-from-firehose goerli.eth.streamingfast.io:443 -a SUBSTREAMS_API_TOKEN 10377700:<recent block rounded to 100s> <destination>.

Note

You should download the blocks to a temporary destination and copy over to your production destination once you have them all.

You can reach to us on Discord if you need help on something.

v2.2.1

• Updated the documentation for some of the upcoming new Cancun hard-fork fields:
  • TransactionTrace.blob_gas
  • TransactionReceipt.blob_gas_used
  • TransactionReceipt.blob_gas_price

v2.2.0

Important

Operators running Goerli chain will need to upgrade to this version, with this geth node release: https://github.com/streamingfast/go-ethereum/releases/tag/geth-v1.13.10-fh2.4

Support for Dencun fork (Goerli: Jan 17th)

• Added support for EIP-4844 (upcoming with activation of Dencun fork), through instrumented go-ethereum nodes with version fh2.4. This adds new fields in the Ethereum Block model, fields that will be non-empty when the Ethereum network your pulling have EIP-4844 activated. The fields in questions are:
  • Block.system_calls
  • BlockHeader.blob_gas_used
  • BlockHeader.excess_blob_gas
  • BlockHeader.parent_beacon_root
  • TransactionTrace.blob_gas
  • TransactionTrace.blob_gas_fee_cap
  • TransactionTrace.blob_hashes
  • TransactionReceipt.BlobGasUsed
  • TransactionReceipt.BlobGasPrice
  • A new TransactionTrace.Type value TRX_TYPE_BLOB

Substreams server (bumped to v1.3.1)

• Fixed error-passing between tier2 and tier1 (tier1 will not retry sending requests that fail deterministicly to tier2)
• Tier1 will now schedule a single job on tier2, quickly ramping up to the requested number of workers after 4 seconds of delay, to catch early exceptions
• "store became too big" is now considered a deterministic error and returns code "InvalidArgument"

Misc

• Added tools poller generic-evm subcommand. It is identical to optimism/arb-one in feature at the moment and should work for most evm chains.

v2.1.0

• Bump to major release firehose-core v1.0.0

Operators

Important

When upgrading your stack to this release, be sure to upgrade all components simultaneously because the block encapsulation format has changed.
There is no simple way to revert, except by deleting the all the one-blocks and merged-blocks that were produced with this version.
Blocks that are merged using the new merger will not be readable by previous versions.
Blocks that are sent on the wire (ex: through the relayer) with the previous version will not be readable by the new components (ex: firehose)
This will require either a "Stop the world" upgrade, or a very careful execution of the following steps:

1. stop merger
2. start a new reader (making sure it is not picked up by the relayer
3. stop relayer+firehose+substreams (DOWNTIME STARTS HERE)
4. launch new version of relayer+firehose+substreams that connects to the new reader (DOWNTIME ENDS HERE)
5. start new merger
6. replace other components like index-builder

Changed

• Blocks files (one-blocks and merged) are now stored with a new format using google.protobuf.any format. Previous blocks can still be read and processed.

Added

• Added RPC pollers for Optimism and Arb-one: These can be used from by running the reader-node with --reader-node-path=/path/to/fireeth and --reader-node-arguments="tools poller {optimism|arb-one} [flags...]" (see additionnal flags by running fireeth tools poller optimism --help)
• Added tools fix-any-type to rewrite the previous merged-blocks (OPTIONAL)

v2.0.2

• Fixed grpc error code when shutting down: changed from Canceled to Unavailable