StreamingFast Firehose data storage
Data and the locations where it is stored are important facets of Firehose deployment and operation.
Firehose Stores are abstractions sitting on top of Object Storage.
Merged blocks files are also referred to as
100-blocks files, and merged bundles. These terms are all used interchangeably within Firehose.
Up to one hundred blocks can be contained within a single 100-blocks file.
The 100-blocks files can include multiple versions such as a fork block or a given block number, ensuring continuity through the previous block link.
Nearly all components in Firehose rely on or utilize 100-blocks files. The bstream library consumes 100-blocks files for example.
Protocol-specific decoded block objects, like Ethereum, are what circulate amongst all processes that work with executed block data in Firehose.
One-block files are consumed by the
Mergercomponent, bundled in executed 100-blocks files. The one-block files are then stored to
dstorestorage and consumed by most of the other Firehose processes.