Comment on page
StreamingFast Firehose data storage
Data and the locations where it is stored are important facets of Firehose deployment and operation.
Firehose Stores are abstractions sitting on top of Object Storage.
Note: Object Storage is a data storage technique that manages data as objects in opposition to other data storage architectures like hierarchical file systems.
Merged blocks files are also referred to as
100-blocks files, and merged bundles. These terms are all used interchangeably within Firehose.
Up to one hundred blocks can be contained within a single 100-blocks file.
The 100-blocks files can include multiple versions such as a fork block or a given block number, ensuring continuity through the previous block link.
Nearly all components in Firehose rely on or utilize 100-blocks files. The bstream library consumes 100-blocks files for example.
Protocol-specific decoded block objects, like Ethereum, are what circulate amongst all processes that work with executed block data in Firehose.
Important: One-block files contain only one
bstream.Blockas a serialized protocol buffer.
One-block files are consumed by the
Mergercomponent, bundled in executed __ 100-blocks files. The one-block files are then stored to
dstorestorage and consumed by most of the other Firehose processes.