High Availability
High availability for StreamingFast Firehose components
Coming soon.
Placing multiple Reader components side by side, and fronted by one or more Relayers, allows for highly available setups; a core attribute of the Firehose design.
A Relayer connected to multiple Readers will deduplicate the incoming stream and push the first block downstream.
Tip: Two Reader components will even race to push the data first. The system is designed to leverage this racing Reader feature to the benefit of the end-user by producing the lowest latency possible.
Firehose also aggregates any forked blocks that would be seen by a single Reader component, and not seen by any other Reader components.
Highly available systems usually connect to the Relayer component to receive real-time blocks. Merged blocked files are used when Relayer components can't provide the requested data or satisfy a range.
Restarts from other components can be sustained and time provided for Merger components to be down when Relayer components provide 200 to 300 blocks in RAM.
Note: Merged blocks generally aren't read by other Firehose components in a running, live highly available system.
A Relayer component in a highly available Firehose will feed from all of the Reader nodes to gain a complete view of all possible forks.
Firehose can be scaled horizontally to provide a highly available system.
The network speed and data throughput between consumers and Firehose deployments will dictate the speed of data availability.
Note: The network speed and data throughput between Relayer components and Firehose gRPC Server components will impact the speed of data availability.
Firehose gRPC Server components have the ability to connect to a subset of Relayer components or all Relayers available.
When the Firehose gRPC Server component is connected to all available Relayer components the probability that all forks will be viewed increases. Inbound requests made by consumers will be fulfilled with in-memory fork data.
Block navigation can be delayed when forked data isn't completely communicated to the Firehose gRPC Server component.
Understanding how data flows through Firehose is beneficial for harnessing its full power.
Last modified 1mo ago