Struct timely::progress::nested::subgraph::Subgraph

pub struct Subgraph<TOuter: Timestamp, TInner: Timestamp> {
    pub path: Vec<usize>,
    // some fields omitted
}

A dataflow subgraph.

The subgraph type contains the infrastructure required to describe the topology of and track progress within a dataflow subgraph.

Fields

path: Vec<usize>

Path of identifiers from the root.

Trait Implementations

impl<TOuter: Timestamp, TInner: Timestamp> Operate<TOuter> for Subgraph<TOuter, TInner>

fn name(&self) -> String

A descripitive name for the operator

fn local(&self) -> bool

Indicates if the operator is strictly local to this worker.

fn inputs(&self) -> usize

The number of inputs.

fn outputs(&self) -> usize

The number of outputs.

fn get_internal_summary(
    &mut self
) -> (Vec<Vec<Antichain<TOuter::Summary>>>, Vec<ChangeBatch<TOuter>>)

Fetches summary information about internal structure of the operator.

fn set_external_summary(
    &mut self,
    summaries: Vec<Vec<Antichain<TOuter::Summary>>>,
    frontier: &mut [ChangeBatch<TOuter>]
)

Receive summaries from outputs to inputs, as well as initial external capabilities on inputs.

This method finalizes the internal reachability of this Subgraph, and provides the corresponding information on to each of its children.

fn push_external_progress(&mut self, external: &mut [ChangeBatch<TOuter>])

Receive changes in the external capabilities of the containing scope.

This method currently also signals the receipt of all previously expressed produced buffers, whose implications must be folded into the external buffer before this method is called. This method can be thought of as transactionally stating "your output messages have been consumed, and the following capabilities produced", which everywhere else in the computation must be atomically updated. It is no different here, which is a serious constraint that happens to be true mostly by good luck.

fn pull_internal_progress(
    &mut self,
    consumed: &mut [ChangeBatch<TOuter>],
    internal: &mut [ChangeBatch<TOuter>],
    produced: &mut [ChangeBatch<TOuter>]
) -> bool

Report changes in messages and capabilities for the subgraph and its peers.

This method populates its arguments with accumulated changes across all of its peers, indicating input messages consumed, internal capabilities held or dropped, and output messages produced.

Importantly, these changes are aggregated across all peers, reflecting the current information received by the operator from its own internal progress tracking. This has the potential to lead to confusing conclusions (from experience), and should be treated carefully and with as much rigor as possible. This behavior is indicated to others by the self.local() method returning false.

fn notify_me(&self) -> bool

Indicates of whether the operator requires push_external_progress information or not.