Topic 2 Question 41
Which Cloud Dataflow / Beam feature should you use to aggregate data in an unbounded data source every hour based on the time when the data entered the pipeline?
An hourly watermark
An event time trigger
The with Allowed Lateness method
A processing time trigger
解説
When collecting and grouping data into windows, Beam uses triggers to determine when to emit the aggregated results of each window. Processing time triggers. These triggers operate on the processing time the time when the data element is processed at any given stage in the pipeline. Event time triggers. These triggers operate on the event time, as indicated by the timestamp on each data element. Beams default trigger is event time-based. Reference: https://beam.apache.org/documentation/programming-guide/#triggers
コメント(4)
Answer: D Description: Process on time
👍 6[Removed]2020/03/28D. The AfterProcessingTime trigger operates on processing time. For example, the AfterProcessingTime.pastFirstElementInPane() trigger emits a window after a certain amount of processing time has passed since data was received. The processing time is determined by the system clock, rather than the data element’s timestamp.
👍 6Callumr2020/06/24D: A related concept, called triggers, determines when to emit the results of aggregation as unbounded data arrives. You can use triggers to refine the windowing strategy for your PCollection. Triggers allow you to deal with late-arriving data or to provide early results. See the triggers section for more information. https://beam.apache.org/documentation/programming-guide
👍 4daghayeghi2021/03/03
シャッフルモード