Topic 1 Question 110
You are creating a new pipeline in Google Cloud to stream IoT data from Cloud Pub/Sub through Cloud Dataflow to BigQuery. While previewing the data, you notice that roughly 2% of the data appears to be corrupt. You need to modify the Cloud Dataflow pipeline to filter out this corrupt data. What should you do?
Add a SideInput that returns a Boolean if the element is corrupt.
Add a ParDo transform in Cloud Dataflow to discard corrupt elements.
Add a Partition transform in Cloud Dataflow to separate valid data from corrupt data.
Add a GroupByKey transform in Cloud Dataflow to group all of the valid data together and discard the rest.
ユーザの投票
コメント(17)
Correct - B
👍 16[Removed]2020/03/22Answer: B Description: ParDo is used to do transformation and create side output
👍 11[Removed]2020/03/27Should be B. The Partition transform would require the element identifying the valid/invalid records for partitioning the pcollection that means there is some logic to be executed before the Partition transformation is invoked. That logic can be implemented in a ParDO transform and which can both identify valid/invalid records and also generate two PCollections one with valid records and other with invalid records.
👍 7SteelWarrior2020/09/22
シャッフルモード