Topic 1 Question 193
An aerospace company uses a proprietary data format to store its flight data. You need to connect this new data source to BigQuery and stream the data into BigQuery. You want to efficiently import the data into BigQuery while consuming as few resources as possible. What should you do?
Write a shell script that triggers a Cloud Function that performs periodic ETL batch jobs on the new data source.
Use a standard Dataflow pipeline to store the raw data in BigQuery, and then transform the format later when the data is used.
Use Apache Hive to write a Dataproc job that streams the data into BigQuery in CSV format.
Use an Apache Beam custom connector to write a Dataflow pipeline that streams the data into BigQuery in Avro format.
ユーザの投票
コメント(11)
This has to be D. How could it even be B? The source is a proprietary format. Dataflow wouldn't have a built-in template to ead the file. You will have to create something custom.
👍 12beanz002022/10/31- 正解だと思う選択肢: D
For me it's clearly D It's between B and D, but read B, store raw data in Big Query? Use a Dataflow pipeline just to store raw data into Big Query, and transform later? You'd need to do another pipeline for that, and is not efficient.
👍 10devaid2022/10/17 - 正解だと思う選択肢: D
D. Use an Apache Beam custom connector to write a Dataflow pipeline that streams the data into BigQuery in Avro format. Reveal Solution
👍 3AWSandeep2022/09/02
シャッフルモード