Topic 1 Question 190
A data engineer wants to orchestrate a set of extract, transform, and load (ETL) jobs that run on AWS. The ETL jobs contain tasks that must run Apache Spark jobs on Amazon EMR, make API calls to Salesforce, and load data into Amazon Redshift.
The ETL jobs need to handle failures and retries automatically. The data engineer needs to use Python to orchestrate the jobs.
Which service will meet these requirements?
Amazon Managed Workflows for Apache Airflow (Amazon MWAA)
AWS Step Functions
AWS Glue
Amazon EventBridge
ユーザの投票
コメント(2)
- 正解だと思う選択肢: A
Even though both MWAA and Step functions can be used for managing task failures, MWAA is more suitable since the engineer would like to use python to orchestrate jobs. Usually, Step functions is used for minimal infrastructure management
👍 7Eleftheriia2024/11/07 Correct Answer: B
AWS Step Functions allows you to build serverless workflows that coordinate various AWS services. It supports integrating with EMR for running Spark jobs, making API calls (including to Salesforce), and loading data into Redshift. Step Functions provide built-in error handling and retry capabilities, making it easier to manage failures in your workflow. Additionally, you can use AWS SDK for Python (Boto3) to interact with Step Functions, enabling you to write your orchestration logic in Python.
👍 1kupo7772024/11/04
シャッフルモード