Topic 1 Question 88
A social media company wants to use a large language model (LLM) to summarize messages. The company has chosen a few LLMs that are available on Amazon SageMaker JumpStart. The company wants to compare the generated output toxicity of these models.
Which strategy gives the company the ability to evaluate the LLMs with the LEAST operational overhead?
Crowd-sourced evaluation
Automatic model evaluation
Model evaluation with human workers
Reinforcement learning from human feedback (RLHF)
ユーザの投票
コメント(4)
- 正解だと思う選択肢: B
automatic model evlauation Automatic model evaluation refers to the process of assessing the performance of a machine learning model using predefined metrics and techniques without manual intervention. This process is crucial for understanding how well a model performs and identifying areas for improvement. Here are some key components and methods used in automatic model evaluation:
👍 126b8fe12024/12/26 - 正解だと思う選択肢: B
B. Automatic model evaluation
Explanation: Using automatic model evaluation is the most efficient and low-overhead approach to evaluate the toxicity of the generated outputs from different LLMs. This strategy involves using automated tools or frameworks designed to assess the toxicity, bias, or other quality metrics of the model outputs, which minimizes operational overhead compared to manual methods.
👍 1aws_Tamilan2024/12/27 - 正解だと思う選択肢: B
The correct answer is B. Automatic model evaluation requires minimal human intervention, making it operationally lighter than human-based approaches.
👍 1may2021_r2024/12/28
シャッフルモード