Free AWS AI Practitioner Practice Exam

Q1.A company is evaluating LLM outputs for a medical information application and wants to understand whether responses are accurate, empathetic, and appropriate for patients before deploying. They are considering using automated metrics but a clinical expert argues that certain qualities cannot be captured by automated scoring. When is human evaluation of LLM outputs most important?

• A.Evaluating subjective qualities like empathy and appropriateness that automated metrics cannot capture
• B.Evaluating high-stakes outputs in domains like healthcare where errors could cause patient harm
• C.Evaluating the inference latency of model API responses under load
• D.Evaluating the cost per API call across different model providers

Domain: Applications of Foundation Models

Q2.A security operations team is responsible for monitoring a production AI inference environment that processes thousands of model invocations per hour. They want to detect anomalies such as sudden spikes in invocation volume, unusual geographic origins of requests, or attempts to extract sensitive information through repeated probing. Which AWS service is most appropriate for setting up this behavioral monitoring and anomaly detection?

• A.Amazon CloudWatch metrics and alarms
• B.Amazon Macie automated PII discovery
• C.AWS Config configuration compliance rules
• D.Amazon Detective security investigation

Domain: Security, Compliance, and Governance for AI Solutions

Q3.A company is evaluating several third-party AI vendors to power a new customer analytics platform. The procurement team wants to ensure the selected vendor meets the company's responsible AI standards before signing a contract. Which of the following should the team evaluate when assessing a third-party AI provider's responsible AI practices?

• A.Reviewing the vendor's transparency documentation on model training, data, and known limitations
• B.Assessing whether the vendor conducts bias testing and fairness evaluations of their AI models
• C.Confirming that the vendor's API response latency consistently meets a 100-millisecond threshold
• D.Verifying that the vendor's headquarters is located in the same country as the procuring company

Domain: Guidelines for Responsible AI

Q4.A data scientist is evaluating a binary classification model that predicts whether a loan applicant will default. The team wants a single metric that measures the model's ability to distinguish between defaulters and non-defaulters across all possible classification thresholds, regardless of which threshold is chosen. Which metric should the data scientist use?

• A.AUC-ROC (Area Under the Receiver Operating Characteristic curve)
• B.Accuracy
• C.F1 score
• D.RMSE (Root Mean Squared Error)

Domain: Fundamentals of AI and ML

Q5.A developer is tuning a foundation model's inference parameters to control the diversity of generated text. They want to limit the model so it only considers the smallest set of top-ranked tokens whose cumulative probability exceeds 90% at each generation step, cutting off the long tail of unlikely words. Which inference parameter controls this behavior?

• A.Top-p (nucleus sampling)
• B.Temperature
• C.Top-k
• D.Max tokens

Domain: Fundamentals of Generative AI

Q6.A company wants to align a foundation model's outputs to be more helpful, honest, and harmless by using human preference data. They collect pairs of model responses and have human raters indicate which response they prefer. The preferred responses are then used to train a separate model that scores outputs, and this score guides further model training. Which technique does this describe?

• A.Reinforcement Learning from Human Feedback (RLHF)
• B.Constitutional AI
• C.Direct Preference Optimization (DPO)
• D.Supervised fine-tuning on expert-written exemplar responses

Domain: Applications of Foundation Models

Q7.A company has a central AI platform team in Account A that manages foundation model deployments. Individual business unit teams in Accounts B, C, and D need to invoke Amazon Bedrock models managed in Account A without embedding long-term credentials in their applications. What is the recommended approach for implementing this cross-account access pattern?

• A.IAM roles in Account A with trust policies allowing Account B, C, and D to assume them via STS
• B.Storing Account A IAM access keys in AWS Secrets Manager in each business unit account
• C.Deploying separate Amazon Bedrock model endpoints in each business unit account
• D.Using a shared IAM root account user with access keys distributed to all teams

Domain: Security, Compliance, and Governance for AI Solutions

Q8.A team is analyzing a dataset used to train a medical diagnosis model and notices that certain rare diseases appear far more frequently in the training data than their true prevalence in the general population because academic papers disproportionately report on unusual or novel cases. Which type of bias does this data collection problem represent?

• A.Reporting bias
• B.Selection bias
• C.Label bias
• D.Availability heuristic bias

Domain: Guidelines for Responsible AI

Q9.A data scientist is evaluating a multi-class classification model that categorizes support tickets into five categories. Two categories have very few examples while three have many. The data scientist wants a metric that treats each class equally regardless of how many examples it has. Which averaging method should be used when computing F1 score?

• A.Macro averaging
• B.Micro averaging
• C.Weighted averaging
• D.Simple averaging

Domain: Fundamentals of AI and ML

Q10.A development team is designing a generative AI application that needs to process documents uploaded by users and return analysis results. The uploads happen sporadically and volume is unpredictable. They want the architecture to scale automatically and only process documents when they arrive. Which architecture pattern best fits this use case?

• A.Event-driven architecture (triggered by upload events via S3 and EventBridge or SQS)
• B.Synchronous request-response architecture
• C.Scheduled batch processing architecture
• D.Polling-based architecture

Domain: Fundamentals of Generative AI