14 free sample questions in the same style as the live exam. Answers and explanations are revealed as you go — nothing here is scored.
Answered 0 of 14
0 correct
1. Cross-validation is primarily used to:
Foundation
Machine Learning Fundamentals for Assurance
2. Transfer learning is most accurately described as:
Intermediate
Machine Learning Fundamentals for Assurance
3. Reporting model performance from only the best of 20 randomly seeded runs, without disclosure, primarily risks:
Advanced
Model Evaluation and Benchmarking
4. F1 score is the:
Foundation
Model Evaluation and Benchmarking
5. Predictive parity is satisfied when:
Intermediate
Fairness and Bias Measurement
6. Intersectional fairness auditing is important because:
Advanced
Fairness and Bias Measurement
7. A jailbreak attempt against an LLM is best defined as:
Foundation
Robustness and LLM Red-Teaming
8. Automated red-teaming tools (e.g., using another LLM to generate adversarial prompts) are primarily valuable because they:
Intermediate
Robustness and LLM Red-Teaming
9. Disclosing a model's 'out-of-scope use' in a model card serves to:
Foundation
Documentation and Transparency Artifacts
10. When auditing a vendor-supplied model with no accompanying model card, the most appropriate assurance response is to:
Advanced
Documentation and Transparency Artifacts
11. A sudden drop in a deployed model's live accuracy, with no corresponding change in the model itself, is most likely explained by:
Intermediate
MLOps Monitoring, Drift, and Governance
12. US federal guidance (e.g., OMB memoranda) on agency use of AI generally requires agencies to:
Advanced
MLOps Monitoring, Drift, and Governance
13. CASE STUDY — Waller County Screening reports 99.2% accuracy for a rare-disease classifier where only 0.5% of patients are positive. Clinicians report it almost never flags anyone. What is the problem?
Intermediate
Model Evaluation and Benchmarking
14. CASE STUDY — Colorado River Staffing removes gender from its resume-ranking model, yet outcomes remain skewed. Investigation shows features like sports club membership and college name correlate with gender. What does this illustrate?
Intermediate
Fairness and Bias Measurement
Ready for the real thing?
The live CAIA exam has 29 questions, runs for 75 minutes, and needs 80% to pass.
