Study guides / CCAO-F / Domain 6

Governance, Risk, and Responsible Use · Lesson 1 of 5

6.1 — Appropriate and Inappropriate Use Cases

Distinguish business use cases where Claude appropriately assists a human decision-maker from fully autonomous, high-stakes use cases that remove human oversight — and recognise the ethical stakes for the people affected.

Not every task is a good fit for Claude, and not every fit is the same kind of fit. Some tasks are appropriate because a person stays in control of the outcome and Claude is simply making that person faster or better-informed. Others are inappropriate specifically because a consequential decision about a real person's life — their job, their credit, their eligibility for something — gets made with no human ever reviewing it before it takes effect. Learning to tell these apart, quickly and confidently, is the core skill this domain of the exam tests.

The question to ask is rarely "is this task technical" or "does this sound risky." It's narrower and more mechanical: does a human with real authority to change the outcome review this before it takes effect on a person? When the answer is yes, Claude assisting — even with something substantial, like drafting a termination letter or scoring loan applications — is appropriate. When the answer is no, the use case is inappropriate regardless of how accurate or well-intentioned the system is.

The Human-in-the-Loop Test, Applied to Business Decisions

Consider two ways a company might use Claude in hiring. In the first, Claude reads resumes against a job description and produces a short summary and a list of clarifying questions for a recruiter, who then decides personally which candidates move forward. In the second, Claude scores every applicant and automatically rejects anyone below a threshold, with no recruiter ever looking at the rejected applications. The first is appropriate: Claude accelerates the recruiter's work, but a person with the authority to say "wait, that's wrong" makes the actual call before the candidate is affected. The second is inappropriate: a person's job prospects are decided by a system with no one positioned to catch an error, a blind spot, or an unfair pattern before it affects them.

The same test applies outside hiring. A financial services team that has Claude flag transactions as suspicious for a human analyst to approve, deny, or escalate is using Claude appropriately — the analyst has both the information and the authority to override Claude's flag. A team that lets Claude autonomously approve or deny loan applications end-to-end, with no analyst reviewing individual decisions before applicants are notified, has crossed into inappropriate territory: nothing stands between a flawed judgment and a real financial consequence for a real applicant.

One detail matters as much as the review step itself: review has to happen before the decision takes effect, not after. A supervisor who spot-checks 5% of already-sent approval or denial notices is doing quality assurance on a process that already ran autonomously — it doesn't put a human between the decision and the person it affects.

Ethical Implications: Fairness and Transparency

Adding a human reviewer doesn't automatically make a use case ethically sound on its own — it's necessary, but two more things matter once review is in place, and both come directly from the blueprint's "ethical implications of AI usage" bullet.

The first is fairness. Claude can reflect patterns in the data or instructions it's given, and an AI-assisted process applied at scale can quietly encode a bias that would have been caught, one case at a time, by a purely human process. A reviewer who's just rubber-stamping Claude's output — glancing at a summary and clicking approve without the context or authority to meaningfully disagree — isn't providing the kind of oversight that catches this. Real human-in-the-loop review means the reviewer has enough information and enough actual power to change the outcome, not just a formality that satisfies a checkbox.

The second is transparency with the people affected. If Claude helped draft the denial letter a customer receives, or generated the interview questions a candidate is asked, that's usually fine to not mention explicitly. But if AI involvement is material to how someone would understand or contest a decision about them — for example, an applicant's ability to know a scoring system was involved at all, so they can meaningfully appeal — concealing that involvement is itself an ethical problem, separate from whether the underlying decision was accurate. Being honest about where AI was involved, when it would matter to the person on the other end, is part of using Claude responsibly, not an optional extra.

Key Concept

Appropriate use rests on three things together: a human with real authority reviews the consequential decision before it takes effect, the review is substantive rather than a rubber stamp, and the AI's involvement isn't concealed from people it would matter to. Fully autonomous high-stakes decisions — autonomous hiring/firing calls, autonomous financial approvals with no human step — fail the first test regardless of how well they otherwise perform.

Common Exam Distractor

Watch for scenarios that technically include a human somewhere in the process but not in a way that provides real oversight: a spot-check after decisions have already gone out, a reviewer with no authority or context to overrule Claude, or "review" that happens after the affected person has already been notified. These read as appropriate on a quick skim but fail the actual test — human review, with real authority, before the decision takes effect.

Exam traps

Practice question

A regional bank is evaluating three internal proposals for using Claude in its personal-loan process and can only pilot one this quarter. Which proposal describes an appropriate use case?

  • A Claude autonomously approves or denies small personal loan applications end-to-end, with no human reviewing individual decisions before applicants are notified

    This is a fully autonomous, high-stakes financial decision with no human review before the decision takes effect on the applicant — exactly the pattern the exam flags as inappropriate.

  • B Claude drafts a summary and risk flags for each application, which a loan officer with full authority to approve, deny, or request more information reviews before any decision is finalized Correct

    A human with real authority reviews the consequential decision before it affects the applicant — Claude is accelerating the loan officer's work, not replacing their judgment.

  • C Claude decides on loan terms for each application, and a compliance analyst reviews a sample of completed decisions after applicants have already been notified

    Review that happens after the applicant has already been notified doesn't put a human between the decision and the person it affects — it's after-the-fact quality assurance, not oversight of the decision itself.

  • D Claude autonomously sends approval or denial notices to applicants, and a supervisor spot-checks 5% of the notices already sent each week

    A low-rate spot-check on notices that have already gone out is not a review step before the decision takes effect — most applicants' outcomes were never reviewed by a human at all.

Build exercise: Evaluate a Business Use Case Against the Human-in-the-Loop Test

Beginner · 20 minutes

You'll practice:

  1. In a claude.ai Project, set up a hypothetical loan-officer assistant. Give Claude a short fictional applicant profile (income, requested amount, a couple of risk factors you invent) and ask it to produce a summary and a list of risk flags only — explicitly instruct it not to state an approve/deny recommendation. Then, in a separate message, write your own approve/deny decision as the human reviewer, referencing the flags Claude raised.

    This is the appropriate pattern from the lesson in miniature: Claude informs, a human with authority decides. Separating the two steps makes the human's role concrete rather than implicit.

    You should see: A neutral summary and risk-flag list from Claude with no decision embedded in it, followed by your own separate decision message that explicitly references specific flags.

    Hints
    1. If Claude's first response sneaks in a recommendation anyway, add 'do not recommend approve or deny — flag only' to your instruction and try again.
    2. Make at least one risk factor genuinely ambiguous so your own decision message has to do real judgment work, not just echo Claude.
    3. Notice that nothing in this exercise has 'happened' to a real applicant yet — that's the point of the review step.
  2. Write two short scenario descriptions of your own, one modeled on an appropriate lending use case and one on an inappropriate fully-autonomous one (you can vary the domain — hiring, financial approval, performance reviews). Ask Claude to evaluate which is appropriate and explain why, using the human-in-the-loop test from this lesson, then check its reasoning against your own.

    Constructing your own scenarios and checking Claude's evaluation against your own reasoning exercises the actual judgment skill the exam tests, rather than recognizing a pattern you've already seen worked out.

    You should see: Claude correctly identifies the autonomous, no-review scenario as inappropriate and names the missing human-review step specifically, not just a vague 'this seems risky.'

    Hints
    1. Make your 'inappropriate' scenario fail the test in a subtle way — e.g. include a spot-check after the fact — so you're testing precision, not an obvious case.
    2. If Claude's explanation doesn't mention timing (before vs. after the decision takes effect), push it with a follow-up question.
    3. Compare Claude's reasoning to the exam trap about token review steps — does it catch a rubber-stamp reviewer as still inappropriate?
  3. For your appropriate scenario, draft one short sentence a company could include in an applicant-facing notice, disclosing that AI assistance was used in preparing the review, without overstating or hiding its role. Check it against the fairness/transparency ideas from this lesson: does it give the applicant enough information to understand and, if needed, contest the decision?

    This exercises the transparency half of the ethical implications bullet directly — writing an honest disclosure is a different skill from just recognizing that transparency matters in the abstract.

    You should see: A one-sentence disclosure that names AI assistance in preparing the summary/flags without claiming the AI made the final decision, since a human did.

    Hints
    1. Avoid both extremes: don't bury the AI's role in vague language, and don't overstate it by implying Claude made the final call if a human actually did.
    2. Read your sentence as if you were the applicant — would it change how you'd approach an appeal or question about the decision?
    3. If you're unsure whether disclosure is even necessary for a given case, ask whether the AI's involvement would matter to someone trying to understand or contest the outcome — that's the test from this lesson.

Sources