PPassTheCertExam

Study guides / CCAO-F / Domain 4

Workflow Integration and Solution Design · Lesson 4 of 6

4.4 — Designing and Iterating on a Claude-Based Solution

Prototype a workflow solution with Claude, such as a Project set up for a recurring task, and iterate based on real output across several runs — not a single test.

Designing a Claude-based solution for a recurring business task is workflow-level work, distinct from tuning a single prompt (that's domain 1 territory). The unit you're designing and iterating on isn't one message — it's a reusable setup, most commonly a Project in claude.ai with custom instructions, relevant reference documents, and a consistent structure for the recurring task, that a person will return to and reuse across many real instances of the same job: weekly status reports, recurring vendor evaluations, monthly board summaries, a standing intake-triage task.

Prototype First, at Small Scale

Start by setting up a Project with a first-pass system prompt describing the role and the recurring task, add the two or three reference documents most central to getting it right (a style guide, a template, a past example of good output), and run it against one real, representative instance of the task — not a toy example, an actual piece of this month's real work. This first run is a prototype, not a finished solution: its purpose is to surface where the setup breaks down against real complexity, not to prove the idea works in principle.

Iterate Across Several Real Runs, Not One Test

This is the part that's easy to skip under time pressure: a single successful run doesn't tell you the solution is ready. Real recurring tasks have variation — different data shapes, different edge cases, different levels of messiness in the source material — and a Project's instructions that work perfectly on one clean example can fail on the next one that has a slightly different structure. The discipline is to run the Project against three to five real instances of the task across its actual natural cadence (if it's a weekly report, that's three to five actual weeks, not the same week's data copied three times), review each output critically, and only then decide whether the Project's instructions need revision.

When a run produces a bad or off output, the diagnostic question is the same one from domain 1: is this a missing role/constraint problem, a missing-example problem, or a missing-structure problem in what you gave Claude? But now you're asking it at the level of "does this Project need a new reference document, an added rule in its instructions, or an example of a tricky edge case added to it" — workflow-level revisions to a reusable setup, not a one-off prompt edit that disappears after a single conversation.

Key Concept

Iterating on a Claude-based solution means revising a reusable setup (most often a Project's instructions and reference documents) based on patterns you see across three to five real runs of a recurring task — not declaring success after one clean test, and not tuning a single prompt in isolation.

Common Exam Distractor

Watch for a scenario where someone tests a Project once on a clean, easy example, gets a good result, and immediately rolls it out to the whole team as finished. One successful test on convenient data doesn't validate a workflow solution — it's the same trap as trusting a single unrepresentative research question in lesson 4.2 or a single unreviewed run of any process. Real variation only shows up across multiple genuine runs.

Exam traps

Practice question

A team sets up a Project to draft weekly customer success reports. They test it once against last week's (unusually clean and simple) data, the output looks great, and they immediately roll it out to the whole team as the new standard process. What is the most likely problem with this rollout?

  • A There's no real problem — one successful test on real data is sufficient validation for a recurring Project-based solution

    One clean test doesn't reveal how the Project handles the variation real weekly data will actually have. Several runs across genuinely different real instances are needed before rolling out.

  • B The team should have used a single, more complex synthetic example instead of a real one for their one test

    The problem isn't the realism of the single example — it's that only one run was tested at all. A synthetic example wouldn't fix the core issue of insufficient iteration across real variation.

  • C The team validated on only one, unusually clean run and should instead have tested against three to five genuinely different real weeks before rolling it out, revising the Project's instructions or reference documents based on what those runs reveal Correct

    This correctly identifies both the problem (a single unrepresentative test) and the fix (iterating across multiple real runs and revising the reusable Project setup, not just the one output).

  • D The team should abandon the Project-based approach and instead write a brand new prompt from scratch every single week

    Rebuilding from scratch each week discards the reusability that makes a Project valuable, and doesn't address the actual issue, which is insufficient testing before rollout, not the reusable-setup approach itself.

Build exercise: Prototype and Iterate a Recurring-Task Project

Intermediate · 30 minutes

You'll practice:

  1. In claude.ai, create a new Project for a recurring task you actually do (a weekly update, a recurring review, a standing intake/triage task — invent a realistic one if needed). Write first-pass custom instructions describing the role and the task, and add one reference document if you have something suitable (a template, a style guide, a past example).

    This sets up the reusable unit you'll iterate on — the exam tests workflow-level design, not a single prompt, so the Project itself is the artifact under test.

    You should see: A working Project with custom instructions saved and at least one reference document attached.

    Hints 
    1. Keep the first-pass instructions reasonably short — you'll be revising them based on real output, so don't over-engineer them yet.
    2. Pick a task with enough real variation week to week that iteration will actually matter.
    3. If you don't have a real reference document handy, a rough template you write yourself is fine for this exercise.

  2. Run the Project against three genuinely different real (or realistic) instances of the task — three different weeks' worth of notes, three different intake requests, etc. — not the same input repeated. Review each output critically and note specifically where each one falls short.

    This is the core discipline the exam tests: validating across several real runs, not one clean test, to surface the variation a single example would hide.

    You should see: At least one of the three runs producing a noticeably worse or different-shaped output than the others, revealing a gap the first-pass instructions didn't anticipate.

    Hints 
    1. Deliberately make at least one of your three inputs messier or more unusual than the others.
    2. Write down, in one sentence per run, exactly what was wrong or missing — vague dissatisfaction won't tell you what to fix.
    3. If all three runs look equally good, make your third input noticeably more of an edge case and rerun.

  3. Based on the patterns across your three runs (not just the worst one), revise the Project's instructions and/or add a reference document or example addressing the gap you found, then rerun the same three inputs to confirm the fix generalizes.

    This closes the iteration loop at the workflow level — revising the reusable setup based on a pattern across runs, then re-validating, rather than patching a single bad output in isolation.

    You should see: Improved, more consistent output across all three reruns, with the specific gap you identified in step 2 visibly addressed in at least two of the three.

    Hints 
    1. Make one clear, specific revision (a new rule, an added example) rather than several vague changes at once, so you can tell what actually fixed it.
    2. If the fix only helps one of the three runs, the revision may be too narrowly targeted — broaden it.
    3. This Project is now reusable for the next real instance of the task, not just a one-off conversation.

Sources