11 · Capital E/E Publisher (diagnostic procedures)
Fault diagnosis
Given symptoms and a few voltmeter readings, can it find the one open circuit?
Each case is a fused feed tree, several loads, ground wires to shared ground splices and a strap to chassis. One element is open: a wire, the fuse, or a load. From which loads work and a handful of readings, name it. Every case is constructed to have exactly one consistent answer.
Capital reference. Creating Diagnostic Procedures (ee-publisher/creating-diagnostic-procedures). This domain stands for the capability that article describes; the task gives the model the rules in full rather than testing whether it remembers Capital’s.
Not tested here. Shorts, intermittent faults, and authoring the published procedure itself.
Scale ladder. S: 3 cases · M: 6 cases · L: 10 cases.
Results
| Model | Verdict | S passed | M passed | L passed | Mean score |
|---|---|---|---|---|---|
| Claude Opus 5.5 | Ready | 3 / 3 | 3 / 3 | 3 / 3 | 100 |
| Claude Fable 5.1 | Ready | 3 / 3 | 3 / 3 | 3 / 3 | 100 |
| Claude Sonnet 5.5 | Ready | 3 / 3 | 3 / 3 | 3 / 3 | 100 |
Every trial
| Model | Size | Seed | Score | Checks | Time |
|---|---|---|---|---|---|
| Claude Fable 5.1 | L#0 | 325172963 | 100 | faults found: 10 of 10 faults identified | 14 s |
| Claude Fable 5.1 | L#1 | 308395344 | 100 | faults found: 10 of 10 faults identified | 16 s |
| Claude Fable 5.1 | L#2 | 358728201 | 100 | faults found: 10 of 10 faults identified | 15 s |
| Claude Fable 5.1 | M#0 | 885637528 | 100 | faults found: 6 of 6 faults identified | 12 s |
| Claude Fable 5.1 | M#1 | 902415147 | 100 | faults found: 6 of 6 faults identified | 12 s |
| Claude Fable 5.1 | M#2 | 919192766 | 100 | faults found: 6 of 6 faults identified | 10 s |
| Claude Fable 5.1 | S#0 | 822899002 | 100 | faults found: 3 of 3 faults identified | 8 s |
| Claude Fable 5.1 | S#1 | 839676621 | 100 | faults found: 3 of 3 faults identified | 9 s |
| Claude Fable 5.1 | S#2 | 789343764 | 100 | faults found: 3 of 3 faults identified | 7 s |
| Claude Opus 5.5 | L#0 | 325172963 | 100 | faults found: 10 of 10 faults identified | 14 s |
| Claude Opus 5.5 | L#1 | 308395344 | 100 | faults found: 10 of 10 faults identified | 22 s |
| Claude Opus 5.5 | L#2 | 358728201 | 100 | faults found: 10 of 10 faults identified | 18 s |
| Claude Opus 5.5 | M#0 | 885637528 | 100 | faults found: 6 of 6 faults identified | 12 s |
| Claude Opus 5.5 | M#1 | 902415147 | 100 | faults found: 6 of 6 faults identified | 14 s |
| Claude Opus 5.5 | M#2 | 919192766 | 100 | faults found: 6 of 6 faults identified | 12 s |
| Claude Opus 5.5 | S#0 | 822899002 | 100 | faults found: 3 of 3 faults identified | 9 s |
| Claude Opus 5.5 | S#1 | 839676621 | 100 | faults found: 3 of 3 faults identified | 10 s |
| Claude Opus 5.5 | S#2 | 789343764 | 100 | faults found: 3 of 3 faults identified | 7 s |
| Claude Sonnet 5.5 | L#0 | 325172963 | 100 | faults found: 10 of 10 faults identified | 10 s |
| Claude Sonnet 5.5 | L#1 | 308395344 | 100 | faults found: 10 of 10 faults identified | 11 s |
| Claude Sonnet 5.5 | L#2 | 358728201 | 100 | faults found: 10 of 10 faults identified | 10 s |
| Claude Sonnet 5.5 | M#0 | 885637528 | 100 | faults found: 6 of 6 faults identified | 8 s |
| Claude Sonnet 5.5 | M#1 | 902415147 | 100 | faults found: 6 of 6 faults identified | 7 s |
| Claude Sonnet 5.5 | M#2 | 919192766 | 100 | faults found: 6 of 6 faults identified | 7 s |
| Claude Sonnet 5.5 | S#0 | 822899002 | 100 | faults found: 3 of 3 faults identified | 7 s |
| Claude Sonnet 5.5 | S#1 | 839676621 | 100 | faults found: 3 of 3 faults identified | 6 s |
| Claude Sonnet 5.5 | S#2 | 789343764 | 100 | faults found: 3 of 3 faults identified | 6 s |
What the model is asked
The small instance at seed 822899002, exactly as sent. Regenerate any instance with npm run sample -- diagnostics M 2.
Show the prompt (2,732 characters)
TASK: Fault diagnosis.
Each case is a separate 12 V circuit. BAT is the battery positive (12 V); CH is chassis ground (0 V).
Segments are wires (F1 is the fuse, treated as a segment). Loads are resistive devices between two nodes.
Wires have negligible resistance; every switch is closed. In each case EXACTLY ONE element is open circuit:
one segment (including the fuse) or one load. Symptoms say which loads operate. Readings are DC voltage at a node, measured to chassis.
Identify the open element in each case.
CASE1:
segments: [
{"id":"F1","a":"BAT","b":"P0"},
{"id":"S1","a":"P0","b":"P1"},
{"id":"S2","a":"P1","b":"P2"},
{"id":"S3","a":"P0","b":"P3"},
{"id":"S4","a":"G1","b":"GS1"},
{"id":"S5","a":"G2","b":"GS1"},
{"id":"S6","a":"G3","b":"GS1"},
{"id":"S7","a":"GS1","b":"CH"}
]
loads: [
{"id":"L1","a":"P2","b":"G1"},
{"id":"L2","a":"P1","b":"G2"},
{"id":"L3","a":"P3","b":"G3"}
]
symptoms: [{"load":"L1","state":"OFF"},{"load":"L2","state":"ON"},{"load":"L3","state":"ON"}]
readings: [{"node":"P2","reading":"12V"},{"node":"P3","reading":"12V"},{"node":"GS1","reading":"0V"},{"node":"G1","reading":"0V"}]
CASE2:
segments: [
{"id":"F1","a":"BAT","b":"P0"},
{"id":"S1","a":"P0","b":"P1"},
{"id":"S2","a":"P1","b":"P2"},
{"id":"S3","a":"P2","b":"P3"},
{"id":"S4","a":"P2","b":"P4"},
{"id":"S5","a":"P4","b":"P5"},
{"id":"S6","a":"G1","b":"GS1"},
{"id":"S7","a":"G2","b":"GS1"},
{"id":"S8","a":"G3","b":"GS1"},
{"id":"S9","a":"G4","b":"GS1"},
{"id":"S10","a":"GS1","b":"CH"}
]
loads: [
{"id":"L1","a":"P4","b":"G1"},
{"id":"L2","a":"P5","b":"G2"},
{"id":"L3","a":"P2","b":"G3"},
{"id":"L4","a":"P3","b":"G4"}
]
symptoms: [{"load":"L1","state":"ON"},{"load":"L2","state":"OFF"},{"load":"L3","state":"ON"},{"load":"L4","state":"ON"}]
readings: [{"node":"P5","reading":"12V"},{"node":"G3","reading":"0V"},{"node":"G2","reading":"0V"}]
CASE3:
segments: [
{"id":"F1","a":"BAT","b":"P0"},
{"id":"S1","a":"P0","b":"P1"},
{"id":"S2","a":"P0","b":"P2"},
{"id":"S3","a":"P1","b":"P3"},
{"id":"S4","a":"P1","b":"P4"},
{"id":"S5","a":"G1","b":"GS1"},
{"id":"S6","a":"G2","b":"GS1"},
{"id":"S7","a":"G3","b":"GS1"},
{"id":"S8","a":"G4","b":"GS1"},
{"id":"S9","a":"GS1","b":"CH"}
]
loads: [
{"id":"L1","a":"P2","b":"G1"},
{"id":"L2","a":"P4","b":"G2"},
{"id":"L3","a":"P1","b":"G3"},
{"id":"L4","a":"P3","b":"G4"}
]
symptoms: [{"load":"L1","state":"ON"},{"load":"L2","state":"ON"},{"load":"L3","state":"OFF"},{"load":"L4","state":"ON"}]
readings: [{"node":"P3","reading":"12V"},{"node":"G2","reading":"0V"},{"node":"G3","reading":"0V"}]
Output format (JSON, exactly these keys):
```json
{
"cases": [
{
"case": "CASE1",
"fault": "S4"
}
]
}
```