10 · Capital Harness Designer (harness synchronization) · Capital Systems Integrator (change history)
Change impact
Can it work out exactly what an engineering change touches?
Compare two revisions of a wire list, in different orders, and report what was added, removed and modified (and which attributes), then every harness and connector the change reaches. This is the input to an ECO and to harness synchronization.
Capital reference. Simulate Harness Synchronization (harness-designer/simulate-harness-synchronization). This domain stands for the capability that article describes; the task gives the model the rules in full rather than testing whether it remembers Capital’s.
Not tested here. Change policies, build lists and Teamcenter change objects.
Scale ladder. S: 15 wires per revision · M: 60 wires per revision · L: 180 wires per revision.
Results
| Model | Verdict | S passed | M passed | L passed | Mean score |
|---|---|---|---|---|---|
| Claude Opus 5.5 | Ready | 3 / 3 | 3 / 3 | 3 / 3 | 100 |
| Claude Fable 5.1 | Ready | 3 / 3 | 3 / 3 | 3 / 3 | 100 |
| Claude Sonnet 5.5 | Ready | 3 / 3 | 3 / 3 | 3 / 3 | 100 |
Every trial
| Model | Size | Seed | Score | Checks | Time |
|---|---|---|---|---|---|
| Claude Fable 5.1 | L#0 | 3722818793 | 100 | added: 14 of 14 · removed: 14 of 14 · modified: 14 of 14 · changed fields: 14 of 14 field lists exact · harnesses: 6 of 6 · connectors: 40 of 40 | 65 s |
| Claude Fable 5.1 | L#1 | 3706041174 | 100 | added: 14 of 14 · removed: 14 of 14 · modified: 14 of 14 · changed fields: 14 of 14 field lists exact · harnesses: 6 of 6 · connectors: 36 of 36 | 68 s |
| Claude Fable 5.1 | L#2 | 3689263555 | 100 | added: 14 of 14 · removed: 14 of 14 · modified: 14 of 14 · changed fields: 14 of 14 field lists exact · harnesses: 6 of 6 · connectors: 38 of 38 | 63 s |
| Claude Fable 5.1 | M#0 | 1868689318 | 100 | added: 5 of 5 · removed: 5 of 5 · modified: 5 of 5 · changed fields: 5 of 5 field lists exact · harnesses: 2 of 2 · connectors: 14 of 14 | 25 s |
| Claude Fable 5.1 | M#1 | 1885466937 | 100 | added: 5 of 5 · removed: 5 of 5 · modified: 5 of 5 · changed fields: 5 of 5 field lists exact · harnesses: 2 of 2 · connectors: 13 of 13 | 18 s |
| Claude Fable 5.1 | M#2 | 1835134080 | 100 | added: 5 of 5 · removed: 5 of 5 · modified: 5 of 5 · changed fields: 5 of 5 field lists exact · harnesses: 2 of 2 · connectors: 11 of 11 | 20 s |
| Claude Fable 5.1 | S#0 | 2976807628 | 100 | added: 2 of 2 · removed: 2 of 2 · modified: 2 of 2 · changed fields: 2 of 2 field lists exact · harnesses: 2 of 2 · connectors: 4 of 4 | 8 s |
| Claude Fable 5.1 | S#1 | 2993585247 | 100 | added: 2 of 2 · removed: 2 of 2 · modified: 2 of 2 · changed fields: 2 of 2 field lists exact · harnesses: 2 of 2 · connectors: 4 of 4 | 9 s |
| Claude Fable 5.1 | S#2 | 3010362866 | 100 | added: 2 of 2 · removed: 2 of 2 · modified: 2 of 2 · changed fields: 2 of 2 field lists exact · harnesses: 2 of 2 · connectors: 4 of 4 | 9 s |
| Claude Opus 5.5 | L#0 | 3722818793 | 100 | added: 14 of 14 · removed: 14 of 14 · modified: 14 of 14 · changed fields: 14 of 14 field lists exact · harnesses: 6 of 6 · connectors: 40 of 40 | 39 s |
| Claude Opus 5.5 | L#1 | 3706041174 | 100 | added: 14 of 14 · removed: 14 of 14 · modified: 14 of 14 · changed fields: 14 of 14 field lists exact · harnesses: 6 of 6 · connectors: 36 of 36 | 36 s |
| Claude Opus 5.5 | L#2 | 3689263555 | 100 | added: 14 of 14 · removed: 14 of 14 · modified: 14 of 14 · changed fields: 14 of 14 field lists exact · harnesses: 6 of 6 · connectors: 38 of 38 | 41 s |
| Claude Opus 5.5 | M#0 | 1868689318 | 100 | added: 5 of 5 · removed: 5 of 5 · modified: 5 of 5 · changed fields: 5 of 5 field lists exact · harnesses: 2 of 2 · connectors: 14 of 14 | 17 s |
| Claude Opus 5.5 | M#1 | 1885466937 | 100 | added: 5 of 5 · removed: 5 of 5 · modified: 5 of 5 · changed fields: 5 of 5 field lists exact · harnesses: 2 of 2 · connectors: 13 of 13 | 20 s |
| Claude Opus 5.5 | M#2 | 1835134080 | 100 | added: 5 of 5 · removed: 5 of 5 · modified: 5 of 5 · changed fields: 5 of 5 field lists exact · harnesses: 2 of 2 · connectors: 11 of 11 | 19 s |
| Claude Opus 5.5 | S#0 | 2976807628 | 100 | added: 2 of 2 · removed: 2 of 2 · modified: 2 of 2 · changed fields: 2 of 2 field lists exact · harnesses: 2 of 2 · connectors: 4 of 4 | 10 s |
| Claude Opus 5.5 | S#1 | 2993585247 | 100 | added: 2 of 2 · removed: 2 of 2 · modified: 2 of 2 · changed fields: 2 of 2 field lists exact · harnesses: 2 of 2 · connectors: 4 of 4 | 11 s |
| Claude Opus 5.5 | S#2 | 3010362866 | 100 | added: 2 of 2 · removed: 2 of 2 · modified: 2 of 2 · changed fields: 2 of 2 field lists exact · harnesses: 2 of 2 · connectors: 4 of 4 | 11 s |
| Claude Sonnet 5.5 | L#0 | 3722818793 | 100 | added: 14 of 14 · removed: 14 of 14 · modified: 14 of 14 · changed fields: 14 of 14 field lists exact · harnesses: 6 of 6 · connectors: 40 of 40 | 25 s |
| Claude Sonnet 5.5 | L#1 | 3706041174 | 100 | added: 14 of 14 · removed: 14 of 14 · modified: 14 of 14 · changed fields: 14 of 14 field lists exact · harnesses: 6 of 6 · connectors: 36 of 36 | 25 s |
| Claude Sonnet 5.5 | L#2 | 3689263555 | 100 | added: 14 of 14 · removed: 14 of 14 · modified: 14 of 14 · changed fields: 14 of 14 field lists exact · harnesses: 6 of 6 · connectors: 38 of 38 | 26 s |
| Claude Sonnet 5.5 | M#0 | 1868689318 | 100 | added: 5 of 5 · removed: 5 of 5 · modified: 5 of 5 · changed fields: 5 of 5 field lists exact · harnesses: 2 of 2 · connectors: 14 of 14 | 10 s |
| Claude Sonnet 5.5 | M#1 | 1885466937 | 100 | added: 5 of 5 · removed: 5 of 5 · modified: 5 of 5 · changed fields: 5 of 5 field lists exact · harnesses: 2 of 2 · connectors: 13 of 13 | 12 s |
| Claude Sonnet 5.5 | M#2 | 1835134080 | 100 | added: 5 of 5 · removed: 5 of 5 · modified: 5 of 5 · changed fields: 5 of 5 field lists exact · harnesses: 2 of 2 · connectors: 11 of 11 | 11 s |
| Claude Sonnet 5.5 | S#0 | 2976807628 | 100 | added: 2 of 2 · removed: 2 of 2 · modified: 2 of 2 · changed fields: 2 of 2 field lists exact · harnesses: 2 of 2 · connectors: 4 of 4 | 7 s |
| Claude Sonnet 5.5 | S#1 | 2993585247 | 100 | added: 2 of 2 · removed: 2 of 2 · modified: 2 of 2 · changed fields: 2 of 2 field lists exact · harnesses: 2 of 2 · connectors: 4 of 4 | 8 s |
| Claude Sonnet 5.5 | S#2 | 3010362866 | 100 | added: 2 of 2 · removed: 2 of 2 · modified: 2 of 2 · changed fields: 2 of 2 field lists exact · harnesses: 2 of 2 · connectors: 4 of 4 | 7 s |
What the model is asked
The small instance at seed 2976807628, exactly as sent. Regenerate any instance with npm run sample -- change M 2.
Show the prompt (3,523 characters)
TASK: Change impact analysis.
Compare revision A and revision B of a wire list. Wires are matched by id. Report:
- added: ids in B but not A; removed: ids in A but not B;
- modified: ids in both whose attributes differ, with the list of fields that changed (fields are from, to, gauge, color, harness);
- impactedHarnesses: every harness that any added, removed or modified wire belongs to, in EITHER revision;
- impactedConnectors: every connector (the part of an end before the "-") at either end of any added, removed or modified wire, in EITHER revision.
Revision A:
[
{"id":"W1001","from":"C1-5","to":"C4-4","gauge":18,"color":"VT","harness":"HN-101"},
{"id":"W1002","from":"C3-1","to":"C3-15","gauge":22,"color":"GY","harness":"HN-101"},
{"id":"W1003","from":"C1-1","to":"C3-19","gauge":16,"color":"BK","harness":"HN-100"},
{"id":"W1004","from":"C3-10","to":"C3-18","gauge":18,"color":"BU","harness":"HN-100"},
{"id":"W1005","from":"C3-20","to":"C3-7","gauge":22,"color":"OG","harness":"HN-100"},
{"id":"W1006","from":"C1-11","to":"C2-6","gauge":16,"color":"GY","harness":"HN-101"},
{"id":"W1007","from":"C1-12","to":"C2-10","gauge":20,"color":"RD","harness":"HN-101"},
{"id":"W1008","from":"C2-13","to":"C3-12","gauge":22,"color":"YE","harness":"HN-101"},
{"id":"W1009","from":"C1-11","to":"C3-7","gauge":22,"color":"RD","harness":"HN-100"},
{"id":"W1010","from":"C2-20","to":"C1-10","gauge":20,"color":"RD","harness":"HN-101"},
{"id":"W1011","from":"C2-16","to":"C1-1","gauge":16,"color":"VT","harness":"HN-101"},
{"id":"W1012","from":"C1-15","to":"C2-1","gauge":22,"color":"GY","harness":"HN-101"},
{"id":"W1013","from":"C3-12","to":"C2-4","gauge":16,"color":"YE","harness":"HN-101"},
{"id":"W1014","from":"C1-3","to":"C2-13","gauge":18,"color":"YE","harness":"HN-100"},
{"id":"W1015","from":"C2-10","to":"C3-13","gauge":20,"color":"OG","harness":"HN-100"}
]
Revision B:
[
{"id":"W1002","from":"C3-1","to":"C3-15","gauge":22,"color":"GY","harness":"HN-101"},
{"id":"W1006","from":"C1-11","to":"C2-6","gauge":16,"color":"BN","harness":"HN-101"},
{"id":"W1001","from":"C1-5","to":"C4-4","gauge":18,"color":"VT","harness":"HN-101"},
{"id":"W1009","from":"C1-11","to":"C3-7","gauge":22,"color":"RD","harness":"HN-100"},
{"id":"W1005","from":"C3-20","to":"C3-7","gauge":22,"color":"OG","harness":"HN-100"},
{"id":"W1004","from":"C3-10","to":"C3-18","gauge":18,"color":"BU","harness":"HN-100"},
{"id":"W1008","from":"C2-13","to":"C3-12","gauge":20,"color":"OG","harness":"HN-101"},
{"id":"W1015","from":"C2-10","to":"C3-13","gauge":20,"color":"OG","harness":"HN-100"},
{"id":"W2002","from":"C1-4","to":"C2-17","gauge":20,"color":"BU","harness":"HN-100"},
{"id":"W2001","from":"C4-9","to":"C1-17","gauge":16,"color":"BU","harness":"HN-100"},
{"id":"W1003","from":"C1-1","to":"C3-19","gauge":16,"color":"BK","harness":"HN-100"},
{"id":"W1007","from":"C1-12","to":"C2-10","gauge":20,"color":"RD","harness":"HN-101"},
{"id":"W1012","from":"C1-15","to":"C2-1","gauge":22,"color":"GY","harness":"HN-101"},
{"id":"W1013","from":"C3-12","to":"C2-4","gauge":16,"color":"YE","harness":"HN-101"},
{"id":"W1014","from":"C1-3","to":"C2-13","gauge":18,"color":"YE","harness":"HN-100"}
]
Output format (JSON, exactly these keys):
```json
{
"added": [
"W2001"
],
"removed": [
"W1004"
],
"modified": [
{
"id": "W1010",
"fields": [
"gauge",
"color"
]
}
],
"impactedHarnesses": [
"HN-100"
],
"impactedConnectors": [
"C3",
"C7"
]
}
```