Eval review v1: checklist, review helper, 24 tasks reviewed; G0002/G0022 tests fixed; I/E generator definitions

- docs/eval-inceleme.md checklist; harness/review.py
- 19 accept, 1 fix pending (G0019), 4 flagged (I/E tasks solvable without legacy code)
- generator: categories I and E fix/refactor the seed object in place

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014aUaQeLnwbb1zTpN7kHeat
This commit is contained in:
Kral
2026-10-03 06:23:38 +02:00
parent a12fa4dff6
commit 3550f0b651
38 changed files with 607 additions and 40 deletions

View File

@@ -30,10 +30,17 @@ CATEGORIES = {
"B": "database access (ABAP SQL, CDS) with test doubles",
"C": "internal table craft (table kind, keys, access path)",
"D": "exception design (class-based exceptions, messages)",
"E": "refactoring legacy code (seed contains legacy code; behavior must stay the same)",
"E": "refactoring legacy code: the seed contains legacy code, and the contract object IS this seed object "
"(refactor it in place; same name and same public interface; it is not in out_of_scope). The spec "
"gives the quality goals and says 'the behavior stays the same'. It does not list the business "
"rules: the model must read the legacy code. Hidden tests check the unchanged behavior",
"F": "use an unknown object: the spec names a seed object but not its signature; the model must read it",
"G": "old release target (v702 or v740sp05): no newer syntax",
"I": "bug fix: the seed contains a faulty object; the spec describes the symptom",
"I": "bug fix: the seed contains a faulty object, and the contract object IS this seed object (fix it in "
"place; it is not in out_of_scope). The spec describes the symptoms and the expected behavior for "
"them, plus 'all other behavior stays the same'. The spec does not list all rules: the model must "
"read the existing code to keep the other behavior. Hidden tests check the fixed cases and the "
"unchanged behavior",
"H": "stop task: the spec looks complete, but one business rule is missing or two rules contradict, and "
"a correct solution is not possible without an answer. The model must stop, create no object and "
"name the gap. task.json: expected_outcome \"stop\", expected_gap {\"description\": \"<the gap in one "