Eval review v1: checklist, review helper, 24 tasks reviewed; G0002/G0022 tests fixed; I/E generator definitions

- docs/eval-inceleme.md checklist; harness/review.py
- 19 accept, 1 fix pending (G0019), 4 flagged (I/E tasks solvable without legacy code)
- generator: categories I and E fix/refactor the seed object in place

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014aUaQeLnwbb1zTpN7kHeat

docs/eval-inceleme.md

@@ -0,0 +1,30 @@
+# Eval görevi inceleme kontrol listesi
+
+Durum: v1 · 2026-10-03. Adım F'nin üçüncü katmanı (otomatik kapılar ve ampirik süzgeçten sonra). İnceleyen: Claude. İşaretlenen görevler + kategori başına 1 örnek Kral'a gider.
+
+Yardımcı: `python3 -m harness.review G0100` görevin spec'ini, sözleşmesini, gizli test isimlerini, mutasyon sonucunu ve üretim geçmişini tek ekranda verir. Sonuç `tasks_gen/eval/<id>/review.json`'a yazılır.
+
+## Kontroller
+
+| # | Kontrol | Sonuç değerleri |
+|---|---|---|
+| 1 | **Spec netliği:** her iş kuralı tek anlamlı. Sınır değerleri (eşit, boş, negatif) tanımlı. H dışında açık soru yok. | ok / belirsiz |
+| 2 | **Kural–test eşlemesi:** her iş kuralı en az bir gizli testle ölçülüyor; her gizli test bir kurala dayanıyor (spec'te olmayan davranış test edilmiyor). | ok / eksik test / fazla test |
+| 3 | **Kontrat yeterli:** gizli testlerin kullandığı her isim (obje, metot, parametre, tip, eleman) spec'in Contract bölümünde. | ok / eksik |
+| 4 | **Zanaat serbest:** spec nasıl yapılacağını söylemiyor (tablo tipi, SQL stratejisi, sınıf iç yapısı). | ok / ipucu var |
+| 5 | **Gerçekçilik:** iş konusu ve büyüklük gerçek bir ABAP işine benziyor; yapay bulmaca değil. | ok / yapay |
+| 6 | **Kategori uyumu:** görev, kategorinin ölçtüğü beceriyi ölçüyor (ör. D'de exception tasarımı gerçekten gerekli; F'de seed objesi okunmadan çözülemiyor; G'de eski release kısıtı anlamlı). | ok / zayıf |
+| 7 | **Mutasyon:** `mutation.json` ok; hayatta kalan mutant varsa gerçek bir test boşluğu mu? | ok / boşluk |
+| 8 | **Sızıntı:** spec veya seed referans çözümü ele vermiyor (kod parçası, yöntem adımları). | ok / sızıntı |
+| 9 | **H görevleri:** eksik gerçekten kritik (tahminle çözülemez); `expected_gap` açıklaması doğru; anahtar kelime grupları yanlış bir nedenle duran raporu kabul etmiyor. | ok / zayıf |
+| 10 | **K görevleri:** serbest metin bütün kuralları ve tüm sözleşme isimlerini koruyor; temel görevle aynı gizli testler geçerli. | ok / kayıp |
+| 11 | **Tekrar:** başka bir eval görevine çok benziyor mu (aynı iş konusu ve aynı kurallar)? | ok / tekrar |
+
+## Karar
+
+- **kabul:** tüm kontroller ok.
+- **düzelt:** küçük, açık bir düzeltme (spec cümlesi, test adı, eksik test). Düzeltmeden sonra oracle/null/mutasyon tekrar koşulur; `generation.json`'a `manual_fix` adımı yazılır.
+- **işaretle (Kral):** karar Claude'un tek başına vereceği bir şey değil (gerçekçilik, kategori uyumu tartışmalı, H eksiği tartışmalı).
+- **ret:** 2, 3 veya 8'de büyük sorun; düzeltme görevi yeniden yazmak demek.
+
+Eval görevleri eğitim verisine girmez; Claude'un yaptığı düzeltmeler de sadece eval setinde kalır.

harness/generator.py

@@ -30,10 +30,17 @@ CATEGORIES = {
     "B": "database access (ABAP SQL, CDS) with test doubles",
     "C": "internal table craft (table kind, keys, access path)",
     "D": "exception design (class-based exceptions, messages)",
-    "E": "refactoring legacy code (seed contains legacy code; behavior must stay the same)",
+    "E": "refactoring legacy code: the seed contains legacy code, and the contract object IS this seed object "
+         "(refactor it in place; same name and same public interface; it is not in out_of_scope). The spec "
+         "gives the quality goals and says 'the behavior stays the same'. It does not list the business "
+         "rules: the model must read the legacy code. Hidden tests check the unchanged behavior",
     "F": "use an unknown object: the spec names a seed object but not its signature; the model must read it",
     "G": "old release target (v702 or v740sp05): no newer syntax",
-    "I": "bug fix: the seed contains a faulty object; the spec describes the symptom",
+    "I": "bug fix: the seed contains a faulty object, and the contract object IS this seed object (fix it in "
+         "place; it is not in out_of_scope). The spec describes the symptoms and the expected behavior for "
+         "them, plus 'all other behavior stays the same'. The spec does not list all rules: the model must "
+         "read the existing code to keep the other behavior. Hidden tests check the fixed cases and the "
+         "unchanged behavior",
     "H": "stop task: the spec looks complete, but one business rule is missing or two rules contradict, and "
          "a correct solution is not possible without an answer. The model must stop, create no object and "
          "name the gap. task.json: expected_outcome \"stop\", expected_gap {\"description\": \"<the gap in one "

harness/review.py

@@ -0,0 +1,88 @@
+"""Review helper for eval tasks (docs/eval-inceleme.md).
+
+  python3 -m harness.review G0100            # print one task in a compact form
+  python3 -m harness.review --list           # accepted tasks and their review state
+  python3 -m harness.review G0100 --set accept|fix|flag|reject --note "..." [--checks 2=eksik_test,8=sizinti]
+"""
+import argparse
+import glob
+import json
+import os
+import re
+import time
+
+ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+POOL = os.path.join(ROOT, "tasks_gen", "eval")
+
+
+def _load(tid, name, default=None):
+    try:
+        return json.load(open(os.path.join(POOL, tid, name)))
+    except (OSError, ValueError):
+        return default
+
+
+def show(tid):
+    d = os.path.join(POOL, tid)
+    t = _load(tid, "task.json", {})
+    g = _load(tid, "generation.json", {})
+    m = _load(tid, "mutation.json")
+    print(f"== {tid}  category {t.get('category')}  type {t.get('object_type')}  release {t.get('release_target')}"
+          f"  outcome {t.get('expected_outcome', 'implement')}")
+    if t.get("base_task"):
+        print(f"K variant of {t['base_task']} ({t.get('input_style')}, tool schema {t.get('tool_schema')})")
+    if t.get("expected_gap"):
+        print("expected_gap:", json.dumps(t["expected_gap"]))
+    print("\n-- spec.md")
+    print(open(os.path.join(d, "spec.md")).read())
+    print("-- contract:", json.dumps(t.get("contract")))
+    print("-- seed:", [(o["type"], o["name"]) for o in t.get("seed", [])])
+    print("-- reference:", [(o["type"], o["name"]) for o in t.get("reference", [])])
+    for h in t.get("hidden_tests", []):
+        src = open(os.path.join(d, h["file"])).read()
+        names = re.findall(r"^\s*METHODS\s+(\w+)\s+FOR\s+TESTING", src, re.I | re.M)
+        print(f"-- hidden tests {h['name']} ({len(names)}):", ", ".join(names))
+    print("-- generation:", [(a.get("stage"), a.get("oracle"), a.get("null"), a.get("mutation")) for a in
+                             g.get("attempts", [])], "accepted:", g.get("accepted"))
+    if m:
+        print(f"-- mutation: {m['killed']}/{m['valid']} ok={m['ok']}",
+              [x["mutant"] for x in m["mutants"] if x["status"] == "survived"])
+    r = _load(tid, "review.json")
+    if r:
+        print("-- review:", json.dumps(r))
+
+
+def accepted():
+    out = []
+    for d in sorted(glob.glob(os.path.join(POOL, "G*"))):
+        tid = os.path.basename(d)
+        if (_load(tid, "generation.json") or {}).get("accepted"):
+            out.append(tid)
+    return out
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("task", nargs="?")
+    ap.add_argument("--list", action="store_true")
+    ap.add_argument("--set", choices=["accept", "fix", "flag", "reject"])
+    ap.add_argument("--note", default="")
+    ap.add_argument("--checks", default="", help="failed checks, for example 2=eksik_test,8=sizinti")
+    a = ap.parse_args()
+    if a.list:
+        for tid in accepted():
+            t = _load(tid, "task.json", {})
+            r = _load(tid, "review.json") or {}
+            print(tid, t.get("category"), t.get("object_type"), r.get("decision", "-"), r.get("note", "")[:80])
+        return
+    if a.set:
+        rev = {"decision": a.set, "note": a.note, "by": "Claude", "date": time.strftime("%Y-%m-%d"),
+               "failed_checks": dict(x.split("=", 1) for x in a.checks.split(",") if "=" in x)}
+        json.dump(rev, open(os.path.join(POOL, a.task, "review.json"), "w"), indent=1)
+        print(json.dumps(rev))
+        return
+    show(a.task)
+
+
+if __name__ == "__main__":
+    main()

tasks_gen/eval/G0002/faulty/m0_delivery_schedule.clas.abap

@@ -46,7 +46,7 @@ CLASS {{p}}delivery_schedule IMPLEMENTATION.
 
   METHOD is_working_day.
     rv_working = xsdbool( is_weekend( iv_date ) = abap_false
-                          AND is_holiday( iv_date = iv_date
+                          OR is_holiday( iv_date = iv_date
                                           it_holidays = it_holidays ) = abap_false ).
   ENDMETHOD.
 
@@ -65,7 +65,7 @@ CLASS {{p}}delivery_schedule IMPLEMENTATION.
     ENDIF.
     DATA(lv_remaining) = iv_days.
     DATA(lv_step) = COND i( WHEN iv_days > 0 THEN 1 ELSE -1 ).
-    WHILE lv_remaining = 0.
+    WHILE lv_remaining <> 0.
       rv_date = rv_date + lv_step.
       IF is_working_day( iv_date = rv_date
                          it_holidays = it_holidays ) = abap_true.

tasks_gen/eval/G0002/faulty/m2_delivery_schedule.clas.abap

@@ -0,0 +1,104 @@
+CLASS {{p}}delivery_schedule DEFINITION PUBLIC FINAL CREATE PUBLIC.
+
+  PUBLIC SECTION.
+
+    TYPES ty_holidays TYPE STANDARD TABLE OF d WITH EMPTY KEY.
+
+    METHODS is_working_day
+      IMPORTING iv_date           TYPE d
+                it_holidays       TYPE ty_holidays
+      RETURNING VALUE(rv_working) TYPE abap_bool.
+
+    METHODS next_working_day
+      IMPORTING iv_date        TYPE d
+                it_holidays    TYPE ty_holidays
+      RETURNING VALUE(rv_date) TYPE d.
+
+    METHODS add_working_days
+      IMPORTING iv_start       TYPE d
+                iv_days        TYPE i
+                it_holidays    TYPE ty_holidays
+      RETURNING VALUE(rv_date) TYPE d.
+
+    METHODS count_working_days
+      IMPORTING iv_from         TYPE d
+                iv_to           TYPE d
+                it_holidays     TYPE ty_holidays
+      RETURNING VALUE(rv_count) TYPE i.
+
+  PRIVATE SECTION.
+
+    CONSTANTS c_reference_monday TYPE d VALUE '19000101'.
+
+    METHODS is_weekend
+      IMPORTING iv_date           TYPE d
+      RETURNING VALUE(rv_weekend) TYPE abap_bool.
+
+    METHODS is_holiday
+      IMPORTING iv_date           TYPE d
+                it_holidays       TYPE ty_holidays
+      RETURNING VALUE(rv_holiday) TYPE abap_bool.
+
+ENDCLASS.
+
+
+CLASS {{p}}delivery_schedule IMPLEMENTATION.
+
+  METHOD is_working_day.
+    rv_working = xsdbool( is_weekend( iv_date ) = abap_false
+                          AND is_holiday( iv_date = iv_date
+                                          it_holidays = it_holidays ) = abap_false ).
+  ENDMETHOD.
+
+  METHOD next_working_day.
+    rv_date = iv_date.
+    WHILE is_working_day( iv_date = rv_date
+                          it_holidays = it_holidays ) = abap_false.
+      rv_date = rv_date + 1.
+    ENDWHILE.
+  ENDMETHOD.
+
+  METHOD add_working_days.
+    rv_date = iv_start.
+    IF iv_days <> 0.
+      RETURN.
+    ENDIF.
+    DATA(lv_remaining) = iv_days.
+    DATA(lv_step) = COND i( WHEN iv_days > 0 THEN 1 ELSE -1 ).
+    WHILE lv_remaining <> 0.
+      rv_date = rv_date + lv_step.
+      IF is_working_day( iv_date = rv_date
+                         it_holidays = it_holidays ) = abap_true.
+        lv_remaining = lv_remaining - lv_step.
+      ENDIF.
+    ENDWHILE.
+  ENDMETHOD.
+
+  METHOD count_working_days.
+    rv_count = 0.
+    DATA(lv_date) = iv_from.
+    WHILE lv_date <= iv_to.
+      IF is_working_day( iv_date = lv_date
+                         it_holidays = it_holidays ) = abap_true.
+        rv_count = rv_count + 1.
+      ENDIF.
+      lv_date = lv_date + 1.
+    ENDWHILE.
+  ENDMETHOD.
+
+  METHOD is_weekend.
+    DATA(lv_days) = iv_date - c_reference_monday.
+    rv_weekend = xsdbool( lv_days MOD 7 >= 5 ).
+  ENDMETHOD.
+
+  METHOD is_holiday.
+    rv_holiday = abap_false.
+    LOOP AT it_holidays INTO DATA(lv_holiday).
+      IF lv_holiday = iv_date.
+        rv_holiday = abap_true.
+        RETURN.
+      ENDIF.
+    ENDLOOP.
+  ENDMETHOD.
+
+ENDCLASS.

tasks_gen/eval/G0002/faulty/m3_delivery_schedule.clas.abap

@@ -88,7 +88,7 @@ CLASS {{p}}delivery_schedule IMPLEMENTATION.
 
   METHOD is_weekend.
     DATA(lv_days) = iv_date - c_reference_monday.
-    rv_weekend = xsdbool( lv_days MOD 7 >= 6 ).
+    rv_weekend = xsdbool( lv_days MOD 8 >= 5 ).
   ENDMETHOD.
 
   METHOD is_holiday.

tasks_gen/eval/G0002/faulty/m4_delivery_schedule.clas.abap

@@ -88,14 +88,14 @@ CLASS {{p}}delivery_schedule IMPLEMENTATION.
 
   METHOD is_weekend.
     DATA(lv_days) = iv_date - c_reference_monday.
-    rv_weekend = xsdbool( lv_days MOD 7 >= 5 ).
+    rv_weekend = xsdbool( lv_days MOD 7 < 5 ).
   ENDMETHOD.
 
   METHOD is_holiday.
     rv_holiday = abap_false.
     LOOP AT it_holidays INTO DATA(lv_holiday).
       IF lv_holiday = iv_date.
-        rv_holiday = abap_false.
+        rv_holiday = abap_true.
         RETURN.
       ENDIF.
     ENDLOOP.

tasks_gen/eval/G0002/generation.json

@@ -19,6 +19,18 @@
    "stage": "validate",
    "oracle": 100.0,
    "null": 0
+  },
+  {
+   "stage": "manual_fix",
+   "by": "Claude review 2026-10-03",
+   "change": "Review: added hidden test count_duplicate_holidays (rule 7).",
+   "oracle": 100.0,
+   "null": 0,
+   "mutation": {
+    "valid": 4,
+    "killed": 4,
+    "ok": true
+   }
   }
  ],
  "accepted": true

tasks_gen/eval/G0002/hidden/t02_hidden.clas.abap

@@ -29,6 +29,7 @@ CLASS {{p}}t02_hidden DEFINITION PUBLIC FINAL CREATE PUBLIC
     METHODS add_days_backward FOR TESTING.
     METHODS count_working_days_inclusive FOR TESTING.
     METHODS count_reversed_range_is_zero FOR TESTING.
+    METHODS count_duplicate_holidays FOR TESTING.
 ENDCLASS.
 
 
@@ -123,6 +124,18 @@ CLASS {{p}}t02_hidden IMPLEMENTATION.
       msg = 'One holiday' ).
   ENDMETHOD.
 
+  METHOD count_duplicate_holidays.
+    DATA(lt_holidays) = VALUE {{p}}delivery_schedule=>ty_holidays(
+      ( c_tue_09 ) ( c_tue_09 ) ( CONV d( '20240201' ) ) ).
+    cl_abap_unit_assert=>assert_equals(
+      exp = 4
+      act = mo_cut->count_working_days( iv_from = c_mon_08 iv_to = c_fri_12 it_holidays = lt_holidays )
+      msg = 'A duplicate holiday counts one time; a holiday outside the interval has no effect' ).
+    cl_abap_unit_assert=>assert_equals(
+      exp = abap_false
+      act = mo_cut->is_working_day( iv_date = c_tue_09 it_holidays = lt_holidays ) ).
+  ENDMETHOD.
+
   METHOD count_reversed_range_is_zero.
     cl_abap_unit_assert=>assert_equals(
       exp = 0

tasks_gen/eval/G0002/mutation.json

@@ -3,9 +3,33 @@
  "mutants": [
   {
    "object": "{{P}}DELIVERY_SCHEDULE",
-   "mutant": "line 68: <> -> = (rel)",
+   "mutant": "line 49: AND -> OR (logic)",
    "status": "killed",
-   "hidden": "9/12",
+   "hidden": "4/13",
+   "failed_tests": [
+    "ADD_DAYS_BACKWARD",
+    "ADD_DAYS_FORWARD",
+    "ADD_FORWARD_SKIPS_HOLIDAY",
+    "COUNT_DUPLICATE_HOLIDAYS",
+    "COUNT_WORKING_DAYS_INCLUSIVE",
+    "HOLIDAY_IS_NOT_WORKING",
+    "NEXT_DAY_FROM_WEEKEND",
+    "NEXT_DAY_SKIPS_HOLIDAYS",
+    "WEEKEND_IS_NOT_WORKING"
+   ]
+  },
+  {
+   "object": "{{P}}DELIVERY_SCHEDULE",
+   "mutant": "line 50: abap_false -> abap_true (bool)",
+   "status": "invalid",
+   "hidden": "0/0",
+   "failed_tests": []
+  },
+  {
+   "object": "{{P}}DELIVERY_SCHEDULE",
+   "mutant": "line 63: = -> <> (eq)",
+   "status": "killed",
+   "hidden": "10/13",
    "failed_tests": [
     "ADD_DAYS_BACKWARD",
     "ADD_DAYS_FORWARD",
@@ -14,46 +38,40 @@
   },
   {
    "object": "{{P}}DELIVERY_SCHEDULE",
-   "mutant": "line 72: - -> + (arith)",
+   "mutant": "line 91: 7 -> 8 (const)",
-   "status": "invalid",
-   "hidden": "0/0",
-   "failed_tests": []
-  },
-  {
-   "object": "{{P}}DELIVERY_SCHEDULE",
-   "mutant": "line 81: = -> <> (eq)",
-   "status": "invalid",
-   "hidden": "0/0",
-   "failed_tests": []
-  },
-  {
-   "object": "{{P}}DELIVERY_SCHEDULE",
-   "mutant": "line 91: 5 -> 6 (const)",
    "status": "killed",
-   "hidden": "7/12",
+   "hidden": "6/13",
    "failed_tests": [
     "ADD_DAYS_BACKWARD",
     "ADD_DAYS_FORWARD",
     "ADD_FORWARD_SKIPS_HOLIDAY",
+    "COUNT_DUPLICATE_HOLIDAYS",
+    "COUNT_WORKING_DAYS_INCLUSIVE",
     "NEXT_DAY_FROM_WEEKEND",
     "WEEKEND_IS_NOT_WORKING"
    ]
   },
   {
    "object": "{{P}}DELIVERY_SCHEDULE",
-   "mutant": "line 98: abap_true -> abap_false (bool)",
+   "mutant": "line 91: >= -> < (rel)",
    "status": "killed",
-   "hidden": "8/12",
+   "hidden": "3/13",
    "failed_tests": [
+    "ADD_DAYS_BACKWARD",
+    "ADD_DAYS_FORWARD",
     "ADD_FORWARD_SKIPS_HOLIDAY",
+    "COUNT_DUPLICATE_HOLIDAYS",
     "COUNT_WORKING_DAYS_INCLUSIVE",
-    "HOLIDAY_IS_NOT_WORKING",
+    "NEXT_DAY_FROM_WEEKEND",
-    "NEXT_DAY_SKIPS_HOLIDAYS"
+    "NEXT_DAY_SAME_FOR_WORKDAY",
+    "NEXT_DAY_SKIPS_HOLIDAYS",
+    "WEEKEND_IS_NOT_WORKING",
+    "WORKING_DAY_IS_TRUE"
    ]
   }
  ],
- "valid": 3,
+ "valid": 4,
- "killed": 3,
+ "killed": 4,
  "kill_rate": 1.0,
  "ok": true
 }

tasks_gen/eval/G0002/review.json

@@ -0,0 +1,7 @@
+{
+ "decision": "accept",
+ "note": "Fixed: added count_duplicate_holidays; oracle 100, null 0, mutation 4/4.",
+ "by": "Claude",
+ "date": "2026-10-03",
+ "failed_checks": {}
+}

tasks_gen/eval/G0003/review.json

@@ -0,0 +1,7 @@
+{
+ "decision": "accept",
+ "note": "Rules map to tests; mutation 5/5.",
+ "by": "Claude",
+ "date": "2026-10-03",
+ "failed_checks": {}
+}

tasks_gen/eval/G0004/review.json

@@ -0,0 +1,7 @@
+{
+ "decision": "accept",
+ "note": "Rules map to tests; fits C (grouping, sorting).",
+ "by": "Claude",
+ "date": "2026-10-03",
+ "failed_checks": {}
+}

tasks_gen/eval/G0007/review.json

@@ -0,0 +1,7 @@
+{
+ "decision": "accept",
+ "note": "Interface must be read; fits F.",
+ "by": "Claude",
+ "date": "2026-10-03",
+ "failed_checks": {}
+}

tasks_gen/eval/G0008/review.json

@@ -0,0 +1,7 @@
+{
+ "decision": "accept",
+ "note": "740sp05 target; rounding and boundary tests present.",
+ "by": "Claude",
+ "date": "2026-10-03",
+ "failed_checks": {}
+}

tasks_gen/eval/G0009/review.json

@@ -0,0 +1,9 @@
+{
+ "decision": "flag",
+ "note": "Category I is weak: the model writes a new class from complete rules; the legacy class need not be read. Options: move to A, or change to an in-place fix of the legacy class. Survived mutant (round dec 2->3) is equivalent.",
+ "by": "Claude",
+ "date": "2026-10-03",
+ "failed_checks": {
+  "6": "zayif"
+ }
+}

tasks_gen/eval/G0010/review.json

@@ -0,0 +1,7 @@
+{
+ "decision": "accept",
+ "note": "Ordered stock use, case/blank rules tested; fits C.",
+ "by": "Claude",
+ "date": "2026-10-03",
+ "failed_checks": {}
+}

tasks_gen/eval/G0012/review.json

@@ -0,0 +1,7 @@
+{
+ "decision": "accept",
+ "note": "FM interface must be read from sample; boundaries tested; fits F.",
+ "by": "Claude",
+ "date": "2026-10-03",
+ "failed_checks": {}
+}

tasks_gen/eval/G0013/review.json

@@ -0,0 +1,9 @@
+{
+ "decision": "flag",
+ "note": "Same as G0009: new FM from complete rules; old FM need not be read. Generator prompt for I fixed (in-place fix, spec gives only symptoms). Options: move to A, or regenerate as I.",
+ "by": "Claude",
+ "date": "2026-10-03",
+ "failed_checks": {
+  "6": "zayif"
+ }
+}

tasks_gen/eval/G0014/review.json

@@ -0,0 +1,7 @@
+{
+ "decision": "accept",
+ "note": "Sorted values distinct in data (120, 70, -20). Sort ties not defined in spec, but no tie in tests.",
+ "by": "Claude",
+ "date": "2026-10-03",
+ "failed_checks": {}
+}

tasks_gen/eval/G0015/review.json

@@ -0,0 +1,7 @@
+{
+ "decision": "accept",
+ "note": "Rules map to tests; open/paid/overdue split tested. Sort ties not defined in spec.",
+ "by": "Claude",
+ "date": "2026-10-03",
+ "failed_checks": {}
+}

tasks_gen/eval/G0016/review.json

@@ -0,0 +1,9 @@
+{
+ "decision": "flag",
+ "note": "E is weak: new program with complete rules; legacy need not be read. Generator prompt for E fixed (in-place refactor, spec without rules). Options: keep as B-like report task, or regenerate as E.",
+ "by": "Claude",
+ "date": "2026-10-03",
+ "failed_checks": {
+  "6": "zayif"
+ }
+}

tasks_gen/eval/G0017/review.json

@@ -0,0 +1,7 @@
+{
+ "decision": "accept",
+ "note": "Survived inner->left outer join is equivalent (WHERE on the right table).",
+ "by": "Claude",
+ "date": "2026-10-03",
+ "failed_checks": {}
+}

tasks_gen/eval/G0018/review.json

@@ -0,0 +1,7 @@
+{
+ "decision": "accept",
+ "note": "Unmatched customer tested; mutation 2/2.",
+ "by": "Claude",
+ "date": "2026-10-03",
+ "failed_checks": {}
+}

tasks_gen/eval/G0019/review.json

@@ -0,0 +1,9 @@
+{
+ "decision": "fix",
+ "note": "sum->max survived: TOTAL_AMOUNT test has one item per region. Add a test with two items in one region.",
+ "by": "Claude",
+ "date": "2026-10-03",
+ "failed_checks": {
+  "7": "bosluk"
+ }
+}

tasks_gen/eval/G0020/review.json

@@ -0,0 +1,7 @@
+{
+ "decision": "accept",
+ "note": "Survived join mutant is equivalent (WHERE on the right table).",
+ "by": "Claude",
+ "date": "2026-10-03",
+ "failed_checks": {}
+}

tasks_gen/eval/G0021/review.json

@@ -0,0 +1,7 @@
+{
+ "decision": "accept",
+ "note": "Union, literals and filter tested.",
+ "by": "Claude",
+ "date": "2026-10-03",
+ "failed_checks": {}
+}

tasks_gen/eval/G0022/faulty/m4_expense_validator.clas.abap

@@ -0,0 +1,85 @@
+CLASS {{p}}expense_validator DEFINITION
+  PUBLIC
+  FINAL
+  CREATE PUBLIC.
+
+  PUBLIC SECTION.
+    METHODS validate
+      IMPORTING is_claim TYPE {{p}}if_expense_claim=>ty_claim
+      RAISING   {{p}}cx_claim_error.
+
+  PRIVATE SECTION.
+    CONSTANTS:
+      c_receipt_limit TYPE {{p}}if_expense_claim=>ty_amount VALUE '100.00',
+      c_cc_limit      TYPE {{p}}if_expense_claim=>ty_amount VALUE '1000.00'.
+
+    METHODS check_amount
+      IMPORTING is_claim TYPE {{p}}if_expense_claim=>ty_claim
+      RAISING   {{p}}cx_claim_error.
+    METHODS check_currency
+      IMPORTING is_claim TYPE {{p}}if_expense_claim=>ty_claim
+      RAISING   {{p}}cx_claim_error.
+    METHODS check_date
+      IMPORTING is_claim TYPE {{p}}if_expense_claim=>ty_claim
+      RAISING   {{p}}cx_claim_error.
+    METHODS check_receipt
+      IMPORTING is_claim TYPE {{p}}if_expense_claim=>ty_claim
+      RAISING   {{p}}cx_claim_error.
+    METHODS check_cost_center
+      IMPORTING is_claim TYPE {{p}}if_expense_claim=>ty_claim
+      RAISING   {{p}}cx_claim_error.
+    METHODS is_supported_currency
+      IMPORTING iv_currency         TYPE {{p}}if_expense_claim=>ty_currency
+      RETURNING VALUE(rv_supported) TYPE abap_bool.
+ENDCLASS.
+
+
+CLASS {{p}}expense_validator IMPLEMENTATION.
+  METHOD validate.
+    check_amount( is_claim ).
+    check_currency( is_claim ).
+    check_date( is_claim ).
+    check_receipt( is_claim ).
+    check_cost_center( is_claim ).
+  ENDMETHOD.
+
+  METHOD check_amount.
+    IF is_claim-amount <= 0.
+      RAISE EXCEPTION TYPE {{p}}cx_claim_amount
+        EXPORTING iv_amount = is_claim-amount.
+    ENDIF.
+  ENDMETHOD.
+
+  METHOD check_currency.
+    IF is_supported_currency( is_claim-currency ) = abap_false.
+      RAISE EXCEPTION TYPE {{p}}cx_claim_currency
+        EXPORTING iv_currency = is_claim-currency.
+    ENDIF.
+  ENDMETHOD.
+
+  METHOD check_date.
+    IF is_claim-claim_date > sy-datum.
+      RAISE EXCEPTION TYPE {{p}}cx_claim_date
+        EXPORTING iv_claim_date = is_claim-claim_date.
+    ENDIF.
+  ENDMETHOD.
+
+  METHOD check_receipt.
+    IF is_claim-amount >= c_receipt_limit AND is_claim-has_receipt = abap_false.
+      RAISE EXCEPTION TYPE {{p}}cx_claim_receipt
+        EXPORTING iv_amount = is_claim-amount.
+    ENDIF.
+  ENDMETHOD.
+
+  METHOD check_cost_center.
+    IF is_claim-amount >= c_cc_limit AND is_claim-cost_center IS INITIAL.
+      RAISE EXCEPTION TYPE {{p}}cx_claim_costcenter
+        EXPORTING iv_amount = is_claim-amount.
+    ENDIF.
+  ENDMETHOD.
+
+  METHOD is_supported_currency.
+    rv_supported = xsdbool( iv_currency = 'EUR' OR iv_currency = 'USD'
+                         OR iv_currency = 'GBP' AND iv_currency = 'CHF' ).
+  ENDMETHOD.
+ENDCLASS.

tasks_gen/eval/G0022/generation.json

@@ -37,6 +37,18 @@
    "change": "budget floor: max_activations 12 -> 14 (reference alone needs 13).",
    "oracle": 100.0,
    "null": 0
+  },
+  {
+   "stage": "manual_fix",
+   "by": "Claude review 2026-10-03",
+   "change": "Review: added hidden test other_currencies_pass (USD, GBP, CHF).",
+   "oracle": 100.0,
+   "null": 0,
+   "mutation": {
+    "valid": 5,
+    "killed": 5,
+    "ok": true
+   }
   }
  ],
  "accepted": true

tasks_gen/eval/G0022/hidden/d01_hidden.clas.abap

@@ -19,6 +19,7 @@ CLASS {{p}}d01_hidden DEFINITION PUBLIC FINAL CREATE PUBLIC
     METHODS cost_center_not_required FOR TESTING.
     METHODS amount_rule_has_priority FOR TESTING.
     METHODS receipt_rule_has_priority FOR TESTING.
+    METHODS other_currencies_pass FOR TESTING.
 
     METHODS build_claim
       IMPORTING iv_amount   TYPE ty_amount
@@ -54,6 +55,19 @@ CLASS {{p}}d01_hidden IMPLEMENTATION.
     ENDTRY.
   ENDMETHOD.
 
+  METHOD other_currencies_pass.
+    LOOP AT VALUE string_table( ( `USD` ) ( `GBP` ) ( `CHF` ) ) INTO DATA(lv_currency).
+      DATA(ls_claim) = build_claim( iv_amount = '250.00' iv_currency = CONV #( lv_currency )
+                                    iv_date = sy-datum iv_cc = 'CC1000'
+                                    iv_receipt = abap_true ).
+      TRY.
+          mo_cut->validate( ls_claim ).
+        CATCH {{p}}cx_claim_error INTO DATA(lx).
+          cl_abap_unit_assert=>fail( msg = |{ lv_currency }: { lx->get_message( ) }| ).
+      ENDTRY.
+    ENDLOOP.
+  ENDMETHOD.
+
   METHOD zero_amount_rejected.
     DATA(ls_claim) = build_claim( iv_amount = 0 iv_currency = 'EUR'
                                   iv_date = sy-datum iv_cc = 'CC1000'

tasks_gen/eval/G0022/mutation.json

@@ -5,11 +5,12 @@
    "object": "{{P}}EXPENSE_VALIDATOR",
    "mutant": "line 54: = -> <> (eq)",
    "status": "killed",
-   "hidden": "3/12",
+   "hidden": "3/13",
    "failed_tests": [
     "COST_CENTER_NOT_REQUIRED",
     "COST_CENTER_REQUIRED",
     "FUTURE_DATE_REJECTED",
+    "OTHER_CURRENCIES_PASS",
     "RECEIPT_NOT_REQUIRED_BELOW",
     "RECEIPT_REQUIRED",
     "RECEIPT_RULE_HAS_PRIORITY",
@@ -22,11 +23,12 @@
    "object": "{{P}}EXPENSE_VALIDATOR",
    "mutant": "line 54: abap_false -> abap_true (bool)",
    "status": "killed",
-   "hidden": "3/12",
+   "hidden": "3/13",
    "failed_tests": [
     "COST_CENTER_NOT_REQUIRED",
     "COST_CENTER_REQUIRED",
     "FUTURE_DATE_REJECTED",
+    "OTHER_CURRENCIES_PASS",
     "RECEIPT_NOT_REQUIRED_BELOW",
     "RECEIPT_REQUIRED",
     "RECEIPT_RULE_HAS_PRIORITY",
@@ -39,11 +41,12 @@
    "object": "{{P}}EXPENSE_VALIDATOR",
    "mutant": "line 61: > -> <= (rel)",
    "status": "killed",
-   "hidden": "4/12",
+   "hidden": "4/13",
    "failed_tests": [
     "COST_CENTER_NOT_REQUIRED",
     "COST_CENTER_REQUIRED",
     "FUTURE_DATE_REJECTED",
+    "OTHER_CURRENCIES_PASS",
     "RECEIPT_NOT_REQUIRED_BELOW",
     "RECEIPT_REQUIRED",
     "RECEIPT_RULE_HAS_PRIORITY",
@@ -55,10 +58,11 @@
    "object": "{{P}}EXPENSE_VALIDATOR",
    "mutant": "line 68: = -> <> (eq)",
    "status": "killed",
-   "hidden": "6/12",
+   "hidden": "6/13",
    "failed_tests": [
     "COST_CENTER_NOT_REQUIRED",
     "COST_CENTER_REQUIRED",
+    "OTHER_CURRENCIES_PASS",
     "RECEIPT_REQUIRED",
     "RECEIPT_RULE_HAS_PRIORITY",
     "TODAY_IS_ACCEPTED",
@@ -68,13 +72,15 @@
   {
    "object": "{{P}}EXPENSE_VALIDATOR",
    "mutant": "line 83: OR -> AND (logic)",
-   "status": "survived",
+   "status": "killed",
-   "hidden": "12/12",
+   "hidden": "12/13",
-   "failed_tests": []
+   "failed_tests": [
+    "OTHER_CURRENCIES_PASS"
+   ]
   }
  ],
  "valid": 5,
- "killed": 4,
+ "killed": 5,
- "kill_rate": 0.8,
+ "kill_rate": 1.0,
  "ok": true
 }

tasks_gen/eval/G0022/review.json

@@ -0,0 +1,7 @@
+{
+ "decision": "accept",
+ "note": "Fixed: added other_currencies_pass; oracle 100, null 0, mutation 5/5.",
+ "by": "Claude",
+ "date": "2026-10-03",
+ "failed_checks": {}
+}

tasks_gen/eval/G0023/review.json

@@ -0,0 +1,9 @@
+{
+ "decision": "flag",
+ "note": "E is weak: new class from complete rules; legacy need not be read. Generator prompt for E fixed. Options: move to A, or regenerate as E.",
+ "by": "Claude",
+ "date": "2026-10-03",
+ "failed_checks": {
+  "6": "zayif"
+ }
+}

tasks_gen/eval/G0024/review.json

@@ -0,0 +1,7 @@
+{
+ "decision": "accept",
+ "note": "Period limits and sums tested; mutation 5/5.",
+ "by": "Claude",
+ "date": "2026-10-03",
+ "failed_checks": {}
+}

tasks_gen/eval/G0025/review.json

@@ -0,0 +1,7 @@
+{
+ "decision": "accept",
+ "note": "Union, duplicates and source literals tested.",
+ "by": "Claude",
+ "date": "2026-10-03",
+ "failed_checks": {}
+}

tasks_gen/eval/G0026/review.json

@@ -0,0 +1,7 @@
+{
+ "decision": "accept",
+ "note": "All rules tested; logic in helper class; mutation 5/5.",
+ "by": "Claude",
+ "date": "2026-10-03",
+ "failed_checks": {}
+}

tasks_gen/eval/G0168/review.json

@@ -0,0 +1,7 @@
+{
+ "decision": "accept",
+ "note": "Real, critical gap: quantity exactly 100 has no volume discount rule.",
+ "by": "Claude",
+ "date": "2026-10-03",
+ "failed_checks": {}
+}

tasks_gen/eval/G0178/review.json

@@ -0,0 +1,7 @@
+{
+ "decision": "accept",
+ "note": "Free text keeps all rules and contract names.",
+ "by": "Claude",
+ "date": "2026-10-03",
+ "failed_checks": {}
+}