Mutation in generator; category H (stop) scoring and judge; category K variants and tool schema variant; evalset plan

- mutation.py: negation mutants, skip WHILE/DO blocks (endless loop blocked RFC ~8 min)
- generator: accept only after mutation check; survivors go back as repair feedback
- runner/judge.py: stop tasks scored 100/30/0; gap by keywords, else judge model
- proxy: tool schema variant generic_v0 (draft); generator make_k_variant
- evalset.py: 88 slots for step F (A-I, H, K)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kral
2026-10-03 05:37:20 +02:00
parent 32f03fcad4
commit 0ba25faeea
20 changed files with 1303 additions and 22 deletions

View File

@@ -0,0 +1,59 @@
{
"task": "G0002",
"mutants": [
{
"object": "{{P}}DELIVERY_SCHEDULE",
"mutant": "line 68: <> -> = (rel)",
"status": "killed",
"hidden": "9/12",
"failed_tests": [
"ADD_DAYS_BACKWARD",
"ADD_DAYS_FORWARD",
"ADD_FORWARD_SKIPS_HOLIDAY"
]
},
{
"object": "{{P}}DELIVERY_SCHEDULE",
"mutant": "line 72: - -> + (arith)",
"status": "invalid",
"hidden": "0/0",
"failed_tests": []
},
{
"object": "{{P}}DELIVERY_SCHEDULE",
"mutant": "line 81: = -> <> (eq)",
"status": "invalid",
"hidden": "0/0",
"failed_tests": []
},
{
"object": "{{P}}DELIVERY_SCHEDULE",
"mutant": "line 91: 5 -> 6 (const)",
"status": "killed",
"hidden": "7/12",
"failed_tests": [
"ADD_DAYS_BACKWARD",
"ADD_DAYS_FORWARD",
"ADD_FORWARD_SKIPS_HOLIDAY",
"NEXT_DAY_FROM_WEEKEND",
"WEEKEND_IS_NOT_WORKING"
]
},
{
"object": "{{P}}DELIVERY_SCHEDULE",
"mutant": "line 98: abap_true -> abap_false (bool)",
"status": "killed",
"hidden": "8/12",
"failed_tests": [
"ADD_FORWARD_SKIPS_HOLIDAY",
"COUNT_WORKING_DAYS_INCLUSIVE",
"HOLIDAY_IS_NOT_WORKING",
"NEXT_DAY_SKIPS_HOLIDAYS"
]
}
],
"valid": 3,
"killed": 3,
"kill_rate": 1.0,
"ok": true
}