Harness: G6 ignores unknown standard superclass; dependency order for seed/reference; max_tokens 80k
- abaplint does not know CX_STATIC_CHECK: exception hierarchies failed G6 (G0022, G0005) - generator orders seed and reference objects by name references (G0022 attempt 1) - max_tokens from pilot distribution (accepted calls 4k-76k) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -92,7 +92,7 @@ python3 -c "from harness.ledger import spent; print(spent())"
|
||||
- Pilot (G0002–G0021) done: 16/20 accepted, 0.16 USD per accepted task (`runs/gen/pilot.json`,
|
||||
analysis in `docs/faz1-tasarim.md` 11e). Generator improved after the pilot: static checks
|
||||
(name length, seed type, reserved words, contract/test classes), local abaplint parser check before SAP,
|
||||
max_tokens 100k, G2 detail in the repair feedback. Runner: G2 works for CDS views with parameters.
|
||||
max_tokens 80k, G2 detail in the repair feedback. Runner: G2 works for CDS views with parameters.
|
||||
The improved generator is not yet tested with new tasks.
|
||||
- G0020 (rejected) passes with oracle 100 after one fix (remove parameter `default`). G0013 (accepted) has a test
|
||||
class in its contract: fix in review.
|
||||
|
||||
@@ -565,7 +565,7 @@ Başarısız denemelerin nedenleri (oracle koşusu < 100 olan 37 deneme):
|
||||
| Ayrılmış kelime alan adı (HOURS, MODE) | 2 | G0017, G0021 | Statik kontrol + prompt kuralı |
|
||||
| Parametreli CDS'te G2 her zaman düşüyor (**harness hatası**: `SELECT *` parametresiz çalışmaz) | 3 | G0020 | Runner: SELECT sütun döndürmezse eleman adları kaynaktan okunur. G0020 tek satır düzeltmeyle oracle 100 |
|
||||
| Tablo seed'i DDLS tipiyle | 2 | G0003 | Statik kontrol + prompt kuralı |
|
||||
| Geçersiz / boş JSON | 3 | G0002, G0006, G0010 | `raw_decode`, `max_tokens` 100k, boş içerikte yeniden deneme |
|
||||
| Geçersiz / boş JSON | 3 | G0002, G0006, G0010 | `raw_decode`, `max_tokens` 80k (kabul edilen son çağrılar 4k–76k, p90 57k; 60k kabul edilen 2 çağrıyı keserdi), boş içerikte yeniden deneme |
|
||||
| Eski seed kodu 816'da aktive olmuyor (yeni/eski SQL karışık) | 1 | G0016 | Prompt kuralı |
|
||||
| Referans mantık hatası (gizli test düşüyor), tip uyumsuzluğu, CX_CDS_FAILURE | 6 | G0013, G0014, G0018, G0019, G0015 | Onarım turu çözdü |
|
||||
| Sözleşmede sınıf var, yerel testi yok → own_tests 7,5 | 2 | G0011 | Statik kontrol: her referans sınıfının `testclasses_file`'ı olmalı (istisna sınıfları hariç) |
|
||||
|
||||
@@ -19,7 +19,8 @@ from .runner import Runner
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
ABAPLINT = os.path.join(ROOT, "node_modules", ".bin", "abaplint")
|
||||
MAX_OUT_TOKENS = 100000
|
||||
# pilot: accepted final calls used 4k-76k output tokens (p90 57k); one runaway call used 393k
|
||||
MAX_OUT_TOKENS = 80000
|
||||
ABAPLINT_VERSIONS = {"v702", "v740sp05", "v750", "v758"}
|
||||
EXAMPLE_FOR = {"CLAS": "T01", "INTF": "T01", "FUNC": "T13", "PROG": "T14", "DDLS": "T15", "TABL": "T15"}
|
||||
CATEGORIES = {
|
||||
@@ -256,9 +257,33 @@ def check_bundle(b):
|
||||
return errs
|
||||
|
||||
|
||||
def dependency_order(objs, files):
|
||||
"""Stable order in which every object comes after the objects that its sources name
|
||||
(for example exception classes before the class that raises them). A cycle keeps the given order."""
|
||||
def text(o):
|
||||
return " ".join(files.get(o.get(k), "") for k in ("file", "testclasses_file")).upper()
|
||||
names = [o.get("name", "").upper() for o in objs]
|
||||
needs = []
|
||||
for o in objs:
|
||||
src, own = text(o), o.get("name", "").upper()
|
||||
dep = {n for n in names if n != own and re.search(rf"(?<![\w{{}}]){re.escape(n)}(?!\w)", src)}
|
||||
if o.get("functionGroup"):
|
||||
dep.add(o["functionGroup"].upper())
|
||||
needs.append(dep & set(names))
|
||||
out, done = [], set()
|
||||
while len(out) < len(objs):
|
||||
ready = [i for i, o in enumerate(objs) if i not in done and needs[i] <= {names[j] for j in done}]
|
||||
i = ready[0] if ready else min(set(range(len(objs))) - done)
|
||||
done.add(i)
|
||||
out.append(objs[i])
|
||||
return out
|
||||
|
||||
|
||||
def write_bundle(b, task_dir, task_id):
|
||||
os.makedirs(task_dir, exist_ok=True)
|
||||
b["task"]["id"] = task_id
|
||||
for k in ("seed", "reference"):
|
||||
b["task"][k] = dependency_order(b["task"].get(k, []), b["files"])
|
||||
json.dump(b["task"], open(os.path.join(task_dir, "task.json"), "w"), indent=2)
|
||||
for rel, content in b["files"].items():
|
||||
path = os.path.join(task_dir, rel)
|
||||
|
||||
@@ -344,7 +344,10 @@ class Runner:
|
||||
# Release gate only on contract objects. Helper objects (smoke runners, test classes)
|
||||
# can use standard APIs that abaplint does not know; A4H syntax check covers them.
|
||||
contract_files = {f"{c['name'].lower()}.{c['type'].lower()}.abap" for c in contract}
|
||||
# abaplint does not know standard superclasses (CX_STATIC_CHECK ...); an exception hierarchy
|
||||
# then gives "Super class ... not found or contains errors". A4H activation (G1) covers this.
|
||||
g["G6_release"] = not any(i["rule"] == "check_syntax" and i["file"] in contract_files
|
||||
and not re.match(r"Super class .* not found or contains errors", i["msg"] or "")
|
||||
for i in issues)
|
||||
|
||||
# 8 score
|
||||
|
||||
Reference in New Issue
Block a user