erhan/abap-llm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Hard batch G0022-G0026: 5/5 accepted; budget floor, CDS checks, mutation check

Browse Source 
- generator: budget floor (2x oracle activations, 3x calls), static checks for CDS
  $parameters and UNION annotation
- harness/mutation.py: deterministic mutants of the reference; hidden tests must fail
- G0022 revalidated with harness fixes: oracle 100, null 0
- results in docs/faz1-tasarim.md 11f

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kral
2026-10-03 05:16:30 +02:00
parent 3570d7ef9b
commit 32f03fcad4
53 changed files with 2931 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
tasks_gen/eval/G0023/generation.json Normal file
View File

@@ -0,0 +1,14 @@
{
"id": "G0023",
"pool": "eval",
"object_type": "CLAS",
"category": "E",
"attempts": [
{
"stage": "validate",
"oracle": 100.0,
"null": 0
}
],
"accepted": true
}

73
tasks_gen/eval/G0023/hidden/e01_hidden.clas.abap Normal file
View File

@@ -0,0 +1,73 @@
CLASS {{p}}e01_hidden DEFINITION PUBLIC FINAL CREATE PUBLIC
FOR TESTING DURATION SHORT RISK LEVEL HARMLESS.
PRIVATE SECTION.
DATA mo_cut TYPE REF TO {{p}}if_freight_calc.
METHODS setup.
METHODS assert_cost
IMPORTING iv_zone TYPE {{p}}if_freight_calc=>ty_zone
iv_weight TYPE {{p}}if_freight_calc=>ty_weight
iv_exp TYPE {{p}}if_freight_calc=>ty_amount.
METHODS unknown_zone_is_zero FOR TESTING.
METHODS minimum_applies_after_charge FOR TESTING.
METHODS weight_charge_rounds_half_up FOR TESTING.
METHODS all_zones_are_rated FOR TESTING.
METHODS surcharge_uses_minimum FOR TESTING.
METHODS heavy_discount_on_total FOR TESTING.
METHODS no_discount_at_exactly_1000 FOR TESTING.
METHODS discount_rounds_half_up FOR TESTING.
METHODS small_weight_uses_minimum FOR TESTING.
ENDCLASS.
CLASS {{p}}e01_hidden IMPLEMENTATION.
METHOD setup.
mo_cut = NEW {{p}}freight_calculator( ).
ENDMETHOD.
METHOD assert_cost.
cl_abap_unit_assert=>assert_equals(
exp = iv_exp
act = mo_cut->calculate( iv_zone = iv_zone iv_weight = iv_weight )
msg = |zone { iv_zone }, weight { iv_weight }| ).
ENDMETHOD.
METHOD unknown_zone_is_zero.
assert_cost( iv_zone = 'XX' iv_weight = '500.000' iv_exp = '0.00' ).
ENDMETHOD.
METHOD minimum_applies_after_charge.
assert_cost( iv_zone = 'DE' iv_weight = '1.000' iv_exp = '26.63' ).
ENDMETHOD.
METHOD weight_charge_rounds_half_up.
assert_cost( iv_zone = 'CN' iv_weight = '20.004' iv_exp = '54.76' ).
ENDMETHOD.
METHOD all_zones_are_rated.
assert_cost( iv_zone = 'DE' iv_weight = '100.000' iv_exp = '103.84' ).
assert_cost( iv_zone = 'FR' iv_weight = '20.000' iv_exp = '39.78' ).
assert_cost( iv_zone = 'NL' iv_weight = '50.000' iv_exp = '50.11' ).
assert_cost( iv_zone = 'US' iv_weight = '50.000' iv_exp = '97.65' ).
assert_cost( iv_zone = 'CN' iv_weight = '100.000' iv_exp = '164.25' ).
ENDMETHOD.
METHOD surcharge_uses_minimum.
assert_cost( iv_zone = 'NL' iv_weight = '0.000' iv_exp = '21.10' ).
ENDMETHOD.
METHOD heavy_discount_on_total.
assert_cost( iv_zone = 'DE' iv_weight = '1500.000' iv_exp = '1302.63' ).
ENDMETHOD.
METHOD no_discount_at_exactly_1000.
assert_cost( iv_zone = 'DE' iv_weight = '1000.000' iv_exp = '918.56' ).
ENDMETHOD.
METHOD discount_rounds_half_up.
assert_cost( iv_zone = 'DE' iv_weight = '1090.070' iv_exp = '950.10' ).
ENDMETHOD.
METHOD small_weight_uses_minimum.
assert_cost( iv_zone = 'FR' iv_weight = '5.000' iv_exp = '32.25' ).
ENDMETHOD.
ENDCLASS.

62
tasks_gen/eval/G0023/reference/freight_calculator.clas.abap Normal file
View File

@@ -0,0 +1,62 @@
CLASS {{p}}freight_calculator DEFINITION PUBLIC FINAL CREATE PUBLIC.
PUBLIC SECTION.
INTERFACES {{p}}if_freight_calc.
PRIVATE SECTION.
TYPES:
BEGIN OF ty_rate,
zone TYPE {{p}}if_freight_calc=>ty_zone,
base TYPE {{p}}if_freight_calc=>ty_amount,
per_kg TYPE {{p}}if_freight_calc=>ty_amount,
minimum TYPE {{p}}if_freight_calc=>ty_amount,
surcharge TYPE p LENGTH 4 DECIMALS 3,
END OF ty_rate,
tt_rate TYPE SORTED TABLE OF ty_rate WITH UNIQUE KEY zone.
CONSTANTS:
c_heavy_limit TYPE {{p}}if_freight_calc=>ty_weight VALUE '1000.000',
c_discount TYPE p LENGTH 3 DECIMALS 2 VALUE '0.95'.
METHODS rates RETURNING VALUE(rt_rate) TYPE tt_rate.
ENDCLASS.
CLASS {{p}}freight_calculator IMPLEMENTATION.
METHOD {{p}}if_freight_calc~calculate.
DATA(lt_rate) = rates( ).
READ TABLE lt_rate INTO DATA(ls_rate) WITH TABLE KEY zone = iv_zone.
IF sy-subrc <> 0.
rv_cost = 0.
RETURN.
ENDIF.
DATA lv_charge TYPE {{p}}if_freight_calc=>ty_amount.
lv_charge = ls_rate-per_kg * iv_weight.
DATA lv_freight TYPE {{p}}if_freight_calc=>ty_amount.
lv_freight = ls_rate-base + lv_charge.
IF lv_freight < ls_rate-minimum.
lv_freight = ls_rate-minimum.
ENDIF.
DATA lv_surcharge TYPE {{p}}if_freight_calc=>ty_amount.
lv_surcharge = lv_freight * ls_rate-surcharge.
DATA lv_total TYPE {{p}}if_freight_calc=>ty_amount.
lv_total = lv_freight + lv_surcharge.
IF iv_weight > c_heavy_limit.
lv_total = lv_total * c_discount.
ENDIF.
rv_cost = lv_total.
ENDMETHOD.
METHOD rates.
rt_rate = VALUE #(
( zone = 'DE' base = '12.50' per_kg = '0.85' minimum = '25.00' surcharge = '0.065' )
( zone = 'FR' base = '15.00' per_kg = '1.10' minimum = '30.00' surcharge = '0.075' )
( zone = 'NL' base = '10.00' per_kg = '0.75' minimum = '20.00' surcharge = '0.055' )
( zone = 'US' base = '20.00' per_kg = '1.40' minimum = '45.00' surcharge = '0.085' )
( zone = 'CN' base = '25.00' per_kg = '1.25' minimum = '50.00' surcharge = '0.095' ) ).
ENDMETHOD.
ENDCLASS.

75
tasks_gen/eval/G0023/reference/freight_calculator.testclasses.abap Normal file
View File

@@ -0,0 +1,75 @@
CLASS ltc_freight_calculator DEFINITION FINAL FOR TESTING
DURATION SHORT RISK LEVEL HARMLESS.
PRIVATE SECTION.
DATA mo_cut TYPE REF TO {{p}}if_freight_calc.
METHODS setup.
METHODS unknown_zone FOR TESTING.
METHODS minimum_charge FOR TESTING.
METHODS all_zones FOR TESTING.
METHODS heavy_discount FOR TESTING.
METHODS no_discount_at_1000 FOR TESTING.
METHODS rounding_half_up FOR TESTING.
ENDCLASS.
CLASS ltc_freight_calculator IMPLEMENTATION.
METHOD setup.
mo_cut = NEW {{p}}freight_calculator( ).
ENDMETHOD.
METHOD unknown_zone.
DATA lv_exp TYPE {{p}}if_freight_calc=>ty_amount.
lv_exp = '0.00'.
cl_abap_unit_assert=>assert_equals(
exp = lv_exp
act = mo_cut->calculate( iv_zone = 'XX' iv_weight = '10.000' ) ).
ENDMETHOD.
METHOD minimum_charge.
DATA lv_exp TYPE {{p}}if_freight_calc=>ty_amount.
lv_exp = '26.63'.
cl_abap_unit_assert=>assert_equals(
exp = lv_exp
act = mo_cut->calculate( iv_zone = 'DE' iv_weight = '1.000' ) ).
ENDMETHOD.
METHOD all_zones.
DATA lv_exp TYPE {{p}}if_freight_calc=>ty_amount.
lv_exp = '103.84'.
cl_abap_unit_assert=>assert_equals(
exp = lv_exp
act = mo_cut->calculate( iv_zone = 'DE' iv_weight = '100.000' ) msg = `DE` ).
lv_exp = '39.78'.
cl_abap_unit_assert=>assert_equals(
exp = lv_exp
act = mo_cut->calculate( iv_zone = 'FR' iv_weight = '20.000' ) msg = `FR` ).
lv_exp = '164.25'.
cl_abap_unit_assert=>assert_equals(
exp = lv_exp
act = mo_cut->calculate( iv_zone = 'CN' iv_weight = '100.000' ) msg = `CN` ).
ENDMETHOD.
METHOD heavy_discount.
DATA lv_exp TYPE {{p}}if_freight_calc=>ty_amount.
lv_exp = '1302.63'.
cl_abap_unit_assert=>assert_equals(
exp = lv_exp
act = mo_cut->calculate( iv_zone = 'DE' iv_weight = '1500.000' ) ).
ENDMETHOD.
METHOD no_discount_at_1000.
DATA lv_exp TYPE {{p}}if_freight_calc=>ty_amount.
lv_exp = '918.56'.
cl_abap_unit_assert=>assert_equals(
exp = lv_exp
act = mo_cut->calculate( iv_zone = 'DE' iv_weight = '1000.000' ) ).
ENDMETHOD.
METHOD rounding_half_up.
DATA lv_exp TYPE {{p}}if_freight_calc=>ty_amount.
lv_exp = '54.76'.
cl_abap_unit_assert=>assert_equals(
exp = lv_exp
act = mo_cut->calculate( iv_zone = 'CN' iv_weight = '20.004' ) ).
ENDMETHOD.
ENDCLASS.

61
tasks_gen/eval/G0023/seed/freight_legacy.clas.abap Normal file
View File

@@ -0,0 +1,61 @@
CLASS {{p}}freight_legacy DEFINITION PUBLIC FINAL CREATE PUBLIC.
PUBLIC SECTION.
INTERFACES {{p}}if_freight_calc.
ENDCLASS.
CLASS {{p}}freight_legacy IMPLEMENTATION.
METHOD {{p}}if_freight_calc~calculate.
DATA: lv_base TYPE {{p}}if_freight_calc=>ty_amount,
lv_per_kg TYPE {{p}}if_freight_calc=>ty_amount,
lv_minimum TYPE {{p}}if_freight_calc=>ty_amount,
lv_surcharge TYPE p LENGTH 4 DECIMALS 3,
lv_charge TYPE {{p}}if_freight_calc=>ty_amount,
lv_freight TYPE {{p}}if_freight_calc=>ty_amount,
lv_fee TYPE {{p}}if_freight_calc=>ty_amount,
lv_total TYPE {{p}}if_freight_calc=>ty_amount.
CASE iv_zone.
WHEN 'DE'.
lv_base = '12.50'.
lv_per_kg = '0.85'.
lv_minimum = '25.00'.
lv_surcharge = '0.065'.
WHEN 'FR'.
lv_base = '15.00'.
lv_per_kg = '1.10'.
lv_minimum = '30.00'.
lv_surcharge = '0.075'.
WHEN 'NL'.
lv_base = '10.00'.
lv_per_kg = '0.75'.
lv_minimum = '20.00'.
lv_surcharge = '0.055'.
WHEN 'US'.
lv_base = '20.00'.
lv_per_kg = '1.40'.
lv_minimum = '45.00'.
lv_surcharge = '0.085'.
WHEN 'CN'.
lv_base = '25.00'.
lv_per_kg = '1.25'.
lv_minimum = '50.00'.
lv_surcharge = '0.095'.
WHEN OTHERS.
rv_cost = 0.
RETURN.
ENDCASE.
lv_charge = lv_per_kg * iv_weight.
lv_freight = lv_base + lv_charge.
IF lv_freight < lv_minimum.
lv_freight = lv_minimum.
ENDIF.
lv_fee = lv_freight * lv_surcharge.
lv_total = lv_freight + lv_fee.
IF iv_weight > '1000.000'.
lv_total = lv_total * '0.95'.
ENDIF.
rv_cost = lv_total.
ENDMETHOD.
ENDCLASS.

11
tasks_gen/eval/G0023/seed/if_freight_calc.intf.abap Normal file
View File

@@ -0,0 +1,11 @@
INTERFACE {{p}}if_freight_calc PUBLIC.
TYPES:
ty_zone TYPE c LENGTH 2,
ty_weight TYPE p LENGTH 9 DECIMALS 3,
ty_amount TYPE p LENGTH 7 DECIMALS 2.
METHODS calculate
IMPORTING iv_zone TYPE ty_zone
iv_weight TYPE ty_weight
RETURNING VALUE(rv_cost) TYPE ty_amount.
ENDINTERFACE.

69
tasks_gen/eval/G0023/spec.md Normal file
View File

@@ -0,0 +1,69 @@
# 1. Goal
A freight forwarder calculates the freight cost of a shipment. A legacy class
calculates this cost today. Write the calculation as a clean class. The cost of
every shipment must stay the same.
# 2. Open questions
None.
# 3. Context
- The interface {{P}}IF_FREIGHT_CALC exists in package $TMP and is active.
- The legacy class {{P}}FREIGHT_LEGACY exists in package $TMP and is active. It
implements {{P}}IF_FREIGHT_CALC. Its results are the reference for the new class.
- The interface declares the types TY_ZONE (character, length 2), TY_WEIGHT
(packed number, length 9, 3 decimal places) and TY_AMOUNT (packed number,
length 7, 2 decimal places), and the method CALCULATE.
# 4. Contract
- Create the class {{P}}FREIGHT_CALCULATOR in package $TMP.
- The class is public, final, and has a public constructor without parameters.
- The class implements {{P}}IF_FREIGHT_CALC.
- Do not add other public methods.
# 5. Business rules
All amounts are packed numbers with 2 decimal places. Round an amount to
2 decimal places before you use it in the next step. Round half away from zero.
1. Zone
The zone is a two-character code. These zones exist:
zone base per kg minimum surcharge factor
DE 12.50 0.85 25.00 0.065
FR 15.00 1.10 30.00 0.075
NL 10.00 0.75 20.00 0.055
US 20.00 1.40 45.00 0.085
CN 25.00 1.25 50.00 0.095
If the zone is not one of these, the result is 0.00 and no other rule applies.
2. Weight charge
weight charge = weight * per kg of the zone, rounded to 2 decimal places.
3. Freight
freight = base of the zone + weight charge.
If freight is less than the minimum of the zone, freight is the minimum.
4. Surcharge
surcharge = freight * surcharge factor of the zone, rounded to 2 decimal places.
5. Total
total = freight + surcharge.
6. Heavy shipment
If the weight is greater than 1000.000 kg, total = total * 0.95, rounded to
2 decimal places.
7. The result is total.
# 6. Constraints
- Release target: SAP_BASIS 816 (ABAP Platform 2025).
- Coding standards: Clean ABAP. Method length below 40 statements. No global
variables. No comment that restates the code.
- Out of scope: do not change {{P}}IF_FREIGHT_CALC and {{P}}FREIGHT_LEGACY.
# 7. Acceptance
- The class is active and has no syntax error.
- The hidden tests pass.
- Write your own ABAP Unit tests for the class.

58
tasks_gen/eval/G0023/task.json Normal file
View File

@@ -0,0 +1,58 @@
{
"id": "G0023",
"category": "E",
"object_type": "CLAS",
"difficulty": 2,
"release_target": "v816",
"expected_outcome": "implement",
"budget": {
"max_tool_calls": 60,
"max_activations": 15
},
"seed": [
{
"type": "INTF",
"name": "{{P}}IF_FREIGHT_CALC",
"file": "seed/if_freight_calc.intf.abap",
"description": "Freight calculation contract with packed number types"
},
{
"type": "CLAS",
"name": "{{P}}FREIGHT_LEGACY",
"file": "seed/freight_legacy.clas.abap",
"description": "Legacy freight calculator with packed arithmetic and a CASE over zones"
}
],
"contract": [
{
"type": "CLAS",
"name": "{{P}}FREIGHT_CALCULATOR",
"implements": "{{P}}IF_FREIGHT_CALC"
}
],
"out_of_scope": [
"{{P}}IF_FREIGHT_CALC",
"{{P}}FREIGHT_LEGACY"
],
"hidden_tests": [
{
"type": "CLAS",
"name": "{{P}}E01_HIDDEN",
"file": "hidden/e01_hidden.clas.abap",
"description": "E01 hidden tests for the refactored freight calculator"
}
],
"reference": [
{
"type": "CLAS",
"name": "{{P}}FREIGHT_CALCULATOR",
"file": "reference/freight_calculator.clas.abap",
"description": "Refactored freight calculator with a rate table",
"testclasses_file": "reference/freight_calculator.testclasses.abap"
}
],
"craft_checks": [
"method_length",
"no_global_variables"
]
}