A character expression of literals and %CHAR of whole numbers recovers into a character field and into a varying-length field, and a static int(20) and int(3) keep their INZ #806

Manually merged
binjovi-bot merged 4 commits from lcat into trunk 2026-11-15 17:27:36 +00:00
Owner

A character expression of literals and %CHAR of whole numbers recovers into a character field and into a varying-length field, and a static int(20) and int(3) keep their INZ

s = 'Total ' + %char(n) recovered (lc2-cat-char-int). Anything more stopped at control_flow /
operation_semantics on the first label of the %CHAR strip, with %SUBST(A.1: 1: 10) = S as the partial
text: a literal after the %CHAR, a second %CHAR, a one-byte literal, and a varying-length target.
rpg_char_chain.zig (new, PURE) reads the whole expression as one unit; rpg_trim.charAssign calls it
where it called charConcat, which is gone.

The unit

The compiler builds the expression in one temporary buffer CB behind one running length T1, then one tail
reads (CB, T1). After the strip of the digits of the first %CHAR (rpg_trim.charStrip, 59 operations):

piece operations
FIRSTLIT, 27 27 P; 33 LIT; 5; 113; 5; 33 CB; 113; 5; 27 L; 35 T1; 27 L; 1; 35 T2; 76 4; 97; 27 L; 62; 113; 5; 27 N2; 1; 27 T2; 47; 35 T1; 76 4; 97; 3
APPLIT, 18 33 CB; 113; 33 LIT; 5; 27 T1; 1; 35 O; 62; 113; 5; 27 L; 1; 27 O; 47; 35 T1; 76 4; 97; 3
APPDIG, 20, after a numeric load and a second strip 27 P2; 33 CB; 113; 5; 113; 5; 27 T1; 1; 35 O; 62; 113; 5; 27 N2; 1; 27 O; 47; 35 T1; 76 4; 97; 3
tail, fixed field rpg_trim.copyTail, mode joined: 34 S 0; 113; 1; 35 Q; 33 CB; 113; ... the copy of min(T1, W) bytes and the blank fill
tail, varying field 34 V 0; 113; 1; 35 Q; 27 2; 62; 113; 33 CB; 113; 27 MAX; 27 T1; 35 A; 35 C; 27 A; 27 C; 11; 17 L; 27 C; 35 A; 15 L 1; 27 A; 35 A; 27 A; 76 4; 97; 3; 27 Q; 27 A; 76 2; 42 V 0 U2

The unit becomes the saved digits (37 T), op_char_integer with the first literal, then for each piece
a 27 LIT and 47, or the numeric load, its 37 T, op_char_integer and 47, and at last 35 target.
No evaluator rule changed: op_char_integer with one operand, 27 of a constant and 47 existed.

Facts decided

The request is requests/lcat/concat (21 cases, one statement each on one template, captures in
captures/lcat-concat); the probes are lcat-cat-* in SC-ILE/EXPERIMENTS.

  • CB is the sum, exactly, in every probe. The size of CB is the bytes of the literals plus D + 1 for each
    %CHAR, where D + 1 is the size of the strip's buffer: cat-base 6 + 11 = 17; cat-trail (+ ' ok') 20;
    cat-trail-long (+ 12 bytes) 29; cat-one-byte 18; cat-first-one 1 + 11 + 3 = 15; cat-int5 6 + 6 + 3 = 15;
    cat-int20 6 + 21 + 3 = 30; cat-two-char 6 + 11 + 3 + 11 = 31; cat-two-packed 6 + 11 + 3 + 6 = 26;
    cat-two-char-lit 35; cat-two-lits 24; cat-vary-lf 19. But the committed lc2-cat-char-int has the same
    expression with CB of 20 where the sum is 17: the buffer is one compiler temporary of the procedure, and the
    largest use of it (a + b in the same module, 20 bytes) sets its size. So the rule is a lower bound: the reader
    refuses a CB smaller than the sum, and never demands the sum.

  • MAX is the declared length of the target. cat-vary-tgt (v varchar(40): the symbol has 42 bytes, the
    tail loads the constant 40) against cat-vary-max20 (varchar(20): 22 bytes, 20), which differ in the
    target only; cat-vary-short (varchar(8): 10 bytes, 8) holds a chain of up to 20 bytes in a target of 8, and
    the object is the same unit: the compiler does not fold min(T1, MAX) for a chain whose length it knows only at
    run time. The reader requires MAX = symbol length - 2.

  • A one-byte literal is the same unit with L = 1. cat-trail (' ok': a text constant, L the integer 3)
    against cat-one-byte ('.'), cat-lf (LF = X'25') and cat-first-one ('#' as the first literal):
    the literal symbol is an integer constant of one byte (75, 37, 123; word(4) == 1) and L is the integer 1.
    The same 18 operations (APPLIT) and the same 27 (FIRSTLIT). rpg_literals.constantText writes it as '.'
    or X'25'; the compiler gave the one-byte constant for LF and for the quoted '.' alike.

  • The scratch is not distinct, so the reader is positional. cat-two-char: the scratch O of the APPLIT is
    the symbol that the strip before it used for N2 (294); the second strip stores the same value, buffer, counter
    and N2 (293, 295, 297, 294) and a new pointer (301); the scratch of the APPDIG is the T2 of the FIRSTLIT
    (299). charConcat required all temporaries pairwise distinct and refused this. Now each piece is read in
    order, and the rule is what the piece needs: O is not CB, not T1 (the state that lives on), and not the
    pointer or the digits length that the same piece reads after it stores O; the strip of a later digits piece
    does not write CB or T1 (value, buffer, pointer, counter, length).

  • The second source. The numeric load before the strip of a later %CHAR is one of three sequences, each
    measured in the place of the second %CHAR (cat-two-char: int(10), cat-two-int20: int(20), cat-two-int5:
    int(5), cat-int3-second: int(3), cat-two-packed: packed(5:0)), D the digits of the strip:

    source operations
    int(10), int(20) 29 N 0 T; 86 D; 61 D 0 0; 85 D 1
    int(5), int(3) 29 N 0 T; 75 4; 86 D; 61 D 0 0; 85 D 1, D = 5 and 3
    packed(D:0) 29 N 0 T; 61 D 0 0; 85 D 1

    The reader takes these three and no other run of the operations, and requires the D of the 86, the 61 and
    the 85 to be the digits of the strip, the decimal positions of the 61 to be 0, and the widening to be 75 4.
    uns(10) loads with 88 D where the int has 86 D (cat-two-uns10) and zoned(5:0) with 84 5 1
    (cat-two-zoned): measured, and refused. The earlier reader took any run of 29, 61, 75, 85 and 86
    with operands unchecked, which the captures did not prove. The order of the digits is the order of the source
    (n then m).

  • The bound is 8 pieces. cat-eight-pieces ('Total ' + %char(n) and eight literals) is read with all eight;
    cat-nine-pieces stops the reader at the eighth, where the ninth is not the tail, and the statement keeps its
    failure line (read s14).

  • A one-byte text constant is not a one-byte literal. The compiler wrote an integer constant of one byte in
    every probe. A text constant of one byte (a test's changed object) is refused at the first literal (read s02) and
    at a literal piece (literalPiece s02).

  • Consecutive literals are two pieces (cat-two-lits): the compiler does not join them.

  • A target shorter than the chain (cat-short-tgt, char(8)): copyTail keeps its minimum block.

What recovers

Eighteen of the 21 probes recover in line 16 that did not before; cat-base was recovered
before, and the two cat-vary-piece* stay open (below). Counts of emitted and failure lines of the 21 stripped
twins, base 0.1.521 against this change:

case before after
cat-trail, cat-trail-long, cat-one-byte, cat-lf, cat-first-one, cat-two-lits 1 emitted, 2 failures 2, 0
cat-two-char, cat-two-packed, cat-two-char-lit, cat-int5, cat-int20, cat-short-tgt 1, 2 2, 0
cat-vary-tgt, cat-vary-max20, cat-vary-short, cat-vary-base, cat-vary-lf, cat-vary-two-char 1, 2 2, 0
cat-base (the control) 2, 1 2, 0
cat-vary-piece, cat-vary-piece-lit 1, 2 1, 1 (refused)

The second failure of the base is statement 17, unresolved suffix after a proved *INLR write. The earlier text of
this document called it the template's (*inlr = *on; and no procedure). It is not: it is the unread static
initialization of dcl-s q int(20) inz(5), the one declaration that the template shares, and it goes with the reader
below. cat-int20 was counted recovered with q declared int(20) and no INZ, so the recompiled program printed
Total 0 ok where the original prints Total 5 ok; now the emit declares inz(5).

The varying target is declared varchar(MAX) by the existing proof of a varying assignment (the pair of the copy
and the 2-byte length store), not by this reader; the tests check the declaration (varchar(40), (20), (8))
beside the assignment.

lc2-cat-varchar-tgt

v = a + b of two char(10) fields into a varchar(40): the same buffer and the same store of V, but the
two fields are copied to CB with no strip, and the length is the constant K = 20 that the compiler folded
(27 K; 35 A, no minimum, no label). rpg_char_chain.pair reads it (S267 = S255 + S256). The one data
point is K = LA + LB <= MAX, so the reader requires exactly that, the fields to be character fields that are
not the target, CB at least K, and refuses the rest: a K that is not the sum, a sum above MAX (the folded
minimum is not measured), a literal or a third piece.

The static int(20) and int(3)

dcl-s q int(20) inz(5) is 27 K; 37 q 0 T in the static initialization: K a constant of the signed class and size
8 (the value in the 8 bytes at byte 12 of its symbol), T the type 1 0 0 0 0 0 0 8, q a static field of kind 1 and
length 8. No reader took it (init.store takes the 4-byte signed form and the unsigned forms of 1, 2, 4 and 8 bytes,
halfword the 2-byte one), so the declaration lost its INZ and the failure of the unread store became the
statement-17 failure above. rpg_static_init.quadword reads it, as an integer of an 8-byte static field.
Measured with 5, 42 (lcat-q20-two), -5 (lcat-q20-neg), 4294967296 (lcat-q20-big) and 9223372036854775807
(lcat-q20-max); with no INZ (lcat-q20-none) the compiler stores 0, which is the default, so the declaration
has none. uns(20) (lcat-q20-uns) was read before and is the control.

int(3), the 1-byte signed field, was the same gap (lc2-arg-uns5 has int(3) inz(7) in the source, and the emit
declared int(3)), and this lane's reader takes an int(3) as the first and as the second %CHAR
(lcat-cat-int3-first, -second, -neg: the load is 29; 75 4; 86 3; 61 3 0 0; 85 3 1, the widening of an int(5)
with D = 3), so it is read as well: rpg_static_init.singlebyte, with a 1-byte signed constant and a 1-byte signed
access type. The symbol decoder gives a 1-byte constant as an unsigned byte (-7 is 249); the reader makes it the signed
byte, and the probe of inz(-7) (lcat-cat-int3-neg) is the test that it does. The program of these three probes
declares int(3) inz(7) and int(3) inz(-7) and the emit now writes both.

The reader reaches beyond the probes of this lane, because every program with such a field had the unread store
as a failure: the emit gate (emit-all.sh of the committed twins against the 0.1.521 baseline) has worse 0, better 21
and 70 twins newly clean, and twins.sh 72 groups newly recovered with no regression. Five tests of
rpg_lr3b_test.zig (lr3-diff-ts-*) had written the gap down ("the 8-byte int(20) start value of the target has no
rule", the target named V00255); they now expect A_00255 int(20) and an emit with no failure line, which is what
the source of the probe says (dcl-s r int(20); r = %diff(a : b : *days)).

Measured: the 33 public scout programs

capture-emit-diff.sh (stripped twins of captures/l10-scout2 and l10-scout2b), base 0.1.521 against this
change. 31 programs are the same; two change:

program statements emitted failure lines the statements that recover
bluemoonjp-1 18 14 to 15 4 to 3 26800: msgText = 'lit' + %CHAR(i) + 'lit' + %CHAR(v) + 'lit' into char(200), packed(5:0) as the second source
advent-2 65 40 to 43 25 to 22 416, 479 and 488: message = 'lit' + %CHAR(n) + '.' + X'25' into varchar(100) (the source says LF)

agree.sh over the 33: same 159 to 160, refused 149 to 148, suspect 13 to 13, wrong parameters 0 to 0,
declarations the same as the source 35 to 36 (advent-2 is not aligned to its source and is not scored; its three
statements were read against the lines 431, 494 and 503 of the source by hand). emit-lint.sh: 0 findings. The
other 31 emit byte for byte the same text as the base, so the reader of the static int(20) changes no public program
beyond these.

Not done, and why

  • A varying field as a piece (cat-vary-piece, cat-vary-piece-lit; advent-2 424, advent-3 412 and advent-4 412 hold it
    beside the call result %CHAR(errno()), which is the other lane's):
    the piece is 33 CB; 113; 34 V2 0; 1; 32 V2 0 U2; 76 8; 1; 76 4; 35 TA; 27 TA; 27 MAX; 12; 17 L; 27 115; 35 ST; 99 3 0 0 0; 15 L 1; ..., the direct form of the length check, which signals error 115 by a store and a
    99 3 0 0 0. rpg_varying_data.head reads the pointer form (29 P 0 PTR, a call of an imported
    procedure). It does not fall out of this reader, so it refuses and the statement keeps its failure line.
  • The call result as the source (%CHAR(errno())), the += consumer and the DSPLY tail: other lanes.
  • uns(10) and zoned(5:0) as a second source: measured, refused (above).
  • A 61 with decimal positions (a packed or zoned source with decimals) as a source: not measured, refused.
  • The numeric load before the first strip is not checked by the chain: the strip is where it starts, and the evaluator
    reads the load. The three sequences are measured in the first place for int(10), int(5), int(3) and int(20) only;
    uns(10), zoned and packed as the first source are not measured.
  • v = a + b with a sum above MAX, with a literal, or with three fields: one data point only.

Probes

lcat-cat-base, -trail, -trail-long, -one-byte, -lf, -first-one, -two-char, -two-packed,
-two-char-lit, -two-lits, -int5, -int20, -short-tgt, -vary-tgt, -vary-max20, -vary-short,
-vary-base, -vary-lf, -vary-two-char, -vary-piece, -vary-piece-lit; the committed
lc2-cat-varchar-tgt is now read. Three more batteries, one per question of the review: requests/lcat/int20
(lcat-q20-neg, -big, -max, -none, -two, -uns), requests/lcat/second (lcat-cat-two-int5, -two-int20,
-two-uns10, -two-zoned), requests/lcat/int3 (lcat-cat-int3-first, -second, -neg) and requests/lcat/pieces
(lcat-cat-eight-pieces, -nine-pieces).

New tests: src/rpg_lcat_concat_test.zig (the emitted text of each case, the declaration of the varying
target and of the int(20), and one changed-object test per rule of the readers: the first literal, a literal
piece, a digits piece, the numeric load, the varying tail, the pair and the static store). A changed-object test
asserts the site of the refusal (rpg_char_chain.read s05, through the trace seam), not only that the statement is
no longer recovered: a refusal at another site, or none, fails it, and so does a rule that is removed from the reader.

Trace sites moved

charConcat is gone. The statements of this family that are still refused now stop in rpg_char_chain.read
(s01 the first literal is not the unit, s10 a piece that is none of the two, s14 a tail that is neither
of the two, s15 a numeric load whose digits are not those of its strip), not in rpg_trim.charConcat s13; a triage token keyed on the old name is stale. pair s01 is a
new trace line on every statement that starts with two field loads (34 A; 113; 34 B).
rpg_static_init.quadword and singlebyte add the tokens s01 to s09 each. A program whose static initialization
stores an 8-byte or a 1-byte signed field no longer has the remaining_effects failure of that store, so its unresolved suffix after a proved *INLR write line goes (the 21 lcat-cat-* probes). The 33 public scout programs do not change for this reason.

Gate

== a. development emits (dev-hash)
p1 same
p2 same
p3 same
p4 same
p5 same
== b. L10-test direction
L10-test unchanged
== c. committed twins (emit-bytes-diff)
emit text differs:
diff -r -x emit.ndjson -x *.receipt.json /Users/someara/data/sourcerer-loop/runs/emitbytes.51ygoY/base-emit/245-P230B127BS.rpgle /Users/someara/data/sourcerer-loop/runs/emitbytes.51ygoY/head-emit/245-P230B127BS.rpgle
12c12,13
< dcl-s INVOKE_00264 int(3);
---
> dcl-s INVOKE_00263 int(20) inz(42);
> dcl-s INVOKE_00264 int(3) inz(7);
38d38
< 
41,42c41
< // SOURCERER-FAIL statement 40: unresolved suffix after a proved *INLR write
< // sourcerer: recovered source of P230B127BS.PGM; statements 2, emitted 2, failures 1
---
> // sourcerer: recovered source of P230B127BS.PGM; statements 2, emitted 2, failures 0
diff -r -x emit.ndjson -x *.receipt.json /Users/someara/data/sourcerer-loop/runs/emitbytes.51ygoY/base-emit/246-P0B1CF4F2S.rpgle /Users/someara/data/sourcerer-loop/runs/emitbytes.51ygoY/head-emit/246-P0B1CF4F2S.rpgle
12c12,13
< dcl-s INVOKE_00264 int(3);
---
> dcl-s INVOKE_00263 int(20) inz(42);
> dcl-s INVOKE_00264 int(3) inz(7);
38d38
forms projection differs:
202,205c202,205
< {"twin":"savf/SC-ILE/EXPERIMENTS/lc2-arg-expr-add/stripped","object":"P230B127BS","status":"done","statements":2,"emitted":2,"failures":1,"lines":43,"form_counts":{"copied_decimal_argument_v1":1,"cycle_body_routine_v1":1,"cycle_end_v1":1,"cycle_return_transfer_v1":1,"emit_call_program_v1":1,"emit_ctl_opt_v1":1,"emit_dcl_pr_v1":1,"emit_dcl_s_v1":13,"emit_end_pr_v1":1,"emit_free_v1":1,"emit_inlr_v1":1,"emit_pr_parameter_field_v1":1,"emit_pr_parameter_literal_v1":1,"emit_summary_v1":2,"fail_trailer_v1":1,"layout_blank_v1":19,"placeholder_decimal_positions_v1":3,"static_initialization_v1":1},"text_sha256":"f090aac9e4d75b09b35e743db914b5a256c4347219ff2515c76cefe36e784d4c"}
< {"twin":"savf/SC-ILE/EXPERIMENTS/lc2-arg-expr-mul/stripped","object":"P0B1CF4F2S","status":"done","statements":2,"emitted":2,"failures":1,"lines":42,"form_counts":{"copied_decimal_argument_v1":1,"cycle_body_routine_v1":1,"cycle_end_v1":1,"cycle_return_transfer_v1":1,"emit_call_program_v1":1,"emit_ctl_opt_v1":1,"emit_dcl_pr_v1":1,"emit_dcl_s_v1":11,"emit_end_pr_v1":1,"emit_free_v1":1,"emit_inlr_v1":1,"emit_pr_parameter_field_v1":1,"emit_pr_parameter_literal_v1":1,"emit_summary_v1":1,"fail_trailer_v1":1,"layout_blank_v1":21,"static_initialization_v1":1},"text_sha256":"84e3a4fff0c2b4175b6ffdf0b199ce754ab5eb96422696d280ccc2f4fa29788d"}
== d. captures (capture-emit-diff)
l10-bluemoonjp-scratchpad-ibmi-learning-1 base=[18,14,4] head=[18,15,3] CHANGED fail-lines=3
l10-briangarland-adventofcode2025-2 base=[65,42,23] head=[65,45,20] CHANGED fail-lines=20
l10-ossile-ossile-4 base=[37,23,14] head=[37,24,13] CHANGED fail-lines=13
lcat-cat-base base=[2,2,1] head=[2,2,0] CHANGED fail-lines=0
lcat-cat-first-one base=[2,1,2] head=[2,2,0] CHANGED fail-lines=0
... 42 lines omitted; the full table is in gate.log ...
cases to round-trip: 37
dev5/p3                                            base=match head=match
dev5/p4                                            base=match head=match
dev5/p5                                            base=match head=match
l10-scout2b/l10-danlong005-openrpg/l10-danlong005-openrpg-1 base=match head=match
l10-scout2b/l10-danlong005-openrpg/l10-danlong005-openrpg-4 base=match head=match
l10-scout2b/l10-danlong005-openrpg/l10-danlong005-openrpg-5 base=match head=match
lcat-concat/lcat-cat-base                          base=different head=different
lcat-concat/lcat-cat-first-one                     base=different head=different
lcat-concat/lcat-cat-int20                         base=different head=different
lcat-concat/lcat-cat-int5                          base=different head=different
lcat-concat/lcat-cat-lf                            base=different head=different
lcat-concat/lcat-cat-one-byte                      base=different head=different
lcat-concat/lcat-cat-short-tgt                     base=different head=different
lcat-concat/lcat-cat-trail-long                    base=different head=different
lcat-concat/lcat-cat-trail                         base=different head=different
lcat-concat/lcat-cat-two-char-lit                  base=different head=different
lcat-concat/lcat-cat-two-char                      base=different head=different
lcat-concat/lcat-cat-two-lits                      base=different head=different
lcat-concat/lcat-cat-two-packed                    base=different head=different
lcat-concat/lcat-cat-vary-base                     base=different head=different
lcat-concat/lcat-cat-vary-lf                       base=different head=different
lcat-concat/lcat-cat-vary-max20                    base=different head=different
lcat-concat/lcat-cat-vary-short                    base=failed head=failed
lcat-concat/lcat-cat-vary-tgt                      base=different head=different
lcat-concat/lcat-cat-vary-two-char                 base=different head=different
lcat-int20/lcat-q20-big                            base=different head=match
lcat-int20/lcat-q20-max                            base=different head=match
lcat-int20/lcat-q20-neg                            base=different head=match
lcat-int20/lcat-q20-none                           base=different head=match
lcat-int20/lcat-q20-two                            base=different head=match
lcat-int20/lcat-q20-uns                            base=different head=match
lcat-int3/lcat-cat-int3-first                      base=different head=match
lcat-int3/lcat-cat-int3-neg                        base=different head=match
lcat-int3/lcat-cat-int3-second                     base=different head=match
lcat-pieces/lcat-cat-eight-pieces                  base=different head=match
lcat-second/lcat-cat-two-int20                     base=different head=match
lcat-second/lcat-cat-two-int5                      base=different head=match
gate: PASS
# A character expression of literals and `%CHAR` of whole numbers recovers into a character field and into a varying-length field, and a static `int(20)` and `int(3)` keep their INZ `s = 'Total ' + %char(n)` recovered (lc2-cat-char-int). Anything more stopped at control_flow / operation_semantics on the first label of the `%CHAR` strip, with `%SUBST(A.1: 1: 10) = S` as the partial text: a literal after the `%CHAR`, a second `%CHAR`, a one-byte literal, and a varying-length target. `rpg_char_chain.zig` (new, PURE) reads the whole expression as one unit; `rpg_trim.charAssign` calls it where it called `charConcat`, which is gone. ## The unit The compiler builds the expression in one temporary buffer CB behind one running length T1, then one tail reads (CB, T1). After the strip of the digits of the first `%CHAR` (`rpg_trim.charStrip`, 59 operations): | piece | operations | |---|---| | FIRSTLIT, 27 | `27 P; 33 LIT; 5; 113; 5; 33 CB; 113; 5; 27 L; 35 T1; 27 L; 1; 35 T2; 76 4; 97; 27 L; 62; 113; 5; 27 N2; 1; 27 T2; 47; 35 T1; 76 4; 97; 3` | | APPLIT, 18 | `33 CB; 113; 33 LIT; 5; 27 T1; 1; 35 O; 62; 113; 5; 27 L; 1; 27 O; 47; 35 T1; 76 4; 97; 3` | | APPDIG, 20, after a numeric load and a second strip | `27 P2; 33 CB; 113; 5; 113; 5; 27 T1; 1; 35 O; 62; 113; 5; 27 N2; 1; 27 O; 47; 35 T1; 76 4; 97; 3` | | tail, fixed field | `rpg_trim.copyTail`, mode `joined`: `34 S 0; 113; 1; 35 Q; 33 CB; 113; ...` the copy of min(T1, W) bytes and the blank fill | | tail, varying field | `34 V 0; 113; 1; 35 Q; 27 2; 62; 113; 33 CB; 113; 27 MAX; 27 T1; 35 A; 35 C; 27 A; 27 C; 11; 17 L; 27 C; 35 A; 15 L 1; 27 A; 35 A; 27 A; 76 4; 97; 3; 27 Q; 27 A; 76 2; 42 V 0 U2` | The unit becomes the saved digits (`37 T`), `op_char_integer` with the first literal, then for each piece a `27 LIT` and `47`, or the numeric load, its `37 T`, `op_char_integer` and `47`, and at last `35 target`. No evaluator rule changed: `op_char_integer` with one operand, `27` of a constant and `47` existed. ## Facts decided The request is `requests/lcat/concat` (21 cases, one statement each on one template, captures in `captures/lcat-concat`); the probes are `lcat-cat-*` in SC-ILE/EXPERIMENTS. - **CB is the sum, exactly, in every probe.** The size of CB is the bytes of the literals plus D + 1 for each `%CHAR`, where D + 1 is the size of the strip's buffer: `cat-base` 6 + 11 = 17; `cat-trail` (+ `' ok'`) 20; `cat-trail-long` (+ 12 bytes) 29; `cat-one-byte` 18; `cat-first-one` 1 + 11 + 3 = 15; `cat-int5` 6 + 6 + 3 = 15; `cat-int20` 6 + 21 + 3 = 30; `cat-two-char` 6 + 11 + 3 + 11 = 31; `cat-two-packed` 6 + 11 + 3 + 6 = 26; `cat-two-char-lit` 35; `cat-two-lits` 24; `cat-vary-lf` 19. But the committed `lc2-cat-char-int` has the same expression with CB of 20 where the sum is 17: the buffer is one compiler temporary of the procedure, and the largest use of it (`a + b` in the same module, 20 bytes) sets its size. So the rule is a lower bound: the reader refuses a CB smaller than the sum, and never demands the sum. - **MAX is the declared length of the target.** `cat-vary-tgt` (`v` varchar(40): the symbol has 42 bytes, the tail loads the constant 40) against `cat-vary-max20` (varchar(20): 22 bytes, 20), which differ in the target only; `cat-vary-short` (varchar(8): 10 bytes, 8) holds a chain of up to 20 bytes in a target of 8, and the object is the same unit: the compiler does not fold min(T1, MAX) for a chain whose length it knows only at run time. The reader requires MAX = symbol length - 2. - **A one-byte literal is the same unit with L = 1.** `cat-trail` (`' ok'`: a text constant, L the integer 3) against `cat-one-byte` (`'.'`), `cat-lf` (`LF` = X'25') and `cat-first-one` (`'#'` as the first literal): the literal symbol is an integer constant of one byte (75, 37, 123; `word(4) == 1`) and L is the integer 1. The same 18 operations (APPLIT) and the same 27 (FIRSTLIT). `rpg_literals.constantText` writes it as `'.'` or `X'25'`; the compiler gave the one-byte constant for `LF` and for the quoted `'.'` alike. - **The scratch is not distinct, so the reader is positional.** `cat-two-char`: the scratch O of the APPLIT is the symbol that the strip before it used for N2 (294); the second strip stores the same value, buffer, counter and N2 (293, 295, 297, 294) and a new pointer (301); the scratch of the APPDIG is the T2 of the FIRSTLIT (299). `charConcat` required all temporaries pairwise distinct and refused this. Now each piece is read in order, and the rule is what the piece needs: O is not CB, not T1 (the state that lives on), and not the pointer or the digits length that the same piece reads after it stores O; the strip of a later digits piece does not write CB or T1 (value, buffer, pointer, counter, length). - **The second source.** The numeric load before the strip of a later `%CHAR` is one of three sequences, each measured in the place of the second `%CHAR` (`cat-two-char`: int(10), `cat-two-int20`: int(20), `cat-two-int5`: int(5), `cat-int3-second`: int(3), `cat-two-packed`: packed(5:0)), D the digits of the strip: | source | operations | |---|---| | int(10), int(20) | `29 N 0 T; 86 D; 61 D 0 0; 85 D 1` | | int(5), int(3) | `29 N 0 T; 75 4; 86 D; 61 D 0 0; 85 D 1`, D = 5 and 3 | | packed(D:0) | `29 N 0 T; 61 D 0 0; 85 D 1` | The reader takes these three and no other run of the operations, and requires the D of the `86`, the `61` and the `85` to be the digits of the strip, the decimal positions of the `61` to be 0, and the widening to be `75 4`. uns(10) loads with `88 D` where the int has `86 D` (`cat-two-uns10`) and zoned(5:0) with `84 5 1` (`cat-two-zoned`): measured, and refused. The earlier reader took any run of `29`, `61`, `75`, `85` and `86` with operands unchecked, which the captures did not prove. The order of the digits is the order of the source (`n` then `m`). - **The bound is 8 pieces.** `cat-eight-pieces` (`'Total ' + %char(n)` and eight literals) is read with all eight; `cat-nine-pieces` stops the reader at the eighth, where the ninth is not the tail, and the statement keeps its failure line (`read s14`). - **A one-byte text constant is not a one-byte literal.** The compiler wrote an integer constant of one byte in every probe. A text constant of one byte (a test's changed object) is refused at the first literal (`read s02`) and at a literal piece (`literalPiece s02`). - **Consecutive literals are two pieces** (`cat-two-lits`): the compiler does not join them. - **A target shorter than the chain** (`cat-short-tgt`, char(8)): `copyTail` keeps its minimum block. ## What recovers Eighteen of the 21 probes recover in line 16 that did not before; `cat-base` was recovered before, and the two `cat-vary-piece*` stay open (below). Counts of emitted and failure lines of the 21 stripped twins, base 0.1.521 against this change: | case | before | after | |---|---|---| | cat-trail, cat-trail-long, cat-one-byte, cat-lf, cat-first-one, cat-two-lits | 1 emitted, 2 failures | 2, 0 | | cat-two-char, cat-two-packed, cat-two-char-lit, cat-int5, cat-int20, cat-short-tgt | 1, 2 | 2, 0 | | cat-vary-tgt, cat-vary-max20, cat-vary-short, cat-vary-base, cat-vary-lf, cat-vary-two-char | 1, 2 | 2, 0 | | cat-base (the control) | 2, 1 | 2, 0 | | cat-vary-piece, cat-vary-piece-lit | 1, 2 | 1, 1 (refused) | The second failure of the base is statement 17, `unresolved suffix after a proved *INLR write`. The earlier text of this document called it the template's (`*inlr = *on;` and no procedure). It is not: it is the unread static initialization of `dcl-s q int(20) inz(5)`, the one declaration that the template shares, and it goes with the reader below. `cat-int20` was counted recovered with `q` declared `int(20)` and no INZ, so the recompiled program printed `Total 0 ok` where the original prints `Total 5 ok`; now the emit declares `inz(5)`. The varying target is declared `varchar(MAX)` by the existing proof of a varying assignment (the pair of the copy and the 2-byte length store), not by this reader; the tests check the declaration (`varchar(40)`, `(20)`, `(8)`) beside the assignment. ### `lc2-cat-varchar-tgt` `v = a + b` of two `char(10)` fields into a `varchar(40)`: the same buffer and the same store of V, but the two fields are copied to CB with no strip, and the length is the constant K = 20 that the compiler folded (`27 K; 35 A`, no minimum, no label). `rpg_char_chain.pair` reads it (`S267 = S255 + S256`). The one data point is K = LA + LB <= MAX, so the reader requires exactly that, the fields to be character fields that are not the target, CB at least K, and refuses the rest: a K that is not the sum, a sum above MAX (the folded minimum is not measured), a literal or a third piece. ## The static `int(20)` and `int(3)` `dcl-s q int(20) inz(5)` is `27 K; 37 q 0 T` in the static initialization: K a constant of the signed class and size 8 (the value in the 8 bytes at byte 12 of its symbol), T the type `1 0 0 0 0 0 0 8`, q a static field of kind 1 and length 8. No reader took it (`init.store` takes the 4-byte signed form and the unsigned forms of 1, 2, 4 and 8 bytes, `halfword` the 2-byte one), so the declaration lost its INZ and the failure of the unread store became the statement-17 failure above. `rpg_static_init.quadword` reads it, as an integer of an 8-byte static field. Measured with 5, 42 (`lcat-q20-two`), -5 (`lcat-q20-neg`), 4294967296 (`lcat-q20-big`) and 9223372036854775807 (`lcat-q20-max`); with no INZ (`lcat-q20-none`) the compiler stores 0, which is the default, so the declaration has none. `uns(20)` (`lcat-q20-uns`) was read before and is the control. `int(3)`, the 1-byte signed field, was the same gap (`lc2-arg-uns5` has `int(3) inz(7)` in the source, and the emit declared `int(3)`), and this lane's reader takes an int(3) as the first and as the second `%CHAR` (`lcat-cat-int3-first`, `-second`, `-neg`: the load is `29; 75 4; 86 3; 61 3 0 0; 85 3 1`, the widening of an int(5) with D = 3), so it is read as well: `rpg_static_init.singlebyte`, with a 1-byte signed constant and a 1-byte signed access type. The symbol decoder gives a 1-byte constant as an unsigned byte (-7 is 249); the reader makes it the signed byte, and the probe of `inz(-7)` (`lcat-cat-int3-neg`) is the test that it does. The program of these three probes declares `int(3) inz(7)` and `int(3) inz(-7)` and the emit now writes both. The reader reaches beyond the probes of this lane, because every program with such a field had the unread store as a failure: the emit gate (`emit-all.sh` of the committed twins against the 0.1.521 baseline) has worse 0, better 21 and 70 twins newly clean, and `twins.sh` 72 groups newly recovered with no regression. Five tests of `rpg_lr3b_test.zig` (`lr3-diff-ts-*`) had written the gap down ("the 8-byte `int(20)` start value of the target has no rule", the target named `V00255`); they now expect `A_00255 int(20)` and an emit with no failure line, which is what the source of the probe says (`dcl-s r int(20); r = %diff(a : b : *days)`). ## Measured: the 33 public scout programs `capture-emit-diff.sh` (stripped twins of `captures/l10-scout2` and `l10-scout2b`), base 0.1.521 against this change. 31 programs are the same; two change: | program | statements | emitted | failure lines | the statements that recover | |---|---|---|---|---| | bluemoonjp-1 | 18 | 14 to 15 | 4 to 3 | 26800: `msgText = 'lit' + %CHAR(i) + 'lit' + %CHAR(v) + 'lit'` into char(200), packed(5:0) as the second source | | advent-2 | 65 | 40 to 43 | 25 to 22 | 416, 479 and 488: `message = 'lit' + %CHAR(n) + '.' + X'25'` into varchar(100) (the source says `LF`) | `agree.sh` over the 33: same 159 to 160, refused 149 to 148, suspect 13 to 13, wrong parameters 0 to 0, declarations the same as the source 35 to 36 (advent-2 is not aligned to its source and is not scored; its three statements were read against the lines 431, 494 and 503 of the source by hand). `emit-lint.sh`: 0 findings. The other 31 emit byte for byte the same text as the base, so the reader of the static `int(20)` changes no public program beyond these. ## Not done, and why - **A varying field as a piece** (`cat-vary-piece`, `cat-vary-piece-lit`; advent-2 424, advent-3 412 and advent-4 412 hold it beside the call result `%CHAR(errno())`, which is the other lane's): the piece is `33 CB; 113; 34 V2 0; 1; 32 V2 0 U2; 76 8; 1; 76 4; 35 TA; 27 TA; 27 MAX; 12; 17 L; 27 115; 35 ST; 99 3 0 0 0; 15 L 1; ...`, the direct form of the length check, which signals error 115 by a store and a `99 3 0 0 0`. `rpg_varying_data.head` reads the pointer form (`29 P 0 PTR`, a call of an imported procedure). It does not fall out of this reader, so it refuses and the statement keeps its failure line. - **The call result as the source** (`%CHAR(errno())`), the `+=` consumer and the DSPLY tail: other lanes. - **uns(10) and zoned(5:0) as a second source**: measured, refused (above). - **A `61` with decimal positions** (a packed or zoned source with decimals) as a source: not measured, refused. - **The numeric load before the first strip** is not checked by the chain: the strip is where it starts, and the evaluator reads the load. The three sequences are measured in the first place for int(10), int(5), int(3) and int(20) only; uns(10), zoned and packed as the first source are not measured. - **`v = a + b` with a sum above MAX, with a literal, or with three fields**: one data point only. ## Probes `lcat-cat-base`, `-trail`, `-trail-long`, `-one-byte`, `-lf`, `-first-one`, `-two-char`, `-two-packed`, `-two-char-lit`, `-two-lits`, `-int5`, `-int20`, `-short-tgt`, `-vary-tgt`, `-vary-max20`, `-vary-short`, `-vary-base`, `-vary-lf`, `-vary-two-char`, `-vary-piece`, `-vary-piece-lit`; the committed `lc2-cat-varchar-tgt` is now read. Three more batteries, one per question of the review: `requests/lcat/int20` (`lcat-q20-neg`, `-big`, `-max`, `-none`, `-two`, `-uns`), `requests/lcat/second` (`lcat-cat-two-int5`, `-two-int20`, `-two-uns10`, `-two-zoned`), `requests/lcat/int3` (`lcat-cat-int3-first`, `-second`, `-neg`) and `requests/lcat/pieces` (`lcat-cat-eight-pieces`, `-nine-pieces`). New tests: `src/rpg_lcat_concat_test.zig` (the emitted text of each case, the declaration of the varying target and of the `int(20)`, and one changed-object test per rule of the readers: the first literal, a literal piece, a digits piece, the numeric load, the varying tail, the pair and the static store). A changed-object test asserts the site of the refusal (`rpg_char_chain.read s05`, through the trace seam), not only that the statement is no longer recovered: a refusal at another site, or none, fails it, and so does a rule that is removed from the reader. ## Trace sites moved `charConcat` is gone. The statements of this family that are still refused now stop in `rpg_char_chain.read` (`s01` the first literal is not the unit, `s10` a piece that is none of the two, `s14` a tail that is neither of the two, `s15` a numeric load whose digits are not those of its strip), not in `rpg_trim.charConcat s13`; a triage token keyed on the old name is stale. `pair s01` is a new trace line on every statement that starts with two field loads (`34 A; 113; 34 B`). `rpg_static_init.quadword` and `singlebyte` add the tokens `s01` to `s09` each. A program whose static initialization stores an 8-byte or a 1-byte signed field no longer has the `remaining_effects` failure of that store, so its `unresolved suffix after a proved *INLR write` line goes (the 21 `lcat-cat-*` probes). The 33 public scout programs do not change for this reason. ## Gate ``` == a. development emits (dev-hash) p1 same p2 same p3 same p4 same p5 same == b. L10-test direction L10-test unchanged == c. committed twins (emit-bytes-diff) emit text differs: diff -r -x emit.ndjson -x *.receipt.json /Users/someara/data/sourcerer-loop/runs/emitbytes.51ygoY/base-emit/245-P230B127BS.rpgle /Users/someara/data/sourcerer-loop/runs/emitbytes.51ygoY/head-emit/245-P230B127BS.rpgle 12c12,13 < dcl-s INVOKE_00264 int(3); --- > dcl-s INVOKE_00263 int(20) inz(42); > dcl-s INVOKE_00264 int(3) inz(7); 38d38 < 41,42c41 < // SOURCERER-FAIL statement 40: unresolved suffix after a proved *INLR write < // sourcerer: recovered source of P230B127BS.PGM; statements 2, emitted 2, failures 1 --- > // sourcerer: recovered source of P230B127BS.PGM; statements 2, emitted 2, failures 0 diff -r -x emit.ndjson -x *.receipt.json /Users/someara/data/sourcerer-loop/runs/emitbytes.51ygoY/base-emit/246-P0B1CF4F2S.rpgle /Users/someara/data/sourcerer-loop/runs/emitbytes.51ygoY/head-emit/246-P0B1CF4F2S.rpgle 12c12,13 < dcl-s INVOKE_00264 int(3); --- > dcl-s INVOKE_00263 int(20) inz(42); > dcl-s INVOKE_00264 int(3) inz(7); 38d38 forms projection differs: 202,205c202,205 < {"twin":"savf/SC-ILE/EXPERIMENTS/lc2-arg-expr-add/stripped","object":"P230B127BS","status":"done","statements":2,"emitted":2,"failures":1,"lines":43,"form_counts":{"copied_decimal_argument_v1":1,"cycle_body_routine_v1":1,"cycle_end_v1":1,"cycle_return_transfer_v1":1,"emit_call_program_v1":1,"emit_ctl_opt_v1":1,"emit_dcl_pr_v1":1,"emit_dcl_s_v1":13,"emit_end_pr_v1":1,"emit_free_v1":1,"emit_inlr_v1":1,"emit_pr_parameter_field_v1":1,"emit_pr_parameter_literal_v1":1,"emit_summary_v1":2,"fail_trailer_v1":1,"layout_blank_v1":19,"placeholder_decimal_positions_v1":3,"static_initialization_v1":1},"text_sha256":"f090aac9e4d75b09b35e743db914b5a256c4347219ff2515c76cefe36e784d4c"} < {"twin":"savf/SC-ILE/EXPERIMENTS/lc2-arg-expr-mul/stripped","object":"P0B1CF4F2S","status":"done","statements":2,"emitted":2,"failures":1,"lines":42,"form_counts":{"copied_decimal_argument_v1":1,"cycle_body_routine_v1":1,"cycle_end_v1":1,"cycle_return_transfer_v1":1,"emit_call_program_v1":1,"emit_ctl_opt_v1":1,"emit_dcl_pr_v1":1,"emit_dcl_s_v1":11,"emit_end_pr_v1":1,"emit_free_v1":1,"emit_inlr_v1":1,"emit_pr_parameter_field_v1":1,"emit_pr_parameter_literal_v1":1,"emit_summary_v1":1,"fail_trailer_v1":1,"layout_blank_v1":21,"static_initialization_v1":1},"text_sha256":"84e3a4fff0c2b4175b6ffdf0b199ce754ab5eb96422696d280ccc2f4fa29788d"} == d. captures (capture-emit-diff) l10-bluemoonjp-scratchpad-ibmi-learning-1 base=[18,14,4] head=[18,15,3] CHANGED fail-lines=3 l10-briangarland-adventofcode2025-2 base=[65,42,23] head=[65,45,20] CHANGED fail-lines=20 l10-ossile-ossile-4 base=[37,23,14] head=[37,24,13] CHANGED fail-lines=13 lcat-cat-base base=[2,2,1] head=[2,2,0] CHANGED fail-lines=0 lcat-cat-first-one base=[2,1,2] head=[2,2,0] CHANGED fail-lines=0 ... 42 lines omitted; the full table is in gate.log ... cases to round-trip: 37 dev5/p3 base=match head=match dev5/p4 base=match head=match dev5/p5 base=match head=match l10-scout2b/l10-danlong005-openrpg/l10-danlong005-openrpg-1 base=match head=match l10-scout2b/l10-danlong005-openrpg/l10-danlong005-openrpg-4 base=match head=match l10-scout2b/l10-danlong005-openrpg/l10-danlong005-openrpg-5 base=match head=match lcat-concat/lcat-cat-base base=different head=different lcat-concat/lcat-cat-first-one base=different head=different lcat-concat/lcat-cat-int20 base=different head=different lcat-concat/lcat-cat-int5 base=different head=different lcat-concat/lcat-cat-lf base=different head=different lcat-concat/lcat-cat-one-byte base=different head=different lcat-concat/lcat-cat-short-tgt base=different head=different lcat-concat/lcat-cat-trail-long base=different head=different lcat-concat/lcat-cat-trail base=different head=different lcat-concat/lcat-cat-two-char-lit base=different head=different lcat-concat/lcat-cat-two-char base=different head=different lcat-concat/lcat-cat-two-lits base=different head=different lcat-concat/lcat-cat-two-packed base=different head=different lcat-concat/lcat-cat-vary-base base=different head=different lcat-concat/lcat-cat-vary-lf base=different head=different lcat-concat/lcat-cat-vary-max20 base=different head=different lcat-concat/lcat-cat-vary-short base=failed head=failed lcat-concat/lcat-cat-vary-tgt base=different head=different lcat-concat/lcat-cat-vary-two-char base=different head=different lcat-int20/lcat-q20-big base=different head=match lcat-int20/lcat-q20-max base=different head=match lcat-int20/lcat-q20-neg base=different head=match lcat-int20/lcat-q20-none base=different head=match lcat-int20/lcat-q20-two base=different head=match lcat-int20/lcat-q20-uns base=different head=match lcat-int3/lcat-cat-int3-first base=different head=match lcat-int3/lcat-cat-int3-neg base=different head=match lcat-int3/lcat-cat-int3-second base=different head=match lcat-pieces/lcat-cat-eight-pieces base=different head=match lcat-second/lcat-cat-two-int20 base=different head=match lcat-second/lcat-cat-two-int5 base=different head=match gate: PASS ```
rpg_char_chain.zig reads the unit that the compiler builds for 'lit' + %CHAR(a) + 'lit' + %CHAR(b) ...:
one buffer behind one running length, the first literal, then literal pieces (a one-byte literal is an
integer constant with L = 1) and the digits of further %CHAR strips, then the copy to a character field
(copyTail, joined) or the new tail to a varying-length field (min(T1, MAX), MAX the declared length).
The scratch of a piece reuses the temporaries of the strip before it, so each piece is read in order and
checked against what it reads after it stores the scratch, not for pairwise distinct temporaries.

Also reads `v = a + b` of two character fields into a varying-length field (lc2-cat-varchar-tgt) when the
folded length is the sum of the two lengths.

A varying field as a piece, the call result and the += consumer stay refused. 21 probes (lcat-cat-*) and
rpg_lcat_concat_test.zig, with one changed-object test per rule.
Review of the character chain found that cat-int20 was counted recovered while q was declared int(20)
with no INZ (the recompiled program prints 'Total 0 ok' where the original prints 'Total 5 ok'). The
store of an 8-byte signed static field, and of a 1-byte one, was read by no reader: the unread store
was the statement-17 failure that every cat-* probe carried. rpg_static_init.quadword and singlebyte
read `27 K; 37 q 0 T` for the two widths (a one-byte constant is the signed byte, -7 is 0xF9), measured
with 5, 42, -5, 4294967296, 9223372036854775807, no INZ, 7 and -7.

numericLoad takes the three measured sequences of the numeric load before a later strip (int(10) and
int(20), int(5) and int(3), packed) and checks their digits against the strip, where it took any run
of 29, 61, 75, 85 and 86. uns(10) and zoned(5:0) as a second source are measured and refused.

Tests assert the refusal site of each rule through the trace seam (the pair test passed with its rule
removed): the first literal, a one-byte text constant, the numeric load, the bound of 8 pieces and the
two static stores. 15 new probes, in four batteries (lcat int20, second, int3, pieces).
The five lr3-diff-ts-* tests allowed failures because the 8-byte int(20) start value of the target had
no rule (the target was named V00255). The static store of an int(20) is read now, so the emit has no
failure line and names the target A_00255, as the probe source declares it.
Regenerate the operations legend
Some checks reported errors
binjovi/ci A newer source or plan superseded this Build
3995db0da6
sean force-pushed lcat from 3995db0da6
Some checks reported errors
binjovi/ci A newer source or plan superseded this Build
to 3116bf8893
All checks were successful
binjovi/ci Binjovi completed the frozen plan
2026-11-15 17:09:52 +00:00
Compare
binjovi-bot manually merged commit 3116bf8893 into trunk 2026-11-15 17:27:36 +00:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
sean/sourcerer!806
No description provided.