A folded %ELEM(array) is written as %ELEM(array), in a store and in a FOR limit over compile-time data #803

Manually merged
binjovi-bot merged 2 commits from ldev3 into trunk 2026-11-14 17:23:16 +00:00
Owner

A folded %ELEM(array) is written as %ELEM(array), in a store and in a FOR limit over compile-time data

Two development programs whose round trips compile with the same instructions
and the same control flow as the original, and differ in creation_instructions,
symbols, literals, constant_area and initialization only, have one cause: the
emit wrote the number for size = %ELEM(list).

What the dumps show (the symbol dumps of the original and the rebuild, a
statement-by-statement operand comparison of statements.raw.txt, and
sourcerer compare on the two stored directories):

  • The declarations are the same. Every symbol before the constants (the
    declared fields, the compile-time array, the arrays) is byte-identical in the
    two objects, and so is the type table. The order of the declarations,
    dim(*ctdata) against dim(n) ctdata, packed(5:0) against packed(5), an
    unused field and an initial value are not a cause.
  • The original has one more constant symbol: a packed constant of the count of
    the array, the first decimal of the literal pool, that no operation loads.
    That one symbol makes every later symbol number one lower in the rebuild
    (each differing creation instruction differs in one operand, by exactly one),
    the symbol rows that name symbols differ by one in those words, and the
    literal offsets after it by the 3 bytes of the constant. instructions,
    control_flow, imports, exports, relocations, types and attributes
    of sourcerer compare are equal; the remaining differences of the symbol
    rows (two 32-byte character constants that carry the identity of the source
    member) differ in a matching round trip as well.
  • size = %ELEM(list) is folded by the compiler to 27 K; 88 n; 61 5 0 0; 37 F with K an unsigned 8-byte constant of the count: the operations of
    size = 4. Our emit wrote the number, which leaves no packed head constant.

The probes show the shape of the head constant (SC-ILE/EXPERIMENTS
lctd2-elem-star, lctd-loop, lr1-lf-up-*; the batteries lctd-ctdata and
lctd-ctdata-index of the loop captures, among them lctd2-perrcd, lctd2-whole-elem
and lctd2-two-arrays): one packed constant of the count for each %ELEM,
between the 1-character constants 'R', 'F', 'I' and the x'F0' of the character
literal '0'. A literal size = 4 or dim(4) puts its constant in the sorted
run, not there. A FOR whose limit is %ELEM loads the head constant itself
(the existing elemLimit); a store of %ELEM loads the unsigned constant and
leaves the head unloaded.

An unloaded decimal ahead of the x'F0' is not always %ELEM: %SIZE and
%LEN leave one (ldsv-asyncprom, h-bifs-char), a fraction literal of
inz(.7) sorts there (d7-lits), and so does a negative literal (c8x-*). So
rpg_elem_count.zig (new) reads the unsigned 8-byte constant as the %ELEM
of an array only when all of these hold:

  • the packed constants of that value ahead of the x'F0' that no operation
    loads are as many as the operations that name the constant, and the module
    holds one such constant (a literal of the same value beside the builtin
    makes the loads more than the heads: the number stays);
  • exactly one array has that count, proved by the unit of a variable index
    (rpg_source_ctdata.unitAt: 34 A 0; 29 N 0 T; ...; 27 K; 138; 78 4; 65 W,
    K + 1 elements);
  • no data symbol that the source declares (the run after the status area) is
    that many bytes or digits long: the count could be its %SIZE or %LEN.

Known exposure: an unloaded head left by another builtin or literal, an
unrelated literal store of its value, and one array of that count together
pass the three facts. No committed twin does.

rpg_eval.load writes %ELEM(array) for such a constant at every load of it
(a store and an expression use the same operation); the constant stays a
number everywhere else. rpg_shape.elemLimit gets the same array finder as a
second way: the limit of a decimal FOR that is the head constant itself named
no array when only a variable index reaches the array (lctd-loop wrote
TO 3); it writes TO %ELEM(array) when the unit proves one array of that
count and no character buffer of the program is that many bytes long (a
numeric field is not counted there: nobody loops to the size of an index).
scale_mod.runStart is public for the new module. When a compile-time array
is refused a declaration (a data record that is blank or starts like a mark of
the source: lctd2-blank-mid, lctd2-dollar, lctd2-equal, lctd2-slash-first,
lctd2-star-first), the FOR header that names it fails too, with the failure
line of its body: those five emit 2 statements and 3 failure lines where they
emitted 4 and 1.

Tests: rpg_elem_count_test.zig (a store of the folded count of the adopted
probe lctd2-elem-star, which has the shape size = %ELEM(list),
FOR i = 1 TO size; the FOR limit of lctd-loop; the refusals on lctd-loop (a
loaded head), d7-lits, h-bifs-char and ldsv-asyncprom; the table of the
evidence). The new src/testdata directory is a matrix row: it is expected to
match, the coordinator classes it if it does not.

Before and after, v0.1.517 against this build: the committed twins (889
programs) change in two emits, both read against their source
(TO 3 to TO %ELEM(A_00254) in lctd-loop, = 4 to = %ELEM(A_00254) in
lctd2-elem-star); counts of statements, emitted and failures are equal. The
census of the 649 committed stripped twins (774 unsigned 8-byte constants)
finds one store, in lctd2-elem-star. Over the batteries lctd-ctdata,
lctd-ctdata-index, lsrt-sorta, lsrt-sorta2 and the Async captures (77 cases)
15 emits change: nine write TO %ELEM(array) for a literal limit, lctd2-elem-star
writes the store, and the five refused-array cases above fail two statements more.
agree.sh over them: same 1681 to 1686, suspect 97 to 82, parameters wrong
14 to 14. The predicted object of the new text has the head constant where the
original has it; a round trip is the open check.

Not in this change: the helper procedure _QRNI_SORT_A_<name> of SORTA is
named for the array of the source, and the rebuild names it after our array.
No component that sourcerer compare reports names it, and no matched round
trip has a SORTA, so whether a rebuild with another name matches is open.
Two %ELEM of equal counts of different arrays (lctd2-two-arrays) stay literal:
no array is unique.

Gate (host round trips, base ~/bin/sourcerer v0.1.517)

lctd2-big-40 base=[5,5,0] head=[5,5,0] CHANGED fail-lines=0
lctd2-blank-mid base=[5,4,1] head=[5,2,3] CHANGED fail-lines=3
lctd2-chars base=[5,5,0] head=[5,5,0] CHANGED fail-lines=0
lctd2-dollar base=[5,4,1] head=[5,2,3] CHANGED fail-lines=3
lctd2-elem-star base=[6,6,0] head=[6,6,0] CHANGED fail-lines=0
lctd2-equal base=[5,4,1] head=[5,2,3] CHANGED fail-lines=3
lctd2-inz-equal base=[5,4,1] head=[5,4,1] CHANGED fail-lines=1
lctd2-inz-scalar base=[6,6,0] head=[6,6,0] CHANGED fail-lines=0
lctd2-perrcd-one base=[5,5,0] head=[5,5,0] CHANGED fail-lines=0
lctd2-perrcd base=[5,5,0] head=[5,5,0] CHANGED fail-lines=0
lctd2-slash-first base=[5,4,1] head=[5,2,3] CHANGED fail-lines=3
lctd2-star-first base=[5,4,1] head=[5,2,3] CHANGED fail-lines=3
lctd2-store base=[6,4,2] head=[6,4,2] CHANGED fail-lines=2
lctd2-width-100 base=[5,5,0] head=[5,5,0] CHANGED fail-lines=0
lctd-loop base=[5,5,0] head=[5,5,0] CHANGED fail-lines=0
capture-emit-diff: 48 cases, 17 changed, 15 with no failure line in the head emit
allowed: lctd2-blank-mid (the ctdata array is refused a declaration; the FOR header that now names it with %ELEM fails with it (the old TO N named no declared field))
allowed: lctd2-dollar (the ctdata array is refused a declaration; the FOR header that now names it with %ELEM fails with it (the old TO N named no declared field))
allowed: lctd2-equal (the ctdata array is refused a declaration; the FOR header that now names it with %ELEM fails with it (the old TO N named no declared field))
allowed: lctd2-slash-first (the ctdata array is refused a declaration; the FOR header that now names it with %ELEM fails with it (the old TO N named no declared field))
allowed: lctd2-star-first (the ctdata array is refused a declaration; the FOR header that now names it with %ELEM fails with it (the old TO N named no declared field))
== e. source agreement (agree)
suspect        base=100 head=83
conflict       base=0 head=0
params_wrong   base=15 head=15
decl_missing   base=86 head=81
decl_extra     base=25 head=25
type_unmarked  base=62 head=62
== f. host round trips (both binaries)
cases to round-trip: 15
dev5/p3                                            base=different head=different
dev5/p4                                            base=different head=different
lctd-ctdata-index/lctd2-big-40                     base=different head=match
lctd-ctdata-index/lctd2-chars                      base=different head=match
lctd-ctdata-index/lctd2-compare                    base=failed head=failed
lctd-ctdata-index/lctd2-elem-star                  base=different head=match
lctd-ctdata-index/lctd2-inz-scalar                 base=different head=match
lctd-ctdata-index/lctd2-perrcd-one                 base=different head=different
lctd-ctdata-index/lctd2-perrcd                     base=different head=different
lctd-ctdata-index/lctd2-two-arrays                 base=different head=different
lctd-ctdata-index/lctd2-whole-elem                 base=failed head=failed
lctd-ctdata-index/lctd2-width-100                  base=different head=different
lctd-ctdata/lctd-dim-star-elem                     base=different head=different
lctd-ctdata/lctd-loop                              base=different head=match
gate: PASS
# A folded `%ELEM(array)` is written as `%ELEM(array)`, in a store and in a FOR limit over compile-time data Two development programs whose round trips compile with the same instructions and the same control flow as the original, and differ in creation_instructions, symbols, literals, constant_area and initialization only, have one cause: the emit wrote the number for `size = %ELEM(list)`. What the dumps show (the symbol dumps of the original and the rebuild, a statement-by-statement operand comparison of `statements.raw.txt`, and `sourcerer compare` on the two stored directories): - The declarations are the same. Every symbol before the constants (the declared fields, the compile-time array, the arrays) is byte-identical in the two objects, and so is the type table. The order of the declarations, `dim(*ctdata)` against `dim(n) ctdata`, `packed(5:0)` against `packed(5)`, an unused field and an initial value are not a cause. - The original has one more constant symbol: a packed constant of the count of the array, the first decimal of the literal pool, that no operation loads. That one symbol makes every later symbol number one lower in the rebuild (each differing creation instruction differs in one operand, by exactly one), the symbol rows that name symbols differ by one in those words, and the literal offsets after it by the 3 bytes of the constant. `instructions`, `control_flow`, `imports`, `exports`, `relocations`, `types` and `attributes` of `sourcerer compare` are equal; the remaining differences of the symbol rows (two 32-byte character constants that carry the identity of the source member) differ in a matching round trip as well. - `size = %ELEM(list)` is folded by the compiler to `27 K; 88 n; 61 5 0 0; 37 F` with K an unsigned 8-byte constant of the count: the operations of `size = 4`. Our emit wrote the number, which leaves no packed head constant. The probes show the shape of the head constant (SC-ILE/EXPERIMENTS lctd2-elem-star, lctd-loop, lr1-lf-up-*; the batteries lctd-ctdata and lctd-ctdata-index of the loop captures, among them lctd2-perrcd, lctd2-whole-elem and lctd2-two-arrays): one packed constant of the count for each `%ELEM`, between the 1-character constants 'R', 'F', 'I' and the x'F0' of the character literal '0'. A literal `size = 4` or `dim(4)` puts its constant in the sorted run, not there. A FOR whose limit is `%ELEM` loads the head constant itself (the existing `elemLimit`); a store of `%ELEM` loads the unsigned constant and leaves the head unloaded. An unloaded decimal ahead of the x'F0' is not always `%ELEM`: `%SIZE` and `%LEN` leave one (ldsv-asyncprom, h-bifs-char), a fraction literal of `inz(.7)` sorts there (d7-lits), and so does a negative literal (c8x-*). So `rpg_elem_count.zig` (new) reads the unsigned 8-byte constant as the `%ELEM` of an array only when all of these hold: - the packed constants of that value ahead of the x'F0' that no operation loads are as many as the operations that name the constant, and the module holds one such constant (a literal of the same value beside the builtin makes the loads more than the heads: the number stays); - exactly one array has that count, proved by the unit of a variable index (`rpg_source_ctdata.unitAt`: `34 A 0; 29 N 0 T; ...; 27 K; 138; 78 4; 65 W`, K + 1 elements); - no data symbol that the source declares (the run after the status area) is that many bytes or digits long: the count could be its `%SIZE` or `%LEN`. Known exposure: an unloaded head left by another builtin or literal, an unrelated literal store of its value, and one array of that count together pass the three facts. No committed twin does. `rpg_eval.load` writes `%ELEM(array)` for such a constant at every load of it (a store and an expression use the same operation); the constant stays a number everywhere else. `rpg_shape.elemLimit` gets the same array finder as a second way: the limit of a decimal FOR that is the head constant itself named no array when only a variable index reaches the array (lctd-loop wrote `TO 3`); it writes `TO %ELEM(array)` when the unit proves one array of that count and no character buffer of the program is that many bytes long (a numeric field is not counted there: nobody loops to the size of an index). `scale_mod.runStart` is public for the new module. When a compile-time array is refused a declaration (a data record that is blank or starts like a mark of the source: lctd2-blank-mid, lctd2-dollar, lctd2-equal, lctd2-slash-first, lctd2-star-first), the FOR header that names it fails too, with the failure line of its body: those five emit 2 statements and 3 failure lines where they emitted 4 and 1. Tests: `rpg_elem_count_test.zig` (a store of the folded count of the adopted probe lctd2-elem-star, which has the shape `size = %ELEM(list)`, `FOR i = 1 TO size`; the FOR limit of lctd-loop; the refusals on lctd-loop (a loaded head), d7-lits, h-bifs-char and ldsv-asyncprom; the table of the evidence). The new `src/testdata` directory is a matrix row: it is expected to match, the coordinator classes it if it does not. Before and after, v0.1.517 against this build: the committed twins (889 programs) change in two emits, both read against their source (`TO 3` to `TO %ELEM(A_00254)` in lctd-loop, `= 4` to `= %ELEM(A_00254)` in lctd2-elem-star); counts of statements, emitted and failures are equal. The census of the 649 committed stripped twins (774 unsigned 8-byte constants) finds one store, in lctd2-elem-star. Over the batteries lctd-ctdata, lctd-ctdata-index, lsrt-sorta, lsrt-sorta2 and the Async captures (77 cases) 15 emits change: nine write `TO %ELEM(array)` for a literal limit, lctd2-elem-star writes the store, and the five refused-array cases above fail two statements more. `agree.sh` over them: same 1681 to 1686, suspect 97 to 82, parameters wrong 14 to 14. The predicted object of the new text has the head constant where the original has it; a round trip is the open check. Not in this change: the helper procedure `_QRNI_SORT_A_<name>` of `SORTA` is named for the array of the source, and the rebuild names it after our array. No component that `sourcerer compare` reports names it, and no matched round trip has a SORTA, so whether a rebuild with another name matches is open. Two `%ELEM` of equal counts of different arrays (lctd2-two-arrays) stay literal: no array is unique. ## Gate (host round trips, base ~/bin/sourcerer v0.1.517) ``` lctd2-big-40 base=[5,5,0] head=[5,5,0] CHANGED fail-lines=0 lctd2-blank-mid base=[5,4,1] head=[5,2,3] CHANGED fail-lines=3 lctd2-chars base=[5,5,0] head=[5,5,0] CHANGED fail-lines=0 lctd2-dollar base=[5,4,1] head=[5,2,3] CHANGED fail-lines=3 lctd2-elem-star base=[6,6,0] head=[6,6,0] CHANGED fail-lines=0 lctd2-equal base=[5,4,1] head=[5,2,3] CHANGED fail-lines=3 lctd2-inz-equal base=[5,4,1] head=[5,4,1] CHANGED fail-lines=1 lctd2-inz-scalar base=[6,6,0] head=[6,6,0] CHANGED fail-lines=0 lctd2-perrcd-one base=[5,5,0] head=[5,5,0] CHANGED fail-lines=0 lctd2-perrcd base=[5,5,0] head=[5,5,0] CHANGED fail-lines=0 lctd2-slash-first base=[5,4,1] head=[5,2,3] CHANGED fail-lines=3 lctd2-star-first base=[5,4,1] head=[5,2,3] CHANGED fail-lines=3 lctd2-store base=[6,4,2] head=[6,4,2] CHANGED fail-lines=2 lctd2-width-100 base=[5,5,0] head=[5,5,0] CHANGED fail-lines=0 lctd-loop base=[5,5,0] head=[5,5,0] CHANGED fail-lines=0 capture-emit-diff: 48 cases, 17 changed, 15 with no failure line in the head emit allowed: lctd2-blank-mid (the ctdata array is refused a declaration; the FOR header that now names it with %ELEM fails with it (the old TO N named no declared field)) allowed: lctd2-dollar (the ctdata array is refused a declaration; the FOR header that now names it with %ELEM fails with it (the old TO N named no declared field)) allowed: lctd2-equal (the ctdata array is refused a declaration; the FOR header that now names it with %ELEM fails with it (the old TO N named no declared field)) allowed: lctd2-slash-first (the ctdata array is refused a declaration; the FOR header that now names it with %ELEM fails with it (the old TO N named no declared field)) allowed: lctd2-star-first (the ctdata array is refused a declaration; the FOR header that now names it with %ELEM fails with it (the old TO N named no declared field)) == e. source agreement (agree) suspect base=100 head=83 conflict base=0 head=0 params_wrong base=15 head=15 decl_missing base=86 head=81 decl_extra base=25 head=25 type_unmarked base=62 head=62 == f. host round trips (both binaries) cases to round-trip: 15 dev5/p3 base=different head=different dev5/p4 base=different head=different lctd-ctdata-index/lctd2-big-40 base=different head=match lctd-ctdata-index/lctd2-chars base=different head=match lctd-ctdata-index/lctd2-compare base=failed head=failed lctd-ctdata-index/lctd2-elem-star base=different head=match lctd-ctdata-index/lctd2-inz-scalar base=different head=match lctd-ctdata-index/lctd2-perrcd-one base=different head=different lctd-ctdata-index/lctd2-perrcd base=different head=different lctd-ctdata-index/lctd2-two-arrays base=different head=different lctd-ctdata-index/lctd2-whole-elem base=failed head=failed lctd-ctdata-index/lctd2-width-100 base=different head=different lctd-ctdata/lctd-dim-star-elem base=different head=different lctd-ctdata/lctd-loop base=different head=match gate: PASS ```
The compiler folds %ELEM(array) to the count (an unsigned 8-byte constant, the
same operations as a literal) and leaves one packed constant of the count
ahead of the character literals that no operation loads. An emit that wrote
the number had no such constant, so the pool, the symbol numbers, the literal
offsets and the initialization differed from the original.

rpg_elem_count.zig reads the folded count as the %ELEM of one array when the
unloaded heads of that value are as many as the loads of the constant, one
array has that count (the unit of a variable index) and no declared symbol is
that many bytes or digits long. rpg_eval.load writes it; rpg_shape.elemLimit
uses the same array finder for a FOR over compile-time data.
Regenerate the operations legend
Some checks reported errors
binjovi/ci A newer source or plan superseded this Build
1f605a8af4
sean force-pushed ldev3 from 1f605a8af4
Some checks reported errors
binjovi/ci A newer source or plan superseded this Build
to 7a4651ec1c
All checks were successful
binjovi/ci Binjovi completed the frozen plan
2026-11-14 17:09:39 +00:00
Compare
binjovi-bot manually merged commit 7a4651ec1c into trunk 2026-11-14 17:23:16 +00:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
sean/sourcerer!803
No description provided.