The program status data structure's status code after a call (touches rpg_source.zig, regenerates docs/OPERATIONS.md) #780

Manually merged
binjovi-bot merged 8 commits from lext3 into trunk 2026-11-10 18:41:36 +00:00
Owner

The program status data structure's status code after a call (touches rpg_source.zig, regenerates docs/OPERATIONS.md)

Lane Lext, round three. Entry condition: the capture lext-psds-call
reproduces the third post-call form of the L10 development programs, and it is a
PSDS family, not a THREAD one. With the lext2 build the tracer shows the stop at
rpg_frames.rewrite s02 (the status unit) for lext-psds-void-c and for the
control lext-psds-void-p, which has no THREAD.

What the captures show

A module with dcl-ds psds psds has after each call of a procedure of the
module, in front of the status unit of the module without it:

  • 29 P 10 T; 37 P 15 T: the status code (bytes 11 to 15 of the 429-byte
    static area P) rolls into the previous status (16 to 20). T is a 5-byte
    character type.
  • 27 T'; 37 P 10 T after each store of the call status R: the status code is
    set, to 00001 after the store of 1 (the callee failed) and to 00000
    after the store of 0.

With THREAD the unit is roll; 27 0; 35 R; set 00000 (lext-psds-void-c, -arg-c,
-read-c, -ret-c). Without THREAD it is
roll; 29 F 96 Q; 32 B 0 I2; 27 1; 7; 16 L; 27 1; 35 R; set 00001; 18 E; 15 L; 27 0; 35 R; set 00000; 15 E (lext-psds-void-p): the unit of rpg_frames.status
with the two sets (the constants of the two texts are F0F0F0F0F1 and
F0F0F0F0F0).

The rule

rpg_frames.psdsStatus, tried first by status when psdsRoll finds the roll.
It takes the roll, then either the thread unit (threadStatus) and the set, or
the plain unit (statusUnit, split out of status with no change to its
checks) with the two sets taken out at the places above. The area must be the
429-byte static symbol, the type 5 bytes, the offsets 10 and 15 and the texts
those two; a set at another offset leaves the statement open (a test changes it).
The roll and the sets are bookkeeping that the program status data structure
of the source reads: the statement text ignores them.

One failure line and one emit rule. ctlOpt adds
SOURCERER-FAIL PSDS: the program status data structure has no emit rule when
rpg_frames.programStatusRolled finds the roll straight after a call (the cycle
and SQL write the same bytes elsewhere: a first version that looked at every
roll made 24 public twins worse). And ctlOpt writes thread(*serialize) into
the emitted ctl-opt of a serialized module, which closes the gap the
coordinator found after round two: lext-serialize-c recovers all its
statements and its text had no keyword, so it could not rebuild the object. A
first version wrote a failure line instead; a failure line raises the failure
count of a committed twin, which the emit gate refuses, and the keyword is the
emit rule. lext-serialize-c keeps its two failures (6 emitted, 2 failures, as
v0.1.496) and gains the keyword. rpg_source.zig is touched for these two
lines only.

The serialized module is rpg_source_decl.serialized: the module imports the
mutex routine of the run time (_QRNX_LOCK_MTX) and has no static symbol with
the thread-local bit. The import alone is not enough. It also fired on
l10-async-asyncage, whose source is thread(*concurrent) (found by the
coordinator's gate). That program is the one of the six that imports the mutex routines
(_QRNX_LOCK_MTX, _QRNX_UNLOCK_MTX, _QRNX_CNL_MTX), and by its source the only one with
a procedure-local static(*allthread) field (dcl-s ms int(10) inz(2000) static(*allthread) in main). The captures lext-allthread-local-c and
lext-allthread-local-noinz-c (adopted as twins) both import the lock: it is the
procedure-local static(*allthread) field that imports it, with or without a start
value. A global one does not (lext-static-all-c), and a serialized module cannot have the
field at all (the compiler refuses it, RNF3837: STATIC(*ALLTHREAD) is only valid in a
concurrent module), so a module with that field is always concurrent, which agrees with the
detection. Every public concurrent capture
has the bit on its own areas (31 of 31) and the serialized one has none, so the
bit separates every public case. After the change thread(*serialize) is in the
ctl-opt of lext-serialize-c and of no other emit of the six l10-async-*
programs and the 31 lext-* captures; THREAD(*CONCURRENT) keeps its failure
line exactly where the capture has the keyword. A test gives the serialized twin the
thread-local bit and expects no serialized report, and asserts that the two
allthread-local twins import the lock, have the bit, and get no keyword. The emit rule of
thread(*concurrent) needs the thread-local declarations and is not this change.

Evidence and counter

Five captures are adopted as twins (EXPERIMENTS/lext-psds-void-c, -void-p,
-arg-c, -read-c, -ret-c); src/rpg_lext3_test.zig asserts their calls
(CALLP BUMP(), CALLP ADDTO(5), = TWICE(3)), the plain control, the
refusal on a changed offset, and the emit lines. Against the lext2 build the
two call tests fail (the refusal test holds in both states; the emit tests need
the new lines).
lext-psds-import-c is not adopted: its call of an imported function with a
result that is dropped (99 75 ... 3) stops in the plain twin as well
(lext-call-import-p), another family.

Counter, L10 development data (six programs, Debug build, sourcerer dump):
recovered 1,490 of 3,449 before, 1,997 after (43.2 to 57.9 percent):
459 unresolved and 48 partial statements recover, 39 unresolved become partial,
3 partial become unresolved, no row leaves. Stops by stage: control_flow 753 to
694, type_inference 794 to 340, classification 217 to 193, value_flow 155 to 185,
call 40. Gates: twins and emit against the newest baseline: no change in any
public twin.

A count to read with the number: 228 of the 1,997 recovered statements have
the text of a select block (SELECT, WHEN, OTHER, ENDSL) where the source
has if/endif; 138 of the 1,490 before. A codeless statement before a condition
is read as the head of a select (rpg_statements.renderStatement), and the
emit refuses those blocks ("its block is not whole"). The recovered count
includes them. That is a family of its own for a next round; this rule did not
change the reading.

Open family: if/endif read as select

228 of the 1,997 recovered statements on the L10 development data (138 of
1,490 before this round) have the text of a select block (SELECT, WHEN,
OTHER, ENDSL) where the source has if/endif/else. A codeless statement
before a condition is read as the head of a select
(rpg_statements.renderStatement), the block is closed as ENDSL, and the emit
refuses it ("its block is not whole"). The reading is false and inside the
recovered count; the first candidate for the round after this one.

What stops next

340 statements stop at a type_inference: a call of an imported procedure whose
result is dropped (99 75 ... 3; pthread_*, QMHSNDPM), and a call with a
result through an address (99 32 35 33 35) in the program status form.
lext-call-import-p reproduces the first without THREAD and without the
program status data structure.

Release gate (coordinator, before this pull request)

Checked on the binary of the shipped commit b821f381. The head of this branch is that commit, re-stamped once after the pull request opened (the same tree).

  • Development set: the five emits are byte-identical to v0.1.496.
  • Committed public twins at v0.1.496 (emit-all.sh against the release baseline): one emit changes,
    lext-serialize-c, whose ctl-opt gains thread(*serialize). Host round trip of that twin: different in 12
    components with v0.1.496, match with this build.
  • L10 test programs: direction unchanged.
  • Captures (capture-emit-diff.sh, 40 public cases: lext-thread-call, lext-psds-call, lext-allthread-local
    and the six L10 development programs): 14 emits change. In the lane captures every newly emitted statement
    agrees with its probe source (BUMP(), ADDTO(5)). In the six L10 development programs 283 newly emitted
    statements were paired with their source lines by statement number: each names the procedure the source calls
    and keeps its literals (operand names are generated, so they are not compared).
  • Module lines: in all 40 cases the thread keyword or refusal line and the PSDS refusal line agree with what
    the source declares. A first build of this branch wrote a THREAD(*SERIALIZE) line on l10-async-asyncage, a
    thread(*concurrent) module; this build does not (the lock import alone does not prove a serialized module).
  • The added PSDS line raises the failure count of three captures by one (lext-psds-import-c, -monitor-c,
    -ret-c); none of them is a committed twin at v0.1.496, and the line is true for each.

Gates (ship.sh, commit b821f381):

{
  "checks": {
    "net_statements": 0
  },
  "census": {
    "statuses": {
      "recovered": 645,
      "partial": 0,
      "unresolved": 0
    },
    "regressions": 0,
    "newly_recovered": 0
  },
  "golden": "golden floor 173 ceiling 0 <= dump 173/5/0",
  "legend": "regenerated",
  "emit": {
    "worse": [],
    "better": 0,
    "newly_clean": 0,
    "clean_delta": 0,
    "base": 588,
    "new": 588
  },
  "twins": {
    "groups": 826,
    "recovered": 804,
    "regressions": [],
    "newly": 0
  },
  "allgroups": {
    "regressions": [],
    "newly": [],
    "worse": []
  },
  "full_check": {
    "exit": 0,
    "seconds": 979
  }
}
# The program status data structure's status code after a call (touches rpg_source.zig, regenerates docs/OPERATIONS.md) Lane Lext, round three. Entry condition: the capture `lext-psds-call` reproduces the third post-call form of the L10 development programs, and it is a PSDS family, not a THREAD one. With the lext2 build the tracer shows the stop at `rpg_frames.rewrite s02` (the status unit) for `lext-psds-void-c` and for the control `lext-psds-void-p`, which has no THREAD. ## What the captures show A module with `dcl-ds psds psds` has after each call of a procedure of the module, in front of the status unit of the module without it: - `29 P 10 T; 37 P 15 T`: the status code (bytes 11 to 15 of the 429-byte static area P) rolls into the previous status (16 to 20). T is a 5-byte character type. - `27 T'; 37 P 10 T` after each store of the call status R: the status code is set, to `00001` after the store of 1 (the callee failed) and to `00000` after the store of 0. With THREAD the unit is `roll; 27 0; 35 R; set 00000` (lext-psds-void-c, -arg-c, -read-c, -ret-c). Without THREAD it is `roll; 29 F 96 Q; 32 B 0 I2; 27 1; 7; 16 L; 27 1; 35 R; set 00001; 18 E; 15 L; 27 0; 35 R; set 00000; 15 E` (lext-psds-void-p): the unit of `rpg_frames.status` with the two sets (the constants of the two texts are `F0F0F0F0F1` and `F0F0F0F0F0`). ## The rule `rpg_frames.psdsStatus`, tried first by `status` when `psdsRoll` finds the roll. It takes the roll, then either the thread unit (`threadStatus`) and the set, or the plain unit (`statusUnit`, split out of `status` with no change to its checks) with the two sets taken out at the places above. The area must be the 429-byte static symbol, the type 5 bytes, the offsets 10 and 15 and the texts those two; a set at another offset leaves the statement open (a test changes it). The roll and the sets are bookkeeping that the program status data structure of the source reads: the statement text ignores them. One failure line and one emit rule. `ctlOpt` adds `SOURCERER-FAIL PSDS: the program status data structure has no emit rule` when `rpg_frames.programStatusRolled` finds the roll straight after a call (the cycle and SQL write the same bytes elsewhere: a first version that looked at every roll made 24 public twins worse). And `ctlOpt` writes `thread(*serialize)` into the emitted `ctl-opt` of a serialized module, which closes the gap the coordinator found after round two: `lext-serialize-c` recovers all its statements and its text had no keyword, so it could not rebuild the object. A first version wrote a failure line instead; a failure line raises the failure count of a committed twin, which the emit gate refuses, and the keyword is the emit rule. `lext-serialize-c` keeps its two failures (6 emitted, 2 failures, as v0.1.496) and gains the keyword. `rpg_source.zig` is touched for these two lines only. The serialized module is `rpg_source_decl.serialized`: the module imports the mutex routine of the run time (`_QRNX_LOCK_MTX`) and has no static symbol with the thread-local bit. The import alone is not enough. It also fired on `l10-async-asyncage`, whose source is `thread(*concurrent)` (found by the coordinator's gate). That program is the one of the six that imports the mutex routines (`_QRNX_LOCK_MTX`, `_QRNX_UNLOCK_MTX`, `_QRNX_CNL_MTX`), and by its source the only one with a procedure-local `static(*allthread)` field (`dcl-s ms int(10) inz(2000) static(*allthread)` in `main`). The captures `lext-allthread-local-c` and `lext-allthread-local-noinz-c` (adopted as twins) both import the lock: it is the procedure-local `static(*allthread)` field that imports it, with or without a start value. A global one does not (`lext-static-all-c`), and a serialized module cannot have the field at all (the compiler refuses it, RNF3837: `STATIC(*ALLTHREAD)` is only valid in a concurrent module), so a module with that field is always concurrent, which agrees with the detection. Every public concurrent capture has the bit on its own areas (31 of 31) and the serialized one has none, so the bit separates every public case. After the change `thread(*serialize)` is in the `ctl-opt` of `lext-serialize-c` and of no other emit of the six `l10-async-*` programs and the 31 `lext-*` captures; `THREAD(*CONCURRENT)` keeps its failure line exactly where the capture has the keyword. A test gives the serialized twin the thread-local bit and expects no serialized report, and asserts that the two allthread-local twins import the lock, have the bit, and get no keyword. The emit rule of `thread(*concurrent)` needs the thread-local declarations and is not this change. ## Evidence and counter Five captures are adopted as twins (`EXPERIMENTS/lext-psds-void-c`, `-void-p`, `-arg-c`, `-read-c`, `-ret-c`); `src/rpg_lext3_test.zig` asserts their calls (`CALLP BUMP()`, `CALLP ADDTO(5)`, `= TWICE(3)`), the plain control, the refusal on a changed offset, and the emit lines. Against the lext2 build the two call tests fail (the refusal test holds in both states; the emit tests need the new lines). `lext-psds-import-c` is not adopted: its call of an imported function with a result that is dropped (`99 75 ... 3`) stops in the plain twin as well (`lext-call-import-p`), another family. Counter, L10 development data (six programs, Debug build, `sourcerer dump`): recovered **1,490 of 3,449 before, 1,997 after** (43.2 to 57.9 percent): 459 unresolved and 48 partial statements recover, 39 unresolved become partial, 3 partial become unresolved, no row leaves. Stops by stage: control_flow 753 to 694, type_inference 794 to 340, classification 217 to 193, value_flow 155 to 185, call 40. Gates: twins and emit against the newest baseline: no change in any public twin. A count to read with the number: 228 of the 1,997 recovered statements have the text of a `select` block (`SELECT`, `WHEN`, `OTHER`, `ENDSL`) where the source has `if`/`endif`; 138 of the 1,490 before. A codeless statement before a condition is read as the head of a `select` (`rpg_statements.renderStatement`), and the emit refuses those blocks ("its block is not whole"). The recovered count includes them. That is a family of its own for a next round; this rule did not change the reading. ## Open family: `if`/`endif` read as `select` 228 of the 1,997 recovered statements on the L10 development data (138 of 1,490 before this round) have the text of a `select` block (`SELECT`, `WHEN`, `OTHER`, `ENDSL`) where the source has `if`/`endif`/`else`. A codeless statement before a condition is read as the head of a `select` (`rpg_statements.renderStatement`), the block is closed as `ENDSL`, and the emit refuses it ("its block is not whole"). The reading is false and inside the recovered count; the first candidate for the round after this one. ## What stops next 340 statements stop at a type_inference: a call of an imported procedure whose result is dropped (`99 75 ... 3`; `pthread_*`, `QMHSNDPM`), and a call with a result through an address (`99 32 35 33 35`) in the program status form. `lext-call-import-p` reproduces the first without THREAD and without the program status data structure. ## Release gate (coordinator, before this pull request) Checked on the binary of the shipped commit b821f381. The head of this branch is that commit, re-stamped once after the pull request opened (the same tree). - Development set: the five emits are byte-identical to v0.1.496. - Committed public twins at v0.1.496 (`emit-all.sh` against the release baseline): one emit changes, `lext-serialize-c`, whose `ctl-opt` gains `thread(*serialize)`. Host round trip of that twin: `different` in 12 components with v0.1.496, `match` with this build. - L10 test programs: direction unchanged. - Captures (`capture-emit-diff.sh`, 40 public cases: `lext-thread-call`, `lext-psds-call`, `lext-allthread-local` and the six L10 development programs): 14 emits change. In the lane captures every newly emitted statement agrees with its probe source (`BUMP()`, `ADDTO(5)`). In the six L10 development programs 283 newly emitted statements were paired with their source lines by statement number: each names the procedure the source calls and keeps its literals (operand names are generated, so they are not compared). - Module lines: in all 40 cases the thread keyword or refusal line and the PSDS refusal line agree with what the source declares. A first build of this branch wrote a THREAD(*SERIALIZE) line on `l10-async-asyncage`, a `thread(*concurrent)` module; this build does not (the lock import alone does not prove a serialized module). - The added PSDS line raises the failure count of three captures by one (`lext-psds-import-c`, `-monitor-c`, `-ret-c`); none of them is a committed twin at v0.1.496, and the line is true for each. Gates (ship.sh, commit b821f381): ```json { "checks": { "net_statements": 0 }, "census": { "statuses": { "recovered": 645, "partial": 0, "unresolved": 0 }, "regressions": 0, "newly_recovered": 0 }, "golden": "golden floor 173 ceiling 0 <= dump 173/5/0", "legend": "regenerated", "emit": { "worse": [], "better": 0, "newly_clean": 0, "clean_delta": 0, "base": 588, "new": 588 }, "twins": { "groups": 826, "recovered": 804, "regressions": [], "newly": 0 }, "allgroups": { "regressions": [], "newly": [], "worse": [] }, "full_check": { "exit": 0, "seconds": 979 } } ```
sean force-pushed lext3 from b821f381b4
Some checks reported errors
binjovi/ci A newer source or plan superseded this Build
to 6b445d7387
All checks were successful
binjovi/ci Binjovi completed the frozen plan
2026-11-10 18:29:00 +00:00
Compare
binjovi-bot manually merged commit 6b445d7387 into trunk 2026-11-10 18:41:36 +00:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
sean/sourcerer!780
No description provided.