sourcerer compare names the check behind an unsupported_layout refusal #794
Loading…
Reference in a new issue
No description provided.
Delete branch "lcmp"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
sourcerer comparenames the check behind an unsupported_layout refusalLane Lcmp. The change touches none of
rpg_source.zig,rpg_eval.zig,rpg_statements.zigor the goldentest. It adds a site to each refusal of the comparer and nothing else: it accepts no new layout and moves no
verdict. The reason stays
unsupported_layout, byte for byte.What changes
Every return of
UnsupportedComparisonLayout(ten, nine incompare_ile.zigand one inapp/compare.zig)goes through
comparison.refuse(site, value), which records the check and the value it read and returns thesame error.
app/compare.zigreads the record once per side into the newEvidence.refusaland clears itbefore each side starts. Three compound checks are split so that one token names one check, with the same
order and the same outcome: the chunk, construction and pointer counts of a static frame (limit 4096 each),
and the registered count and the handler count of the exports.
compare_ile.zigunless named)receipt_typeapp/compare.zig load: the receipt is not aPGMobject_kindprepare: the saved object is not a programlayout_kindprepare: the image is not a program layoutmodule_countprepare: the module table does not hold one moduleheader_78_size,header_80_sizecreationData: size of the table at module header +0x78 / +0x80 outside 16 B to 1 MiBconstant_countstaticData: the activation header does not count one constant areastatic_frame_countstaticData: the activation header (+56) does not count one static framestatic_frame_sizestaticData: frame size 0 or above 16 MiBstatic_chunk_count,static_construction_count,static_pointer_countstaticData: more than 4096 rowsregistered_count,handler_countexportData: registered count above the limit, or handlers differ from itOutput. The first line is unchanged (
comparison refused: unsupported_layout, exit 3). The text form adds oneline per side that carries a site, after it and before the
different component:lines, in a form that startswith neither
comparisonnordifferent component:(roundtrip.shandsame-side.shkey on those):The JSON verdict (schema
sourcerer.comparison.v1) addsreference.refusalandtwin.refusal, present onlyon a side that carries a site:
{"site": "...", "value": N or null}(a reader seesnullfor an absent key).A report without a site is byte for byte what it was: the three golden
structural-comparison.jsonfiles oftests/cli/rpg-corpus.shpin it, and an always-presentnullkey failed that check on the first ship.result,reason,components,counts,different_componentsand the exit code are as before.docs/COMPARISON.mdlists the report fields and is now short of this key and this line; it is not edited here(coordinator).
staticDataispubso that a test without the decompiler can reach it.compare_site_test.zig(registered in
src/root.zig) asserts: the three refused two-frame twins namestatic_frame_countwith thevalue read straight from the fixture, the eight matching twins name no site, every check of the static data
names itself and its value on a mutated copy,
preparenames the object kind, the report keeps the reason andcarries the site in the text and in the JSON, a recorded refusal does not move to a side that failed for
another reason, and a scan of the sources proves that no
return error.UnsupportedComparisonLayoutis leftand that every site token is used by exactly one check. Red: with the
static_frame_countcall replaced bythe bare error, three of the seven tests fail.
No verdict moved
The twins: every committed
*.savfundersrc/testdata(1718; 1669 programs) was dumped with the basebinary (trunk
84eeba13) and compared with the base and the head binary, each program with itself (1669) andeach observable twin with its stripped twin (749): 2418 comparisons. For each, the exit code, the text without
the new lines, and the JSON (without the two
refusalkeys of a refused side, the only ones there are) arebyte-identical between the two binaries. The
six Async captures and the round-trip objects give the same verdicts as before (refused, exit 3).
Measurements (not shipped)
The five programs and
asyncprom(it does not compile, so there is no rebuilt object), the original objects(
runs/pub-l10-async-<id>-11*/cases/1/ref/dump/<NAME>.PGM) and the captured observable twins, each comparedwith itself and with its rebuilt or stripped twin: every comparison refuses at
static_frame_count,value 2, on the reference and on the twin. Nothing else fires first; no other site fires on any of them.
0, 0x14000, 0xb0, 00, 0x14000, 0x80, 0Frame 1 has flag word 0x20000 and frame 2 0x10000 (header +4). The frame header table is at activation +48,
80 bytes per header; the two relocation rows that fill the frame slots are type 0x0c (ordinal 1) and type 0
(ordinal 2), as in the committed two-frame twins. The observable and the stripped twin of a program have the
same values. 0x14000 = 4096 pointers + 4096 int(10):
gTaskandgGenof the Async source (dcl-c ASYNC_MAX_TASKS 4096, bothstatic(*allthread)).The THREAD twins. Of the 14 committed twin directories whose source has
thread(*concurrent)(28 objects),22 compare (
matchagainst themselves) with no site, and exactly 6 refuse, all atstatic_frame_count,value 2: the observable and the stripped object of
lext-static-all-c,lext-allthread-local-candlext-allthread-local-noinz-c. The six twins that match have one static frame (+56 = 1):lext-call-void-c,-call-const-c,-call-value-c,-call-ref-c,-call-nested-c,-call-ret-c. Across all 2418 comparisonsthe site
static_frame_countfires only on the three refused lext twins; no other site fires on any twin.So thread mode is not the cause: a second, plain static frame is (
static(*allthread), which the matchingtwins do not have).
Design for change 2 (what the comparer would read; not shipped)
Four places read the static frames as if there were one. All read the frame count at activation +56 and the
header table at +48 (80 bytes per header):
compare_ile.staticData(count != 1, then one header),compare_ile.relocationData(a frame row must have ordinal 1 at +8,header+56 == 1, and a type 0x0c rowrequires +60 == 0 and +68 == 1),
constant_references.initialPointers(count != 1, one header), andconstant_references.tables, which proves the tables of the module static header at +8, +16, +32, +48 and +64(the header that starts with 0x0100000000000000) and not the one at +24.
Experiment, in a scratch copy of the tree that is never shipped (
runs/lcmp-scratch/exp), measured by theverdicts of
sourcerer compareon the six programs, the round trips and the 2418 committed comparisons. Theedits are cumulative; each step says what the build before it refused with:
staticDataloops over thecountheaders (the same checks and hashes per frame) and hashes the 16-byterecord behind header +16 instead of the address (inferred necessary, because the address differs between
an observable and its stripped twin; never seen failing), and
relocationDatatakes a frame ordinal in1..
header+56and a type 0x0c row with +68 >= 1 (the +60 == 0 rule dropped, not replaced). Refused:ambiguous_referenceat the construction check of frame 1. The row word is 0x81000001 where the 16constructions of
lext-serialize-candlext-call-void-pare 0x80000001, and the target is beyond thesize of frame 1 (5944 against 4384) but inside frame 2 (82112).
unsupported_relocationatinitialPointers(count != 1).initialPointersloops over the frames. Each program matches itself and differs from its own observable orstripped twin in
constant_areaonly: the area (24064 B in each of the six) holds one 8-byte address ofitself that no reference proves (
constant_area_unproved_pointersis 1 in each, 0 in the single-frametwins).
tablesalso reads the table atstatics + 24, stride 16. That word is its pointer: the target has length0x20 and count 1, the format
table()checks, and its row is the descriptor0, 0x14000, 0xb0, 0thatheader +16 of frame 2 names (read on asyncage; the committed twins have a null +24, since they matched
before this step). The six originals match themselves, each observable matches its stripped twin with no
different component, and
constant_area_unproved_pointersis 0.With the four steps, the five programs and
asyncpromcompare, and the three refused lext twins compare(
match, self and observable against stripped). Of the 2418 committed comparisons only those 9 rows move(
refusedtomatch); the other 2409 keep result, reason and components. The round trips then give averdict: each original against its rebuilt twin is
different(in instructions, control_flow,creation_instructions, symbols, types, literals, constant_area, initialization, imports, exports and
relocations, and on asyncage in attributes too) where it was
refused.Proposed for change 2 and not exercised: bound the frame offset of a relocation row and the target of a
construction by the size of the frame that the row (ordinal) or the row word names (0x81000001 against
0x80000001: one value seen), instead of skipping the bound; require +60 and +68 to equal the number of rows of
type 0 and of type 0x0c, each ordinal used once; cap the frame count (2 observed). Inferred, not seen failing:
the hash of the record behind header +16.
Request
~/data/sourcerer-loop/requests/lcmp/frames-two/(12 cases, README names the question): from which size azero-initialised
static(*allthread)array gives frame 2 the +8 = 1 flag, the record and the table at +24;inzagainst noinz; the same array withoutstatic(*allthread); and a thread-local pointer start value ofan allthread field, of a thread-local field and of the big array, to read the row word 0x81000001. Not run (no
host job in this lane). The reader is a throwaway test of the lane (in the data tree,
runs/lcmp-scratch/lcmp_probe_scratch_test.zig.txt), not shipped.Gate
86666a077bc4b33816e6