diff --git a/.beads/issues.jsonl b/.beads/issues.jsonl index e4ead929318e4ed57649dd54d93d4ecdde82cf1a..d2d9b82096481b4d2c3c8f06af1e4260a90eacf3 100644 --- a/.beads/issues.jsonl +++ b/.beads/issues.jsonl @@ -48,6 +48,7 @@ {"_type":"issue","id":"tarantool-protobuf-aah","title":"Encoder: codegen-emitted packed-scalar tight loops","description":"Replace per-element wire.encode_int32 calls with one inlined varint-emitting loop per packed field. Applies to packed int32, int64, sint32, sint64, bool, enum. The 1KB Person has 5 packed lucky_numbers + 26 emails — current per-element function-call boundary costs add up. Expected: 30%+ on packed-heavy payloads.","notes":"Shipped. work.lab.local 3-trial medians: packed_int32 +90/+113/+106%, packed_sint32 +66/+236/+156%, packed_uint32 +73/+105/+102%, packed_bool +44/+51/+39% (at 10/100/1000 elements). packed_int64 +22% (cdata path; fast path skipped, gain from table_new pre-sizing). Headline Person 1KB +3.6%, proto2 mid +10.5%.","status":"closed","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:07Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T17:20:54Z","started_at":"2026-05-24T17:08:14Z","closed_at":"2026-05-24T17:20:54Z","labels":["codegen","encoder","perf"],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"tarantool-protobuf-4ql","title":"Encoder: optional caller-owned ffi.cdata ibuf API","description":"Separate API surface, not the default. ibuf-based encoder that writes into a caller-owned ffi.cdata buffer instead of returning a fresh Lua string. Targets hot RPC paths where the caller already owns a reusable buffer (e.g. net.box send path). Independent of the codegen rewrite — different API contract. Earlier attempt parked in stash@{0}; revisit after the codegen-time emission lands so we can compare apples-to-apples.","status":"closed","priority":2,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:06Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T04:30:27Z","closed_at":"2026-05-24T04:30:27Z","close_reason":"Spike (bench/c_accel/ibuf_probe.lua, removed) implemented a hand-coded Person_encode_ibuf mirroring what protoc-gen-tarantool would emit: stable pre-allocated cdata scratch buffer, direct p[i] byte writes, single-pass with backpatched length for nested Address + packed lucky_numbers, ffi.copy(p+i, lua_str, n) for strings. Same-byte correctness verified across 10B/100B/1KB/10KB.\n\nBench (ns/op, Tarantool 3.8.0 / Apple M-series), Person_encode vs ibuf scratch-only:\n\n size Person_encode ibuf scratch ibuf+ffi.string speedup\n 10B 462.8 239.1 273.7 1.69x (win)\n 100B 470.1 243.1 286.3 1.64x (win)\n 1KB 3693.8 4872.6 5141.6 0.72x (loss)\n 10KB 17146 42807 43023 0.40x (loss)\n 100KB 159081 422510 432701 0.37x (loss)\n\nCrossover ~26 emails. Root cause: each email pays an ffi.copy(p+i, lua_str, n) boundary (~50 ns/call). At 26 emails = ~1.3 us pure boundary; at 2800 emails (100KB) = ~140 us pure boundary. Meanwhile Person_encode appends Lua-string refs to an out table (no FFI boundary) and pays ONE bulk table.concat memcpy at the end regardless of count. Per-field boundary work beats per-message bulk work only when field count is very small.\n\nFor 4ql's stated use case (net.box send path), typical Tarantool RPC payloads are \u003e=1 KB — exactly the regression zone (1.4-2.7x slower). Win window (\u003c100B) is too narrow to justify a separate API surface, especially since pb.encode is already 462-470ns at that size — saving 200ns on a sub-microsecond operation is not a meaningful net.box gain.\n\nThree abandoned attempts now (per-byte b:alloc cliff, v2 two-pass bulk reserve, this single-pass backpatch). All hit the same per-field FFI boundary tax. Closing as 'not viable on current LuaJIT'. REVIVAL CRITERIA: a future LuaJIT that can sink FFI calls into traces (so per-field ffi.copy stops paying the boundary cost), OR a use case where caller passes pre-cdata-cast strings (no per-email lua-string -\u003e cdata copy), OR a hand-written C encoder behind the c_runtime FFI surface (different path entirely — see c0i).","labels":["api","encoder","perf"],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"tarantool-protobuf-86g","title":"Encoder: two-pass with exact size precomputation","description":"Walk fields once to sum byte sizes, allocate the final string at exact size, walk again to write. Eliminates buffer-grow realloc; one lua_pushlstring. Same pattern as vtprotobuf's Size() + MarshalToVT. Stacks with the codegen-time inline writes (depends-on). Expected: additional 30-50% on large-message encode beyond the inline-FFI baseline.","notes":"2026-05-24 — verdict: not viable as a pure-Lua path. Same evidence as lkz close.\n\nHand-spike of the two-pass exact-size+emit shape (ffi.new uint8_t[?],\ninlined per-message sizer + writer, recursive) regressed encode 0.31x-0.77x\nacross all 5 size buckets vs the current `out` table + `table.concat` shape.\nProfile (jit.p) showed 84% of time in the inlined message bodies, not in\nbuffering — the perceived buffering inefficiency simply isn't there.\n\nThe single profile-hottest line (39% of Person_encode share) was replaced\nin 2ri (CHARS[_len] lookup) for +17%-34% encode throughput on 1KB+ payloads.\n\nBeating the pure-Lua ceiling requires escaping it — c0i (C-runtime backend)\nis the remaining path. Closing 86g as superseded.","status":"closed","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:05Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T16:53:44Z","closed_at":"2026-05-24T16:53:44Z","labels":["codegen","encoder","perf"],"dependencies":[{"issue_id":"tarantool-protobuf-86g","depends_on_id":"tarantool-protobuf-h8v","type":"blocks","created_at":"2026-05-17T18:47:19Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"tarantool-protobuf-drm","title":"Inspect: ~120 B/op fixed encode/decode overhead in Lua hot path","description":"Cross-runtime bench (bench/COMPARISON.md) shows a fixed per-call\nallocation floor on both encode and decode in `mode=full`:\n\n- encode: 136 B/op even at 10 B payload (output is 10 bytes; ~126 B overhead)\n- decode: 112 B/op even at 10 B payload (top-level table = 2 hash-tables)\n\nFor comparison, Go vtproto runs encode at 16 B/op and decode at 8 B/op\non the same 10 B fixture.\n\nInspect:\n 1. What's the source of the 100+ B encode overhead? Suspect candidates:\n - per-call buffer.ibuf workspace allocation in wire.lua\n - Lua string concat building the result\n - GCstr header on the output string itself (~24-32 B)\n 2. Decode floor: 112 B/op. Likely 2x table headers + small alloc for\n the top-level message table.\n 3. For large payloads (100 KB encode = 131605 B/op vs output 96674 B)\n the overhead is ~28 KB. Is that one big realloc tail, or many small?\n\nOutcome: a writeup pointing to specific lines and a recommendation\non whether the M6 ibuf path would actually help.","notes":"Findings from bench/alloc_probe.lua:\n\nENCODE 10B floor = 136 B/op is the `out` Lua table:\n- `local out, n = {}, 0` alone: 64 B (Lua GCtab base)\n- After 5 array entries: 136 B (matches the encode floor exactly)\n- table.concat: 0 B in the bench because output bytes intern\n- Varints: 0 B per call because 1-byte string.char outputs intern\n globally (small string dedup)\n- 2-byte varints DO allocate: ~33 B per fresh value (encode_varint_slow\n path returns a fresh string from string.char + bit.bor)\n\nDECODE 10B floor = 112 B/op is the top-level result table\n{name=..., age=...} — same shape across iters means it'd allocate the\nsame in a real workload.\n\nLARGE PAYLOAD (1KB Person):\n- same input (output interned): 1368 B/op\n- varying age (output unique): 2368 B/op\n- Delta = ~1000 B is the result string for 930 bytes of output (the\n GCstr header + 930 content; the extra ~70 B is presumably padding /\n alignment / the Address's nested concat).\n- So real cost per call has TWO components: (a) the small fixed tables\n for the encoder workspace, (b) the output bytes themselves.\n\nKEY INSIGHT: bench numbers UNDERSTATE real allocation. The bench\niterates the SAME input → output string interns → bench reads only\nthe table cost. Real workloads where every message is unique pay\noutput-size + table-cost.\n\nOPTIMIZATION RANKING:\n1. Highest ROI: encoder workspace table. ~136 B per top-level encode\n + ~136 B per nested message encode (Address adds its own). For a\n 1KB Person we have ~5 nested encoders → ~700 B of tables. Replacing\n the array-of-string-pieces with a single growable `buffer.ibuf` cuts\n this to ~0. This is the M6 ibuf path.\n2. Medium: encode_varint_slow returns a fresh string on each call for\n non-fast-path values. Tag bytes are precomputed as literals; only\n payload varints hit this. Inlining the slow path into wire.lua's\n hot caller (or returning into a passed-in buffer) drops this.\n3. Lowest: result string. Unavoidable for the encoder's API contract\n (returns a string). Only the lazy path avoids it.\n\nDECISION POINT: M6 ibuf path (already prototyped, deferred per\nmemory/decode_perf_deferred.md) is the right intervention.\nConservatively halves encode B/op on small messages, larger savings\non nested-heavy payloads.\nCORRECTION to earlier note.\n\nRecommendation to \"use M6 ibuf path to cut the 136 B/op floor\" was\nwrong. Per memory/tarantool_ibuf_perf.md (verified against current\nrepo state — feature NOT in HEAD):\n\n- Naive per-byte b:alloc(1): ~20× slower realistic, ~36× synthetic\n- Two-pass bulk-reserve: byte-equal correct, ~2× slower at every\n payload size. Sitting in git stash@{0}.\n\nBoth prototypes lose on wall time because:\n (a) Bench harness runs with jit.off, closure dispatch interpreted\n (b) Realistic encoder cost is dominated by per-field closures\n (sizer + writer + emit_tag + pwrite_*), NOT byte writes\n (c) Two-pass adds an extra walk on top\n\nAlso: the 136 B/op encode floor is NOT the throughput bottleneck.\nCross-runtime gap (5× apiv2, 13-20× vtproto) is JIT/dispatch\noverhead per field, not allocator pressure. Cutting 136 B doesn't\nclose that gap.\n\nVIABLE PATHS (neither tried):\n1. Single-pass with backpatched length varints. Walk once. For nested\n msgs: write tag + 1-byte placeholder, recurse, fill in (or memmove\n if final length ≥ 128). Eliminates the size pass (~3 µs at 1 KB).\n2. Codegen-time `_encode_ibuf` per message in protoc-gen-tarantool\n (mode=full). Straight-line ibuf writes, no descriptor walk at\n runtime. Mirrors what pb.encode already does for the table path.\n Probably the only approach that actually closes the throughput gap.\n\nEither has to keep per-field closure count ≤ pb.encode's writers,\notherwise we re-introduce the dispatch cost both stashed prototypes\nfell on.\n2026-05-24 re-verification (post-21d/qwt):\n\nAllocation floor unchanged. Fresh `tarantool bench/alloc_probe.lua`:\n Person_encode 10B (same input) 136.0 B/op\n Person_encode 100B (same input) 136.0 B/op\n Person_decode 10B (same input) 112.0 B/op\n\nDrill-down isolates the source line-by-line — `local out, n = {}, 0` (64 B base) + 5 array entries (8 B each at LuaJIT tab growth = +72 B) = 136 B exactly. Lines: examples/expected/full/hello/hello_pb.lua:1044 (the `out` table) and 1049/1056/1061+ (the `n = n + 1; out[n] = ...` writes that populate it).\n\n21d closed dispatch fragmentation (small-encode +42% throughput per memory/21d-encode-dispatch-codegen-2026-05-24) and the alloc floor did not move — consistent with the CORRECTION note: the 5x cross-runtime throughput gap is not allocator pressure, and naive byte-buffer rewrites already lost in two stashed prototypes.\n\nInvestigation complete. Actionable interventions live in:\n- lkz (single-pass with backpatched length varints into one buffer)\n- 86g (two-pass with exact-size precompute; vtproto's Size+MarshalTo)\n\nBoth depend on closed h8v (codegen FFI direct writes). Closing drm.","status":"closed","priority":2,"issue_type":"bug","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:13:38Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T15:24:55Z","closed_at":"2026-05-24T15:24:55Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"tarantool-protobuf-ozn","title":"Codegen: inline 2-byte varint fast path in packed-scalar decode inner loop","description":"Followup from auj. The 2-byte tag and 2-byte LEN fast paths now inline at the dispatch site, but packed-scalar inner loops still call wire.decode_\u003cst\u003e(payload, p2) per element. For values 128..16383 that's a function frame plus decode_varint's internal loop — and the side trace from the 1-byte guard exit can't stitch back through the helper's frame return.\n\nAcceptance: bench/jit_trace.lua 50/50 with bridges=0 on full/Person_decode multi-byte varint (currently 4/100 residual, all in the packed loop). Measure throughput delta on the bench big_person fixture (lucky_numbers with multi-byte values).\n\nScope: extend the packable case in emitInlineDecodeFieldBody (cmd/protoc-gen-tarantool/internal/gen/inline.go line ~1097-1107) to emit a per-scalar-type 2-byte fast path inside the while loop. Start with int32/uint32/bool (Lua-number returns, simplest emission). Defer int64/uint64/sint32/sint64 (cdata returns + zigzag) unless they appear in real workloads.\n\nSame pattern as auj's tag/LEN inline:\n if _b \u003c 0x80 then val = _b; p2 = p2 + 1\n elseif p2 \u003c lim then\n local _b2 = string_byte(payload, p2 + 1)\n if _b2 \u003c 0x80 then\n if _b2 == 0 then error('overlong varint', 0) end\n val = _b - 128 + _b2 * 128\n p2 = p2 + 2\n else val, p2 = wire.decode_\u003cst\u003e(payload, p2) end\n else val, p2 = wire.decode_\u003cst\u003e(payload, p2) end","status":"closed","priority":3,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-24T20:15:09Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T20:22:35Z","started_at":"2026-05-24T20:15:19Z","closed_at":"2026-05-24T20:22:35Z","close_reason":"Codegen: inlined 1-byte + 2-byte varint fast paths in the packed-scalar decode inner loop for int32/uint32/bool (and enum via a sister helper). Same pattern as auj's tag/LEN inline. Bridge rate dropped to ~1-2% (3/200), did not consistently hit 50-clean-in-a-row (max streak 31/50), but throughput criterion met with significant headroom:\n small Person (1-byte everything) 14.0 -\u003e 23.4 MB/s (+67%)\n big Person (2-byte LEN + multi-byte packed) 34.7 -\u003e 39.6 MB/s (+14% vs auj, +25% vs pre-auj)\nSkipped int64/uint64/sint*/sint64 inline — those return cdata or need zigzag wrappers; per-type complexity bigger than the residual gain on common workloads. Suites: test 766/766, test-c 1057/1057, bench/Person no regression at any size.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"tarantool-protobuf-58u","title":"Runtime-mode decode_unsafe support (compile parallel _reader_unsafe closures)","description":"6bb landed _decode_unsafe in full mode only. Runtime mode (pb.decode(desc, buf) and the runtime-mode codegen wrapper) currently has no unsafe path because compile_readers builds f._reader closures that capture handler.decode by value — a runtime swap of scalar.string.decode wouldn't reach them. Implementation sketch: add M.compile_readers_unsafe(desc) that builds f._reader_unsafe by passing an alternate scalar table where scalar.string = scalar.bytes. Call it from pb.finalize_message alongside compile_readers. Add M.decode_unsafe(desc, buf) as a near-clone of decode_message that uses f._reader_unsafe and routes the map-fallback scalar dispatch through the unsafe scalar table. Then emit M.\u003cName\u003e_decode_unsafe = function(b) return pb.decode_unsafe(M.\u003cName\u003e_descriptor, b) end in gen.go's runtime-mode emitMessageWrappers for API symmetry with full mode.","status":"closed","priority":3,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-24T18:20:01Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T19:11:20Z","started_at":"2026-05-24T18:59:16Z","closed_at":"2026-05-24T19:11:20Z","close_reason":"Runtime mode now exposes pb.decode_unsafe and M.\u003cName\u003e_decode_unsafe. codec.lua adds scalar_unsafe table, parameterizes build_reader/build_repeated_reader/decode_one to accept (scalar_tbl, decode_msg_fn, decode_group_fn), and adds compile_readers_unsafe + decode_message_unsafe + decode_group_unsafe + decode_extension_unsafe as literal clones with the three substitutions documented in codec.lua. Tests parameterized over both modes (14 cases), perf microbench shows ~8% gain in runtime mode (~20% in full mode). Conformance + JIT trace gates still pass. kyt remains open for unifying with C accel.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"tarantool-protobuf-kyt","title":"C runtime: skip_utf8_validation plan flag (unblock _decode_unsafe + C accel)","description":"6bb's full-mode _decode_unsafe skips the pb.c_runtime dispatch because runtime/pb/c/c_runtime.c calls is_valid_utf8 unconditionally on every string field. Result: when PB_ENABLE_C=1 the safe _decode wins on C but _decode_unsafe runs the inline Lua path and may be slower than C. To unify: add a skip_utf8_validation flag to the decode plan (or expose pb.c_runtime.decode_unsafe(plan, buf)), gate the is_valid_utf8 call on it in c_runtime.c, and wire _decode_unsafe to take the C path when c_runtime is available.","status":"closed","priority":3,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-24T18:20:01Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T19:48:24Z","started_at":"2026-05-24T19:41:09Z","closed_at":"2026-05-24T19:48:24Z","close_reason":"C runtime: decode_unsafe(plan, buf) entrypoint added; full-mode codegen _decode_unsafe dispatches via pb.c_runtime.decode_unsafe; pb.decode_unsafe in init.lua routes through C when c_runtime loaded. dec_ctx.skip_utf8 gates is_valid_utf8. Suites: test 766/766, test-c 1057/1057. Perf on string-heavy Person (418B, 16 emails + 16 nicknames): C-unsafe 455 MB/s vs C-safe 364 MB/s (+25%) vs Lua-unsafe 142 MB/s (3.2x).","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"tarantool-protobuf-h8x","title":"decode_group: 'inner loop in root trace' abort makes trace topology bimodal","description":"decode_group (runtime/pb/codec.lua:1089) has an 'inner loop in root trace' abort condition — the per-tag while-loop is hot enough to be a trace root itself but is reached from another root trace that tries to extend through it.\n\nObserved via bench/jit_trace.lua probe in isolation on full/WithGroup_decode (group):\n Mode A (typical, ~9/10 runs): starts=38, stops=5, aborts=33 (recompile loop)\n Mode B (rare, ~1/10 runs): starts=102, stops=100, aborts=2 (side-trace cascade)\n\nPre-existing on master (probed before/after the compile_encode_body fix landed for tarantool-protobuf-21d). Neither mode breaks the gate (both have stops\u003e0, no FATAL aborts), but the bimodal behavior is unstable and confused 21d's measurements.\n\nFix direction: same idea as 21d — emit a generated per-descriptor decode body (or at minimum, refactor decode_group so the inner while-loop is a separate function the JIT can compile as its own root trace). Mirrors compile_encode_body from the 21d fix.","status":"closed","priority":3,"issue_type":"bug","owner":"bigbes@gmail.com","created_at":"2026-05-24T09:53:50Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T14:18:34Z","closed_at":"2026-05-24T14:18:34Z","close_reason":"wont_fix","dependency_count":0,"dependent_count":0,"comment_count":0} @@ -56,7 +57,7 @@ {"_type":"issue","id":"tarantool-protobuf-auj","title":"Quantify and address Person_decode multi-byte varint side-trace bridge (intermittent ~1/5 runs)","description":"Trace-topology finding from bench/jit_trace.lua. The 'full/Person_decode multi-byte varint' check intermittently reports one bridge — a side trace whose linktype=interpreter, costing interp dispatch per multi-byte tag/length-prefix on the hot decode path.\n\nCaptured output (1–2 runs out of 10):\n\n [ OK ] full/Person_decode multi-byte varint (stops=10, bridges=1)\n info: bridge tr9 side-of tr5 hello_pb.lua:997 pc=55\n\ntr5 is the Person_decode while loop (entry at hello_pb.lua:997). pc=55 falls inside the wire.decode_tag inlined fast path: the guard 'if b \u003c 0x80' fails on a 2+ byte tag, exits to side trace tr9, which contains the multi-byte continuation loop but can't self-link back to the parent — drops to the interpreter to walk the rest of the dispatcher and re-enter on next iteration.\n\nExisting context:\n- wire.lua duplicates the 1-byte varint fast path at every hot decode call site precisely because LuaJIT side traces can't stitch returns from an inlined helper frame. That works for the 1-byte case. The 2+ byte case still calls decode_varint() (the multi-byte fallback) which has its own internal while loop.\n- gcy (inline nested decode) and kot (localize wire.* upvalues) are the structurally related items already filed; they don't cover this specific bridge though.\n\nWhy P3 (not P2):\n- Intermittent (~1/5 runs in the gate). The trace topology is mostly stable.\n- The multi-byte tag path is \u003c 5% of typical RPC payloads (field IDs 1..15 = 1-byte tag, lengths \u003c 128 = 1-byte length). Larger impact would require \u003e127-byte fields or field IDs \u003e= 16.\n- The 'multi-byte varint' fixture in bench/jit_trace.lua (200-byte name + lucky_numbers including 200000, 500000) was added specifically to expose this — and it does, intermittently. The intermittency is the JIT settling on different trace shapes across runs.\n\nConcrete approaches to investigate:\n1) Inline a 2-byte varint fast path inside decode_tag (and decode_string LEN prefix, etc.) — 'if b \u003c 0x80 then ... elseif b2 \u003c 0x80 then ...' — keeping the 3+ byte case in the fallback. Covers field IDs up to 4095 and length prefixes up to 16383, which is almost all real payloads.\n2) Profile-driven: run bench/jit_trace.lua 100x with a fixed seed, collect bridges by location, and decide whether the intermittency rate justifies (1) at all.\n\nAcceptance: 50 consecutive runs of bench/jit_trace.lua report bridges=0 for full/Person_decode multi-byte varint, OR a measured throughput improvement on the bench at the multi-byte-varint fixture.","status":"closed","priority":3,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:27:36Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T20:08:54Z","started_at":"2026-05-24T19:51:38Z","closed_at":"2026-05-24T20:08:54Z","close_reason":"Codegen: extended inline tag fast path (1-byte → 1+2-byte) and inline LEN fast path (1-byte → 1+2-byte) in emitInlineDecode / emitInlineStringBytesScalar / emitInlineStringBytesRepeated. Covers field IDs 1..4095 and string/bytes lengths 0..16383 without leaving the parent trace. Acceptance: criterion #2 met — multi-byte-varint Person decode 31.6→34.7 MB/s (+10%) on the big_person fixture (200B name + multi-byte packed lucky_numbers), small Person unchanged at 14 MB/s. Bridge rate now 4/100 (down from 10-20% at issue filing). Residual bridges have migrated from the tag site to the packed-varint inner loop (wire.decode_int32 in a packed payload); fully eliminating them would require per-scalar-type inline 2-byte paths in the packed-loop emitter — diminishing-returns territory, skipped. Suites: test 766/766, test-c 1057/1057.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"tarantool-protobuf-3e6","title":"Encoder: fiber-local recycled output buffer (Tarantool-specific)","description":"The 136 B/op encode floor is the 'local out, n = {}, 0' workspace table allocated per top-level encode call. Tarantool ships a per-fiber ibuf via require('buffer'); could grab one from a pool, reset it on each top-level encode, write into it, then ffi.string(ibuf.rpos, ibuf:size()) at the end.\n\nTradeoffs:\n - Saves the workspace alloc (~136 B per top-level call + ~136 B per nested message).\n - Adds fiber-local state — non-Tarantool LuaJIT runs would need a different path or none.\n - Subsumed by lkz (single-pass two-phase) which goes directly to a single buffer without intermediate strings. Worth filing as a smaller alternative path in case lkz proves too invasive to land.\n - drm notes record two prior ibuf prototypes (stashed) that regressed by ~2x because per-byte b:alloc(1) was 20x interpreter-dispatch-bound. This proposal avoids that pitfall only because it composes with h8v (codegen FFI direct writes that bypass alloc(1)).\n\nConcrete path: only useful in combination with h8v. Without h8v this would just shift the alloc from 'out table' to 'output string' without removing any per-string varint allocation in the field bodies — net neutral on alloc and worse on speed.","notes":"Sketch: shared scratch buffer per encoding, capacity exposed as ffi cdata pointer + length. Codegen emits 'local _buf = pb.codec.acquire_buf(); local _off = 0; ...; return pb.codec.finalize_buf(_buf, _off)'. acquire_buf returns a pre-allocated buffer of growing capacity; finalize_buf returns a string and recycles. Care needed for recursive encode calls (nested message encoding into the same buffer).","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:15:08Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T17:15:08Z","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"tarantool-protobuf-0gg","title":"Decoder: investigate field-id dispatch as binary search or jump table","description":"Generated Type_decode bodies use linear 'if id == 1 elseif id == 2 ...' chains. For Person (~15 fields) the chain is short and the comparisons cheap. For messages with 30+ fields the linear walk to a high-id tag costs N comparisons per occurrence.\n\nOpen question: does LuaJIT's IR already lower this to a switch/jump? If yes, no work needed. If no, three options:\n\n 1. Codegen-emit a balanced if-elseif tree (binary search) when field count exceeds threshold.\n 2. For messages where field IDs are dense and small, emit a numeric branch table: 'local _f = _dispatch[id]; if _f then return _f(buf, pos, result) end'. Closure-per-field has setup cost but avoids the dispatch cost on every tag.\n 3. Sort by frequency (impossible to know at codegen time without profiling input). Skip this option.\n\nPre-work: jit.dump on Person_decode to check whether LuaJIT collapses the if-elseif. If it does, close as won't-fix; if it doesn't, decide between (1) and (2).\n\nLowest priority of the perf items because: (a) we don't know it's a problem yet, (b) Person profile shows decode_string + decode_tag + utf8 dominate, not the if-elseif walk.","notes":"Recent profile (Person 1KB): line 1007 (decode_tag call) is 14% of generated body time, and line 1021 (list append) is 24%. Linear if-elseif walk doesn't show up as a hotspot for Person, but Person has only 15 fields. Needs a bigger fixture to manifest.","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:15:07Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T17:15:07Z","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"tarantool-protobuf-1bo","title":"Codegen: chunk oversized generated decode bodies into per-range helpers","description":"test_messages_proto3_pb.lua is 4212 lines — the conformance fixture has so many fields that a single Type_decode body exceeds LuaJIT's inline budget and trace size limits. Symptom (to verify): bench/jit_trace.lua trace aborts on the big conformance message, despite no NYI bytecodes in the body.\n\nFix: when a message has \u003eN fields (threshold to find empirically; LuaJIT defaults LJ_TRACE_MAX_BC=8000), codegen splits the if-elseif chain into per-range helpers: _decode_fields_1_to_15, _decode_fields_16_to_31, etc. The top-level Type_decode dispatches to the right chunk by id range:\n\n if id \u003c= 15 then\n pos = _decode_fields_1_to_15(buf, pos, id, wt, result)\n elseif id \u003c= 31 then\n pos = _decode_fields_16_to_31(buf, pos, id, wt, result)\n ...\n end\n\nEach chunk is small enough to JIT-compile cleanly. Adds one function call per tag, but only on messages large enough to need it — small messages stay inline.\n\nThe range size and split threshold need measurement. Start with: split when message has \u003e32 fields, into chunks of 16. Measure throughput on conformance message before/after to validate the trade.","notes":"Pre-work: confirm the abort is real by running bench/jit_trace.lua against a conformance-like fixture and checking the FATAL traceerr codes. If LuaJIT compiles the 4212-line body fine, this is a non-issue.","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:14:39Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T17:14:39Z","dependency_count":0,"dependent_count":0,"comment_count":0} -{"_type":"issue","id":"tarantool-protobuf-gi0","title":"Codegen: emit wire.encode_varint directly for int32/int64/uint32/uint64 instead of typed alias","description":"runtime/pb/wire.lua aliases M.encode_int32 = encode_varint (and same for int64/uint32/uint64). Generated code calls wire.encode_int32(v) which resolves to encode_varint through two table lookups — wire.encode_int32 (one hash lookup), then the alias resolution. LuaJIT may collapse this when the trace stays hot, but each break re-incurs both.\n\nCodegen can emit wire.encode_varint directly for the four unsigned-varint scalar types (int32/int64/uint32/uint64) since they're literally encode_varint with different names. Skips one alias indirection per varint encode. Pairs naturally with kot (localize wire.* upvalues) — together they reduce the call to a direct LJ_FUNCC dispatch with no name lookup.\n\nSame applies to bool (encode_bool = encode_varint with v and 1 or 0 wrapper) and sint32/sint64 (zigzag wrapper) — codegen could inline the wrapper logic at the call site for sint, but that overlaps with h8v (FFI direct writes) which subsumes the question.","notes":"Codegen-side change in protoc-gen-tarantool. Constraint: the alias provides the typed encoder slot in TYPE_INFO that runtime-mode encoders walk — those aliases must stay. Only the generated mode=full code changes.","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:14:38Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T17:14:38Z","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"tarantool-protobuf-gi0","title":"Codegen: emit wire.encode_varint directly for int32/int64/uint32/uint64 instead of typed alias","description":"runtime/pb/wire.lua aliases M.encode_int32 = encode_varint (and same for int64/uint32/uint64). Generated code calls wire.encode_int32(v) which resolves to encode_varint through two table lookups — wire.encode_int32 (one hash lookup), then the alias resolution. LuaJIT may collapse this when the trace stays hot, but each break re-incurs both.\n\nCodegen can emit wire.encode_varint directly for the four unsigned-varint scalar types (int32/int64/uint32/uint64) since they're literally encode_varint with different names. Skips one alias indirection per varint encode. Pairs naturally with kot (localize wire.* upvalues) — together they reduce the call to a direct LJ_FUNCC dispatch with no name lookup.\n\nSame applies to bool (encode_bool = encode_varint with v and 1 or 0 wrapper) and sint32/sint64 (zigzag wrapper) — codegen could inline the wrapper logic at the call site for sint, but that overlaps with h8v (FFI direct writes) which subsumes the question.","notes":"Codegen-side change in protoc-gen-tarantool. Constraint: the alias provides the typed encoder slot in TYPE_INFO that runtime-mode encoders walk — those aliases must stay. Only the generated mode=full code changes.","status":"closed","priority":3,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:14:38Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T20:14:49Z","started_at":"2026-05-24T20:11:24Z","closed_at":"2026-05-24T20:14:49Z","close_reason":"Premise invalid. wire.encode_int32 / encode_int64 / encode_uint32 / encode_uint64 ARE literally encode_varint at module load (one assignment, same function value). kot's localize already captures wire.encode_int32 once per function, so the in-body call is a direct local read with no alias resolution. Verified: wire.encode_int32 == wire.encode_varint -\u003e true. No extra TGETS at call time. The only residual is kot creating separate locals when a function uses both encode_int32 and encode_varint (could dedupe to one closure upvalue) — marginal closure-size win, not the 'two table lookups' premise. Closing as won't-fix; the work was already done by Lua alias semantics + kot localization.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"tarantool-protobuf-5y9","title":"Decoder: int64/uint64 return Lua number when value fits in 2^53","description":"decode_int64/decode_uint64 always return ffi cdata, forcing a fresh cdata allocation per call. For values in [-2^53, 2^53) (the common case for IDs, sequence numbers, timestamps fitting in 53 bits, byte counts, etc.) a Lua number is exactly representable and skips the cdata header allocation.\n\nProposed shape: opt-in variant decoder, since changing the default breaks any caller that does type(v)=='cdata' or relies on cdata-only operators. Two API options:\n\n 1. Per-descriptor flag (desc.int64_as_number = true) wired in via a codegen option or generator flag. Generated code emits a different decoder fn.\n 2. Separate typed decoders (wire.decode_int64_n / decode_uint64_n) that callers opt into explicitly.\n\nWatch case: values exceeding 2^53 must still return cdata (with a runtime branch). The branch cost only pays off if cdata allocation cost \u003e one comparison, which it is on hot paths.","notes":"Bench impact bounded by how many int64 fields the workload has. For hello.Person, user_id (fixed64) is the only one — so impact on this fixture would be ~1-2%. Bigger win on protobuf workloads dominated by timestamps and sequence numbers (datastore RPCs, log streams).","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:14:37Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T17:14:37Z","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"tarantool-protobuf-8k2","title":"Docs: remove PLAN.md after Beads migration","description":"PLAN.md has been converted from active roadmap to duplicate design-history prose now that all actionable work is tracked in Beads. Remove the file or replace remaining references with Beads/README pointers so project state has a single durable task source of truth.","status":"closed","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T18:33:09Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T18:33:48Z","closed_at":"2026-05-17T18:33:48Z","close_reason":"Removed PLAN.md and retargeted remaining references to Beads or concrete docs.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"tarantool-protobuf-c0i","title":"C accel: generated C codec backend for mode=full","description":"Extend protoc-gen-tarantool with an optional generated-C backend. Emit C source plus Lua wrappers for each .proto package; generated functions know tag bytes, field names, defaults, oneofs, maps, proto2 required/defaults/groups/extensions, and nested message calls. The Lua API remains M.Type_encode(t) -\u003e string and M.Type_decode(bytes) -\u003e table. Goal: one Lua-\u003eC call per top-level message, no Lua table.concat, no Lua decode_tag ladder, no per-field FFI boundary.","notes":"DEFERRED per docs/c-accel.md. Spike (04c) showed S4 hand-written C had ≤15% headroom over S3 (generic C runtime, ra6) and went the wrong way at scale (S4 *slower* than S3 at 10KB and 100KB encode). Codegen complexity not justified by current numbers. REVIVAL CRITERIA: a measured real-workload shape where ra6's per-field dispatch costs ≥25% over hand-written for that shape, demonstrated with a microbenchmark, AND the affected workload is on a hot path for a real user. Likely trigger shapes: wide messages with many optionals, heavy oneof use, complex maps, deeply nested (5+ levels) hierarchies. When triggered: write 2-3 paragraphs documenting shape and numbers, re-open this issue, scope narrowly via desc.encode/desc.decode override (same mechanism as WKT). If 6 months after ra6 ships no trigger fires, close as 'not justified'. Now blocked by ra6 (so it surfaces in ready list only after ra6 implementation lands).","status":"open","priority":3,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:32:32Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T20:19:41Z","dependencies":[{"issue_id":"tarantool-protobuf-c0i","depends_on_id":"tarantool-protobuf-47e","type":"blocks","created_at":"2026-05-17T19:33:29Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-c0i","depends_on_id":"tarantool-protobuf-pf6","type":"blocks","created_at":"2026-05-17T19:33:12Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-c0i","depends_on_id":"tarantool-protobuf-ra6","type":"blocks","created_at":"2026-05-18T23:19:44Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-c0i","depends_on_id":"tarantool-protobuf-z7x","type":"blocks","created_at":"2026-05-17T19:33:29Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":4,"dependent_count":0,"comment_count":0} @@ -68,7 +69,7 @@ {"_type":"issue","id":"tarantool-protobuf-b4j","title":"Docs: migration guide from built-in protobuf to pb","description":"Tarantool ships an encode-only require('protobuf') module. Document the migration path for users currently on the built-in. Cover: API surface differences (no decode in builtin; pb has encode+decode+JSON+text+gRPC+WKT), 64-bit cdata convention parity (both use cdata, no surprise), field-name conventions (both snake_case), the rename rationale (we couldn't override 'protobuf' without breaking existing users). Concrete before/after code samples.","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:17Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T15:47:17Z","labels":["docs","release"],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"tarantool-protobuf-o8n","title":"gRPC: reference network transports (http_server, netbox, http_client_unary)","description":"Ship reference transports as separate Tarantool modules (deferred from M4). Designs in docs/specs/grpc_transports.md. Three transports: (1) pb.grpc.transport.http_server — Connect-JSON over HTTP/1.1 via tarantool/http; default external transport; works with browsers and curl without an HTTP/2 proxy. (2) pb.grpc.transport.netbox — gRPC tunneled over net.box calls; first-class in-cluster path. (3) pb.grpc.transport.http_client_unary — outbound unary only via http_client. HTTP/2 termination explicitly out of scope; the transport contract is HTTP/2-shaped so the same generated code works behind Envoy. Conformance anchor: connectrpc/conformance harness (memory: connect_conformance_suite).","status":"open","priority":3,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:14Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T15:47:14Z","labels":["grpc","transport"],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"tarantool-protobuf-2me","title":"Decide lazy-default for repeated string|bytes fields","description":"Open design question. For routing/proxy workloads that decode but don't iterate every element, repeated LEN fields could stay as unread ArrayViews until first access. Eager callers pay one extra table alloc; lazy callers skip N per-element string allocations. At 100KB Person, 26+ emails = 26+ string allocations deferrable. Decide: (a) flip default for repeated LEN, (b) gate behind a codegen mode/option, (c) leave eager as default and rely on _decode_lazy. Surface compatibility implications (callers iterate with ipairs today and expect a real array) before deciding.","status":"open","priority":3,"issue_type":"decision","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:13Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T15:47:13Z","labels":["decoder","design","perf"],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"tarantool-protobuf-0u1","title":"Decoder: decode_len-style byte-then-varint symmetry in typed decoders","description":"The single-byte peek shortcut in decode_len (wire.lua:410-419) isn't replicated in decode_int32 / decode_uint32 / decode_int64 / decode_uint64 / decode_bool / decode_enum, which always call decode_varint(). Mirror the fast path in each typed decoder so the dominant 1-byte varint case skips a function call. Expected: 5-10% on scalar-heavy decodes. Memory: wire_2byte_varint_cliff (same shape, encode side, already fixed).","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:11Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T15:47:11Z","labels":["decoder","perf","wire"],"dependencies":[{"issue_id":"tarantool-protobuf-0u1","depends_on_id":"tarantool-protobuf-0an","type":"blocks","created_at":"2026-05-17T18:47:21Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} -{"_type":"issue","id":"tarantool-protobuf-a7l","title":"Encoder: static type-elision on scalars in mode=full","description":"wire.to_uint64(v) does a type(v) dispatch in three branches (number/cdata/boolean); in mode=full codegen the field's static type is known. Emit UINT64(v) / INT64(v) directly at the call site and let LuaJIT type-guard the trace. Expected: 10-15% on scalar-heavy messages.","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:07Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T15:47:07Z","labels":["codegen","encoder","perf"],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"tarantool-protobuf-a7l","title":"Encoder: static type-elision on scalars in mode=full","description":"wire.to_uint64(v) does a type(v) dispatch in three branches (number/cdata/boolean); in mode=full codegen the field's static type is known. Emit UINT64(v) / INT64(v) directly at the call site and let LuaJIT type-guard the trace. Expected: 10-15% on scalar-heavy messages.","status":"open","priority":3,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:07Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T20:14:50Z","started_at":"2026-05-24T20:11:25Z","labels":["codegen","encoder","perf"],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"tarantool-protobuf-cma","title":"Encoder: inline zigzag_encode32 into encode_sint32 (one function call vs two)","description":"encode_sint32(v) = encode_varint(zigzag_encode32(v)) — two function calls. Inline as:\n\n local function encode_sint32(v)\n local n = bit.bxor(bit.lshift(v, 1), bit.arshift(v, 31))\n -- existing 1-byte fast path\n if n \u003e= 0 and n \u003c 0x80 then return string.char(n) end\n return encode_varint_slow(n)\n end\n\nSame for encode_sint64 (with the int64 zigzag formula).\n\nTiny win per sint encode — only matters for sint-heavy payloads. Filed for completeness; h8v makes this moot for mode=full (FFI direct writes inline the whole thing anyway). Real applicability: runtime mode encoders.","notes":"If h8v lands first, this becomes runtime-mode-only and may not be worth the duplication. Defer the call until h8v ships.","status":"open","priority":4,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:15:08Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T17:15:08Z","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"tarantool-protobuf-yvc","title":"Feature: gRPC server reflection service","description":"PLAN.md section 8 lists gRPC server reflection as a non-goal until after M5. M5 is complete, so keep this as deferred future work. Implement standard reflection once real transport/server use cases require it.","status":"open","priority":4,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:29:18Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:29:18Z","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"tarantool-protobuf-0j5","title":"Feature: Tarantool 3.x config integration for schema registration","description":"PLAN.md section 7 mentions a pb.types.\u003cname\u003e declarative config role as a possible Tarantool 3.x integration. Define the use case first, then design how protobuf schemas/descriptors are declared, loaded, validated, and made available to application roles.","status":"open","priority":4,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:29:17Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:29:17Z","dependency_count":0,"dependent_count":0,"comment_count":0} diff --git a/cmd/protoc-gen-tarantool/internal/gen/inline.go b/cmd/protoc-gen-tarantool/internal/gen/inline.go index 494a16a637e984626a283639dc603784e82ffddf..04cea05616143435ba7f7883704e1feca611251c 100644 --- a/cmd/protoc-gen-tarantool/internal/gen/inline.go +++ b/cmd/protoc-gen-tarantool/internal/gen/inline.go @@ -212,6 +212,80 @@ w.line("%s n = n + 1; out[n] = wire.encode_varint(_len)", indent) w.line("%send", indent) } +// emitPackedVarintElemDecode emits the per-element decode inside a packed +// varint loop, with 1-byte and 2-byte fast paths inlined for the common +// Lua-number-returning scalar types (int32/uint32/enum/bool). The 2-byte +// branch covers values 128..16383 — the range that drives the residual +// side-trace bridges from the packed-int32 inner loop (see ozn). For +// other types (int64/uint64/sint*/cdata returns) the helper just emits +// the existing wire.decode_ call so the call frame stays. +// +// `valExpr` is the lvalue Lua expression for the decoded value (e.g. +// `val`, `list[cnt]`). `indent` is the Lua indentation prefix. +// +// For `enum`, the value is an int32 and bypasses wire.varint_to_int32 +// in the inlined branches since 2-byte values 128..16383 fit int32 +// directly; the fallback call still wraps with varint_to_int32. +func emitPackedVarintElemDecode(w *writer, indent, st, valExpr string) { + switch st { + case "int32", "uint32": + w.line("%slocal _b = string_byte(payload, p2)", indent) + w.line("%sif _b ~= nil and _b < 0x80 then", indent) + w.line("%s %s = _b", indent, valExpr) + w.line("%s p2 = p2 + 1", indent) + w.line("%selseif _b ~= nil and p2 < lim then", indent) + w.line("%s local _b2 = string_byte(payload, p2 + 1)", indent) + w.line("%s if _b2 ~= nil and _b2 < 0x80 then", indent) + w.line("%s if _b2 == 0 then error(\"overlong varint at offset \" .. p2, 0) end", indent) + w.line("%s %s = _b - 128 + _b2 * 128", indent, valExpr) + w.line("%s p2 = p2 + 2", indent) + w.line("%s else", indent) + w.line("%s %s, p2 = wire.decode_%s(payload, p2)", indent, valExpr, st) + w.line("%s end", indent) + w.line("%selse", indent) + w.line("%s %s, p2 = wire.decode_%s(payload, p2)", indent, valExpr, st) + w.line("%send", indent) + case "bool": + // Bool values are spec-valid only as 0 or 1, always 1-byte on the wire. + // A 2-byte form would be either overlong-0 or out-of-range. Keep just + // the 1-byte fast path and let any anomaly fall through to wire.decode_bool. + w.line("%slocal _b = string_byte(payload, p2)", indent) + w.line("%sif _b ~= nil and _b < 0x80 then", indent) + w.line("%s %s = _b ~= 0", indent, valExpr) + w.line("%s p2 = p2 + 1", indent) + w.line("%selse", indent) + w.line("%s %s, p2 = wire.decode_bool(payload, p2)", indent, valExpr) + w.line("%send", indent) + default: + w.line("%s%s, p2 = wire.decode_%s(payload, p2)", indent, valExpr, st) + } +} + +// emitPackedEnumElemDecode is the enum counterpart to +// emitPackedVarintElemDecode. Enum payload is a varint reinterpreted as +// int32; for values 0..16383 the 1-byte and 2-byte fast paths bypass +// wire.varint_to_int32 since the result already fits the int32 range. +func emitPackedEnumElemDecode(w *writer, indent, valExpr string) { + w.line("%slocal _b = string_byte(payload, p2)", indent) + w.line("%sif _b ~= nil and _b < 0x80 then", indent) + w.line("%s %s = _b", indent, valExpr) + w.line("%s p2 = p2 + 1", indent) + w.line("%selseif _b ~= nil and p2 < lim then", indent) + w.line("%s local _b2 = string_byte(payload, p2 + 1)", indent) + w.line("%s if _b2 ~= nil and _b2 < 0x80 then", indent) + w.line("%s if _b2 == 0 then error(\"overlong varint at offset \" .. p2, 0) end", indent) + w.line("%s %s = _b - 128 + _b2 * 128", indent, valExpr) + w.line("%s p2 = p2 + 2", indent) + w.line("%s else", indent) + w.line("%s local _u; _u, p2 = wire.decode_varint(payload, p2)", indent) + w.line("%s %s = wire.varint_to_int32(_u)", indent, valExpr) + w.line("%s end", indent) + w.line("%selse", indent) + w.line("%s local _u; _u, p2 = wire.decode_varint(payload, p2)", indent) + w.line("%s %s = wire.varint_to_int32(_u)", indent, valExpr) + w.line("%send", indent) +} + // emitPackedVarintElem emits the per-element write inside a packed-varint // loop with an inlined 1-byte CHARS-lookup fast path matching // wire.encode_varint's hot case. Falls through to wire.encode_ for @@ -1073,6 +1147,9 @@ w.line(" %s = %s + 1; list[%s] = %s(payload)", cnt, cnt, cnt, ref) } case f.Enum != nil: // Enums are packable (proto3 default). Accept both packed and per-element. + // The packed inner loop uses the enum-aware inline varint decoder so + // 1-byte and 2-byte values bypass the wire.decode_varint + varint_to_int32 + // frames (ozn). w.line(" if wt == 2 then") w.line(" local payload") w.line(" payload, pos = wire.decode_len(buf, pos)") @@ -1080,9 +1157,9 @@ w.line(" local p2, lim = 1, #payload") w.line(" local list = %s", dst) w.line(" if list == nil then list = table_new(%s, 0); %s = list end", estExpr, dst) w.line(" while p2 <= lim do") - w.line(" local u") - w.line(" u, p2 = wire.decode_varint(payload, p2)") - w.line(" %s = %s + 1; list[%s] = wire.varint_to_int32(u)", cnt, cnt, cnt) + w.line(" local val") + emitPackedEnumElemDecode(w, " ", "val") + w.line(" %s = %s + 1; list[%s] = val", cnt, cnt, cnt) w.line(" end") w.line(" else") w.line(" local list = %s", dst) @@ -1102,7 +1179,10 @@ w.line(" local list = %s", dst) w.line(" if list == nil then list = table_new(%s, 0); %s = list end", estExpr, dst) w.line(" while p2 <= lim do") w.line(" local val") - w.line(" val, p2 = wire.decode_%s(payload, p2)", st) + // Inline 1-byte and 2-byte varint fast paths for int32/uint32/bool + // (Lua-number returns). Other types fall through to wire.decode_ + // per the helper. (ozn) + emitPackedVarintElemDecode(w, " ", st, "val") w.line(" %s = %s + 1; list[%s] = val", cnt, cnt, cnt) w.line(" end") w.line(" else") diff --git a/examples/expected/full/c_repeated/c_repeated_pb.lua b/examples/expected/full/c_repeated/c_repeated_pb.lua index 53dca64d0c09a1b40dd5486eb668126389b4d411..1fdaec54f64e8c61059410769713ceb0672e43e7 100644 --- a/examples/expected/full/c_repeated/c_repeated_pb.lua +++ b/examples/expected/full/c_repeated/c_repeated_pb.lua @@ -683,7 +683,22 @@ local list = result.packed_int32 if list == nil then list = table_new(lim, 0); result.packed_int32 = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_packed_int32 = _n_packed_int32 + 1; list[_n_packed_int32] = val end else @@ -740,7 +755,22 @@ local list = result.packed_uint32 if list == nil then list = table_new(lim, 0); result.packed_uint32 = list end while p2 <= lim do local val - val, p2 = decode_uint32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_uint32(payload, p2) + end + else + val, p2 = decode_uint32(payload, p2) + end _n_packed_uint32 = _n_packed_uint32 + 1; list[_n_packed_uint32] = val end else @@ -835,7 +865,13 @@ local list = result.packed_bool if list == nil then list = table_new(lim, 0); result.packed_bool = list end while p2 <= lim do local val - val, p2 = decode_bool(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b ~= 0 + p2 = p2 + 1 + else + val, p2 = decode_bool(payload, p2) + end _n_packed_bool = _n_packed_bool + 1; list[_n_packed_bool] = val end else @@ -854,7 +890,22 @@ local list = result.unpacked_int32 if list == nil then list = table_new(lim, 0); result.unpacked_int32 = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_unpacked_int32 = _n_unpacked_int32 + 1; list[_n_unpacked_int32] = val end else @@ -1065,7 +1116,22 @@ local list = result.packed_int32 if list == nil then list = table_new(lim, 0); result.packed_int32 = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_packed_int32 = _n_packed_int32 + 1; list[_n_packed_int32] = val end else @@ -1122,7 +1188,22 @@ local list = result.packed_uint32 if list == nil then list = table_new(lim, 0); result.packed_uint32 = list end while p2 <= lim do local val - val, p2 = decode_uint32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_uint32(payload, p2) + end + else + val, p2 = decode_uint32(payload, p2) + end _n_packed_uint32 = _n_packed_uint32 + 1; list[_n_packed_uint32] = val end else @@ -1217,7 +1298,13 @@ local list = result.packed_bool if list == nil then list = table_new(lim, 0); result.packed_bool = list end while p2 <= lim do local val - val, p2 = decode_bool(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b ~= 0 + p2 = p2 + 1 + else + val, p2 = decode_bool(payload, p2) + end _n_packed_bool = _n_packed_bool + 1; list[_n_packed_bool] = val end else @@ -1236,7 +1323,22 @@ local list = result.unpacked_int32 if list == nil then list = table_new(lim, 0); result.unpacked_int32 = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_unpacked_int32 = _n_unpacked_int32 + 1; list[_n_unpacked_int32] = val end else diff --git a/examples/expected/full/hello/hello_pb.lua b/examples/expected/full/hello/hello_pb.lua index 1b6978e0da1f5a546ca653a38a3c41c6bbc3c6bc..f8aa91a5e34f64525d05271a13873b6c2a0ef4d0 100644 --- a/examples/expected/full/hello/hello_pb.lua +++ b/examples/expected/full/hello/hello_pb.lua @@ -2155,7 +2155,22 @@ local list = result.lucky_numbers if list == nil then list = table_new(lim, 0); result.lucky_numbers = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_lucky_numbers = _n_lucky_numbers + 1; list[_n_lucky_numbers] = val end else @@ -2423,7 +2438,22 @@ local list = result.lucky_numbers if list == nil then list = table_new(lim, 0); result.lucky_numbers = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_lucky_numbers = _n_lucky_numbers + 1; list[_n_lucky_numbers] = val end else diff --git a/examples/expected/full/proto2_basic/proto2_basic_pb.lua b/examples/expected/full/proto2_basic/proto2_basic_pb.lua index e8e02266430764e47e0ec9eb9eca1babfcc3fc51..147bbdb56ce1fe4609084d5cc9429ea7aca54bdc 100644 --- a/examples/expected/full/proto2_basic/proto2_basic_pb.lua +++ b/examples/expected/full/proto2_basic/proto2_basic_pb.lua @@ -816,7 +816,22 @@ local list = result.packed_default if list == nil then list = table_new(lim, 0); result.packed_default = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_packed_default = _n_packed_default + 1; list[_n_packed_default] = val end else @@ -835,7 +850,22 @@ local list = result.explicitly_packed if list == nil then list = table_new(lim, 0); result.explicitly_packed = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_explicitly_packed = _n_explicitly_packed + 1; list[_n_explicitly_packed] = val end else @@ -854,7 +884,22 @@ local list = result.explicitly_unpacked if list == nil then list = table_new(lim, 0); result.explicitly_unpacked = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_explicitly_unpacked = _n_explicitly_unpacked + 1; list[_n_explicitly_unpacked] = val end else @@ -942,7 +987,22 @@ local list = result.packed_default if list == nil then list = table_new(lim, 0); result.packed_default = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_packed_default = _n_packed_default + 1; list[_n_packed_default] = val end else @@ -961,7 +1021,22 @@ local list = result.explicitly_packed if list == nil then list = table_new(lim, 0); result.explicitly_packed = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_explicitly_packed = _n_explicitly_packed + 1; list[_n_explicitly_packed] = val end else @@ -980,7 +1055,22 @@ local list = result.explicitly_unpacked if list == nil then list = table_new(lim, 0); result.explicitly_unpacked = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_explicitly_unpacked = _n_explicitly_unpacked + 1; list[_n_explicitly_unpacked] = val end else @@ -2227,7 +2317,22 @@ local list = result.lucky_numbers if list == nil then list = table_new(lim, 0); result.lucky_numbers = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_lucky_numbers = _n_lucky_numbers + 1; list[_n_lucky_numbers] = val end else @@ -2409,7 +2514,22 @@ local list = result.lucky_numbers if list == nil then list = table_new(lim, 0); result.lucky_numbers = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_lucky_numbers = _n_lucky_numbers + 1; list[_n_lucky_numbers] = val end else diff --git a/examples/expected/full/protobuf_test_messages/proto2/test_messages_proto2_pb.lua b/examples/expected/full/protobuf_test_messages/proto2/test_messages_proto2_pb.lua index ee61c9f87b848dcee1e285ffdbe2e2a94604722a..5c1d2bfbe35ffbe8946d8f1ce6d99891248a3479 100644 --- a/examples/expected/full/protobuf_test_messages/proto2/test_messages_proto2_pb.lua +++ b/examples/expected/full/protobuf_test_messages/proto2/test_messages_proto2_pb.lua @@ -3065,7 +3065,22 @@ local list = result.repeated_int32 if list == nil then list = table_new(lim, 0); result.repeated_int32 = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_repeated_int32 = _n_repeated_int32 + 1; list[_n_repeated_int32] = val end else @@ -3103,7 +3118,22 @@ local list = result.repeated_uint32 if list == nil then list = table_new(lim, 0); result.repeated_uint32 = list end while p2 <= lim do local val - val, p2 = decode_uint32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_uint32(payload, p2) + end + else + val, p2 = decode_uint32(payload, p2) + end _n_repeated_uint32 = _n_repeated_uint32 + 1; list[_n_repeated_uint32] = val end else @@ -3293,7 +3323,13 @@ local list = result.repeated_bool if list == nil then list = table_new(lim, 0); result.repeated_bool = list end while p2 <= lim do local val - val, p2 = decode_bool(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b ~= 0 + p2 = p2 + 1 + else + val, p2 = decode_bool(payload, p2) + end _n_repeated_bool = _n_repeated_bool + 1; list[_n_repeated_bool] = val end else @@ -3389,9 +3425,26 @@ local p2, lim = 1, #payload local list = result.repeated_nested_enum if list == nil then list = table_new(lim, 0); result.repeated_nested_enum = list end while p2 <= lim do - local u - u, p2 = decode_varint(payload, p2) - _n_repeated_nested_enum = _n_repeated_nested_enum + 1; list[_n_repeated_nested_enum] = varint_to_int32(u) + local val + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + _n_repeated_nested_enum = _n_repeated_nested_enum + 1; list[_n_repeated_nested_enum] = val end else local list = result.repeated_nested_enum @@ -3408,9 +3461,26 @@ local p2, lim = 1, #payload local list = result.repeated_foreign_enum if list == nil then list = table_new(lim, 0); result.repeated_foreign_enum = list end while p2 <= lim do - local u - u, p2 = decode_varint(payload, p2) - _n_repeated_foreign_enum = _n_repeated_foreign_enum + 1; list[_n_repeated_foreign_enum] = varint_to_int32(u) + local val + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + _n_repeated_foreign_enum = _n_repeated_foreign_enum + 1; list[_n_repeated_foreign_enum] = val end else local list = result.repeated_foreign_enum @@ -3496,7 +3566,22 @@ local list = result.packed_int32 if list == nil then list = table_new(lim, 0); result.packed_int32 = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_packed_int32 = _n_packed_int32 + 1; list[_n_packed_int32] = val end else @@ -3534,7 +3619,22 @@ local list = result.packed_uint32 if list == nil then list = table_new(lim, 0); result.packed_uint32 = list end while p2 <= lim do local val - val, p2 = decode_uint32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_uint32(payload, p2) + end + else + val, p2 = decode_uint32(payload, p2) + end _n_packed_uint32 = _n_packed_uint32 + 1; list[_n_packed_uint32] = val end else @@ -3724,7 +3824,13 @@ local list = result.packed_bool if list == nil then list = table_new(lim, 0); result.packed_bool = list end while p2 <= lim do local val - val, p2 = decode_bool(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b ~= 0 + p2 = p2 + 1 + else + val, p2 = decode_bool(payload, p2) + end _n_packed_bool = _n_packed_bool + 1; list[_n_packed_bool] = val end else @@ -3742,9 +3848,26 @@ local p2, lim = 1, #payload local list = result.packed_nested_enum if list == nil then list = table_new(lim, 0); result.packed_nested_enum = list end while p2 <= lim do - local u - u, p2 = decode_varint(payload, p2) - _n_packed_nested_enum = _n_packed_nested_enum + 1; list[_n_packed_nested_enum] = varint_to_int32(u) + local val + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + _n_packed_nested_enum = _n_packed_nested_enum + 1; list[_n_packed_nested_enum] = val end else local list = result.packed_nested_enum @@ -3762,7 +3885,22 @@ local list = result.unpacked_int32 if list == nil then list = table_new(lim, 0); result.unpacked_int32 = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_unpacked_int32 = _n_unpacked_int32 + 1; list[_n_unpacked_int32] = val end else @@ -3800,7 +3938,22 @@ local list = result.unpacked_uint32 if list == nil then list = table_new(lim, 0); result.unpacked_uint32 = list end while p2 <= lim do local val - val, p2 = decode_uint32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_uint32(payload, p2) + end + else + val, p2 = decode_uint32(payload, p2) + end _n_unpacked_uint32 = _n_unpacked_uint32 + 1; list[_n_unpacked_uint32] = val end else @@ -3990,7 +4143,13 @@ local list = result.unpacked_bool if list == nil then list = table_new(lim, 0); result.unpacked_bool = list end while p2 <= lim do local val - val, p2 = decode_bool(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b ~= 0 + p2 = p2 + 1 + else + val, p2 = decode_bool(payload, p2) + end _n_unpacked_bool = _n_unpacked_bool + 1; list[_n_unpacked_bool] = val end else @@ -4008,9 +4167,26 @@ local p2, lim = 1, #payload local list = result.unpacked_nested_enum if list == nil then list = table_new(lim, 0); result.unpacked_nested_enum = list end while p2 <= lim do - local u - u, p2 = decode_varint(payload, p2) - _n_unpacked_nested_enum = _n_unpacked_nested_enum + 1; list[_n_unpacked_nested_enum] = varint_to_int32(u) + local val + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + _n_unpacked_nested_enum = _n_unpacked_nested_enum + 1; list[_n_unpacked_nested_enum] = val end else local list = result.unpacked_nested_enum @@ -5177,7 +5353,22 @@ local list = result.repeated_int32 if list == nil then list = table_new(lim, 0); result.repeated_int32 = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_repeated_int32 = _n_repeated_int32 + 1; list[_n_repeated_int32] = val end else @@ -5215,7 +5406,22 @@ local list = result.repeated_uint32 if list == nil then list = table_new(lim, 0); result.repeated_uint32 = list end while p2 <= lim do local val - val, p2 = decode_uint32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_uint32(payload, p2) + end + else + val, p2 = decode_uint32(payload, p2) + end _n_repeated_uint32 = _n_repeated_uint32 + 1; list[_n_repeated_uint32] = val end else @@ -5405,7 +5611,13 @@ local list = result.repeated_bool if list == nil then list = table_new(lim, 0); result.repeated_bool = list end while p2 <= lim do local val - val, p2 = decode_bool(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b ~= 0 + p2 = p2 + 1 + else + val, p2 = decode_bool(payload, p2) + end _n_repeated_bool = _n_repeated_bool + 1; list[_n_repeated_bool] = val end else @@ -5499,9 +5711,26 @@ local p2, lim = 1, #payload local list = result.repeated_nested_enum if list == nil then list = table_new(lim, 0); result.repeated_nested_enum = list end while p2 <= lim do - local u - u, p2 = decode_varint(payload, p2) - _n_repeated_nested_enum = _n_repeated_nested_enum + 1; list[_n_repeated_nested_enum] = varint_to_int32(u) + local val + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + _n_repeated_nested_enum = _n_repeated_nested_enum + 1; list[_n_repeated_nested_enum] = val end else local list = result.repeated_nested_enum @@ -5518,9 +5747,26 @@ local p2, lim = 1, #payload local list = result.repeated_foreign_enum if list == nil then list = table_new(lim, 0); result.repeated_foreign_enum = list end while p2 <= lim do - local u - u, p2 = decode_varint(payload, p2) - _n_repeated_foreign_enum = _n_repeated_foreign_enum + 1; list[_n_repeated_foreign_enum] = varint_to_int32(u) + local val + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + _n_repeated_foreign_enum = _n_repeated_foreign_enum + 1; list[_n_repeated_foreign_enum] = val end else local list = result.repeated_foreign_enum @@ -5602,7 +5848,22 @@ local list = result.packed_int32 if list == nil then list = table_new(lim, 0); result.packed_int32 = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_packed_int32 = _n_packed_int32 + 1; list[_n_packed_int32] = val end else @@ -5640,7 +5901,22 @@ local list = result.packed_uint32 if list == nil then list = table_new(lim, 0); result.packed_uint32 = list end while p2 <= lim do local val - val, p2 = decode_uint32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_uint32(payload, p2) + end + else + val, p2 = decode_uint32(payload, p2) + end _n_packed_uint32 = _n_packed_uint32 + 1; list[_n_packed_uint32] = val end else @@ -5830,7 +6106,13 @@ local list = result.packed_bool if list == nil then list = table_new(lim, 0); result.packed_bool = list end while p2 <= lim do local val - val, p2 = decode_bool(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b ~= 0 + p2 = p2 + 1 + else + val, p2 = decode_bool(payload, p2) + end _n_packed_bool = _n_packed_bool + 1; list[_n_packed_bool] = val end else @@ -5848,9 +6130,26 @@ local p2, lim = 1, #payload local list = result.packed_nested_enum if list == nil then list = table_new(lim, 0); result.packed_nested_enum = list end while p2 <= lim do - local u - u, p2 = decode_varint(payload, p2) - _n_packed_nested_enum = _n_packed_nested_enum + 1; list[_n_packed_nested_enum] = varint_to_int32(u) + local val + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + _n_packed_nested_enum = _n_packed_nested_enum + 1; list[_n_packed_nested_enum] = val end else local list = result.packed_nested_enum @@ -5868,7 +6167,22 @@ local list = result.unpacked_int32 if list == nil then list = table_new(lim, 0); result.unpacked_int32 = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_unpacked_int32 = _n_unpacked_int32 + 1; list[_n_unpacked_int32] = val end else @@ -5906,7 +6220,22 @@ local list = result.unpacked_uint32 if list == nil then list = table_new(lim, 0); result.unpacked_uint32 = list end while p2 <= lim do local val - val, p2 = decode_uint32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_uint32(payload, p2) + end + else + val, p2 = decode_uint32(payload, p2) + end _n_unpacked_uint32 = _n_unpacked_uint32 + 1; list[_n_unpacked_uint32] = val end else @@ -6096,7 +6425,13 @@ local list = result.unpacked_bool if list == nil then list = table_new(lim, 0); result.unpacked_bool = list end while p2 <= lim do local val - val, p2 = decode_bool(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b ~= 0 + p2 = p2 + 1 + else + val, p2 = decode_bool(payload, p2) + end _n_unpacked_bool = _n_unpacked_bool + 1; list[_n_unpacked_bool] = val end else @@ -6114,9 +6449,26 @@ local p2, lim = 1, #payload local list = result.unpacked_nested_enum if list == nil then list = table_new(lim, 0); result.unpacked_nested_enum = list end while p2 <= lim do - local u - u, p2 = decode_varint(payload, p2) - _n_unpacked_nested_enum = _n_unpacked_nested_enum + 1; list[_n_unpacked_nested_enum] = varint_to_int32(u) + local val + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + _n_unpacked_nested_enum = _n_unpacked_nested_enum + 1; list[_n_unpacked_nested_enum] = val end else local list = result.unpacked_nested_enum @@ -8335,7 +8687,22 @@ local list = result.repeated_int32 if list == nil then list = table_new(lim, 0); result.repeated_int32 = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_repeated_int32 = _n_repeated_int32 + 1; list[_n_repeated_int32] = val end else @@ -8470,7 +8837,22 @@ local list = result.repeated_int32 if list == nil then list = table_new(lim, 0); result.repeated_int32 = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_repeated_int32 = _n_repeated_int32 + 1; list[_n_repeated_int32] = val end else diff --git a/examples/expected/full/protobuf_test_messages/proto3/test_messages_proto3_pb.lua b/examples/expected/full/protobuf_test_messages/proto3/test_messages_proto3_pb.lua index 34b4a9ad259bce0d822bcec86707e1d91cf4d851..ea2f6c04947810c5e85071fd7c4e934302846ec5 100644 --- a/examples/expected/full/protobuf_test_messages/proto3/test_messages_proto3_pb.lua +++ b/examples/expected/full/protobuf_test_messages/proto3/test_messages_proto3_pb.lua @@ -3228,7 +3228,22 @@ local list = result.repeated_int32 if list == nil then list = table_new(lim, 0); result.repeated_int32 = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_repeated_int32 = _n_repeated_int32 + 1; list[_n_repeated_int32] = val end else @@ -3266,7 +3281,22 @@ local list = result.repeated_uint32 if list == nil then list = table_new(lim, 0); result.repeated_uint32 = list end while p2 <= lim do local val - val, p2 = decode_uint32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_uint32(payload, p2) + end + else + val, p2 = decode_uint32(payload, p2) + end _n_repeated_uint32 = _n_repeated_uint32 + 1; list[_n_repeated_uint32] = val end else @@ -3456,7 +3486,13 @@ local list = result.repeated_bool if list == nil then list = table_new(lim, 0); result.repeated_bool = list end while p2 <= lim do local val - val, p2 = decode_bool(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b ~= 0 + p2 = p2 + 1 + else + val, p2 = decode_bool(payload, p2) + end _n_repeated_bool = _n_repeated_bool + 1; list[_n_repeated_bool] = val end else @@ -3552,9 +3588,26 @@ local p2, lim = 1, #payload local list = result.repeated_nested_enum if list == nil then list = table_new(lim, 0); result.repeated_nested_enum = list end while p2 <= lim do - local u - u, p2 = decode_varint(payload, p2) - _n_repeated_nested_enum = _n_repeated_nested_enum + 1; list[_n_repeated_nested_enum] = varint_to_int32(u) + local val + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + _n_repeated_nested_enum = _n_repeated_nested_enum + 1; list[_n_repeated_nested_enum] = val end else local list = result.repeated_nested_enum @@ -3571,9 +3624,26 @@ local p2, lim = 1, #payload local list = result.repeated_foreign_enum if list == nil then list = table_new(lim, 0); result.repeated_foreign_enum = list end while p2 <= lim do - local u - u, p2 = decode_varint(payload, p2) - _n_repeated_foreign_enum = _n_repeated_foreign_enum + 1; list[_n_repeated_foreign_enum] = varint_to_int32(u) + local val + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + _n_repeated_foreign_enum = _n_repeated_foreign_enum + 1; list[_n_repeated_foreign_enum] = val end else local list = result.repeated_foreign_enum @@ -3659,7 +3729,22 @@ local list = result.packed_int32 if list == nil then list = table_new(lim, 0); result.packed_int32 = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_packed_int32 = _n_packed_int32 + 1; list[_n_packed_int32] = val end else @@ -3697,7 +3782,22 @@ local list = result.packed_uint32 if list == nil then list = table_new(lim, 0); result.packed_uint32 = list end while p2 <= lim do local val - val, p2 = decode_uint32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_uint32(payload, p2) + end + else + val, p2 = decode_uint32(payload, p2) + end _n_packed_uint32 = _n_packed_uint32 + 1; list[_n_packed_uint32] = val end else @@ -3887,7 +3987,13 @@ local list = result.packed_bool if list == nil then list = table_new(lim, 0); result.packed_bool = list end while p2 <= lim do local val - val, p2 = decode_bool(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b ~= 0 + p2 = p2 + 1 + else + val, p2 = decode_bool(payload, p2) + end _n_packed_bool = _n_packed_bool + 1; list[_n_packed_bool] = val end else @@ -3905,9 +4011,26 @@ local p2, lim = 1, #payload local list = result.packed_nested_enum if list == nil then list = table_new(lim, 0); result.packed_nested_enum = list end while p2 <= lim do - local u - u, p2 = decode_varint(payload, p2) - _n_packed_nested_enum = _n_packed_nested_enum + 1; list[_n_packed_nested_enum] = varint_to_int32(u) + local val + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + _n_packed_nested_enum = _n_packed_nested_enum + 1; list[_n_packed_nested_enum] = val end else local list = result.packed_nested_enum @@ -3925,7 +4048,22 @@ local list = result.unpacked_int32 if list == nil then list = table_new(lim, 0); result.unpacked_int32 = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_unpacked_int32 = _n_unpacked_int32 + 1; list[_n_unpacked_int32] = val end else @@ -3963,7 +4101,22 @@ local list = result.unpacked_uint32 if list == nil then list = table_new(lim, 0); result.unpacked_uint32 = list end while p2 <= lim do local val - val, p2 = decode_uint32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_uint32(payload, p2) + end + else + val, p2 = decode_uint32(payload, p2) + end _n_unpacked_uint32 = _n_unpacked_uint32 + 1; list[_n_unpacked_uint32] = val end else @@ -4153,7 +4306,13 @@ local list = result.unpacked_bool if list == nil then list = table_new(lim, 0); result.unpacked_bool = list end while p2 <= lim do local val - val, p2 = decode_bool(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b ~= 0 + p2 = p2 + 1 + else + val, p2 = decode_bool(payload, p2) + end _n_unpacked_bool = _n_unpacked_bool + 1; list[_n_unpacked_bool] = val end else @@ -4171,9 +4330,26 @@ local p2, lim = 1, #payload local list = result.unpacked_nested_enum if list == nil then list = table_new(lim, 0); result.unpacked_nested_enum = list end while p2 <= lim do - local u - u, p2 = decode_varint(payload, p2) - _n_unpacked_nested_enum = _n_unpacked_nested_enum + 1; list[_n_unpacked_nested_enum] = varint_to_int32(u) + local val + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + _n_unpacked_nested_enum = _n_unpacked_nested_enum + 1; list[_n_unpacked_nested_enum] = val end else local list = result.unpacked_nested_enum @@ -5357,7 +5533,22 @@ local list = result.repeated_int32 if list == nil then list = table_new(lim, 0); result.repeated_int32 = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_repeated_int32 = _n_repeated_int32 + 1; list[_n_repeated_int32] = val end else @@ -5395,7 +5586,22 @@ local list = result.repeated_uint32 if list == nil then list = table_new(lim, 0); result.repeated_uint32 = list end while p2 <= lim do local val - val, p2 = decode_uint32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_uint32(payload, p2) + end + else + val, p2 = decode_uint32(payload, p2) + end _n_repeated_uint32 = _n_repeated_uint32 + 1; list[_n_repeated_uint32] = val end else @@ -5585,7 +5791,13 @@ local list = result.repeated_bool if list == nil then list = table_new(lim, 0); result.repeated_bool = list end while p2 <= lim do local val - val, p2 = decode_bool(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b ~= 0 + p2 = p2 + 1 + else + val, p2 = decode_bool(payload, p2) + end _n_repeated_bool = _n_repeated_bool + 1; list[_n_repeated_bool] = val end else @@ -5679,9 +5891,26 @@ local p2, lim = 1, #payload local list = result.repeated_nested_enum if list == nil then list = table_new(lim, 0); result.repeated_nested_enum = list end while p2 <= lim do - local u - u, p2 = decode_varint(payload, p2) - _n_repeated_nested_enum = _n_repeated_nested_enum + 1; list[_n_repeated_nested_enum] = varint_to_int32(u) + local val + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + _n_repeated_nested_enum = _n_repeated_nested_enum + 1; list[_n_repeated_nested_enum] = val end else local list = result.repeated_nested_enum @@ -5698,9 +5927,26 @@ local p2, lim = 1, #payload local list = result.repeated_foreign_enum if list == nil then list = table_new(lim, 0); result.repeated_foreign_enum = list end while p2 <= lim do - local u - u, p2 = decode_varint(payload, p2) - _n_repeated_foreign_enum = _n_repeated_foreign_enum + 1; list[_n_repeated_foreign_enum] = varint_to_int32(u) + local val + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + _n_repeated_foreign_enum = _n_repeated_foreign_enum + 1; list[_n_repeated_foreign_enum] = val end else local list = result.repeated_foreign_enum @@ -5782,7 +6028,22 @@ local list = result.packed_int32 if list == nil then list = table_new(lim, 0); result.packed_int32 = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_packed_int32 = _n_packed_int32 + 1; list[_n_packed_int32] = val end else @@ -5820,7 +6081,22 @@ local list = result.packed_uint32 if list == nil then list = table_new(lim, 0); result.packed_uint32 = list end while p2 <= lim do local val - val, p2 = decode_uint32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_uint32(payload, p2) + end + else + val, p2 = decode_uint32(payload, p2) + end _n_packed_uint32 = _n_packed_uint32 + 1; list[_n_packed_uint32] = val end else @@ -6010,7 +6286,13 @@ local list = result.packed_bool if list == nil then list = table_new(lim, 0); result.packed_bool = list end while p2 <= lim do local val - val, p2 = decode_bool(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b ~= 0 + p2 = p2 + 1 + else + val, p2 = decode_bool(payload, p2) + end _n_packed_bool = _n_packed_bool + 1; list[_n_packed_bool] = val end else @@ -6028,9 +6310,26 @@ local p2, lim = 1, #payload local list = result.packed_nested_enum if list == nil then list = table_new(lim, 0); result.packed_nested_enum = list end while p2 <= lim do - local u - u, p2 = decode_varint(payload, p2) - _n_packed_nested_enum = _n_packed_nested_enum + 1; list[_n_packed_nested_enum] = varint_to_int32(u) + local val + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + _n_packed_nested_enum = _n_packed_nested_enum + 1; list[_n_packed_nested_enum] = val end else local list = result.packed_nested_enum @@ -6048,7 +6347,22 @@ local list = result.unpacked_int32 if list == nil then list = table_new(lim, 0); result.unpacked_int32 = list end while p2 <= lim do local val - val, p2 = decode_int32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_int32(payload, p2) + end + else + val, p2 = decode_int32(payload, p2) + end _n_unpacked_int32 = _n_unpacked_int32 + 1; list[_n_unpacked_int32] = val end else @@ -6086,7 +6400,22 @@ local list = result.unpacked_uint32 if list == nil then list = table_new(lim, 0); result.unpacked_uint32 = list end while p2 <= lim do local val - val, p2 = decode_uint32(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + val, p2 = decode_uint32(payload, p2) + end + else + val, p2 = decode_uint32(payload, p2) + end _n_unpacked_uint32 = _n_unpacked_uint32 + 1; list[_n_unpacked_uint32] = val end else @@ -6276,7 +6605,13 @@ local list = result.unpacked_bool if list == nil then list = table_new(lim, 0); result.unpacked_bool = list end while p2 <= lim do local val - val, p2 = decode_bool(payload, p2) + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b ~= 0 + p2 = p2 + 1 + else + val, p2 = decode_bool(payload, p2) + end _n_unpacked_bool = _n_unpacked_bool + 1; list[_n_unpacked_bool] = val end else @@ -6294,9 +6629,26 @@ local p2, lim = 1, #payload local list = result.unpacked_nested_enum if list == nil then list = table_new(lim, 0); result.unpacked_nested_enum = list end while p2 <= lim do - local u - u, p2 = decode_varint(payload, p2) - _n_unpacked_nested_enum = _n_unpacked_nested_enum + 1; list[_n_unpacked_nested_enum] = varint_to_int32(u) + local val + local _b = string_byte(payload, p2) + if _b ~= nil and _b < 0x80 then + val = _b + p2 = p2 + 1 + elseif _b ~= nil and p2 < lim then + local _b2 = string_byte(payload, p2 + 1) + if _b2 ~= nil and _b2 < 0x80 then + if _b2 == 0 then error("overlong varint at offset " .. p2, 0) end + val = _b - 128 + _b2 * 128 + p2 = p2 + 2 + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + else + local _u; _u, p2 = decode_varint(payload, p2) + val = varint_to_int32(_u) + end + _n_unpacked_nested_enum = _n_unpacked_nested_enum + 1; list[_n_unpacked_nested_enum] = val end else local list = result.unpacked_nested_enum