Popping transaction bundles

[OlivierBBB]

The zkTracer must have the ability to pop multiple transactions at once, that is, transactions that belong to a given bundle.

One solution is to keep our current StackedSet/StackedList mechanisms, to start tracing at traceStartBundle and and to trigger the pop/commit (or merge?) operations at traceEndBundle. This solution isn't favoured by @fab-10. IIRC, and following explanations by @ahamlat, the zkTracer would be the only user of these new methods.

Alternatively we can achieve the same result in traceStartTransaction and trigger the pop/commit (or merge?) operations at traceEndTransaction when a bundle starts / when it ends.

Notes.

• the issue will become mostly obsolete with the limitless prover (for everything except for precompiles)
• tracing a bundle that becomes untraceable after several transactions were already traced is time consuming and could be problematic

[fab-10]

Let's start reviewing a proposal that does not changes the operation tracer, since this bundle feature only make sense when building a block, and not when processing an existing block.

We can leverage the fact that Besu will think in terms of tx groups (aka bundles) so there will always be only a sequence of bundles, even a single tx will be represented as a bundle, so the current bundle-less use case will be a sequencer of bundles with only one tx.

The ZkTracer is not required to thinks in terms of bundles, but only be able to persist the pending changes when told, and not after each tx like now.

The proposal could follow this approach where we only need 2 methods: pop and merge

• pop acts like the current popTransaction but could undo everything was done (cleans the pending state) since the last merge
• merge persists the current pending state, and immediately starts a new empty pending state

At the beginning the ZkTracer just starts with an empty pending state, like it was just merged with an empty state.

At this point the sequencer will be able to call merge after all the txs in the bundle are selected, or call pop asap one tx in the bundle fails.

[letypequividelespoubelles]

Having two method pop and merge would be trivial to implement for us. At the moment, what we are doing is, merge the nth transactions operation when we enter the n+1 transaction

[ahamlat]

@fab-10 Could you elaborate on the statement below, I'm not sure to understand

Let's start reviewing a proposal that does not changes the operation tracer, since this bundle feature only make sense when building a block, and not when processing an existing block.

Currently, we do line counting with zkTracer (implementation of the OperationTracer) during block building (zktracing when each transaction is executed and line counting as a post processing selector). We can call again the zkTracer during RPC requests on the conflation endpoint to generate all the conflated traces of a range of blocks, by re-executing the blocks with zktracer.

[fab-10]

@fab-10 Could you elaborate on the statement below, I'm not sure to understand

Let's start reviewing a proposal that does not changes the operation tracer, since this bundle feature only make sense when building a block, and not when processing an existing block.

Currently, we do line counting with zkTracer (implementation of the OperationTracer) during block building (zktracing when each transaction is executed and line counting as a post processing selector). We can call again the zkTracer during RPC requests on the conflation endpoint to generate all the conflated traces of a range of blocks, by re-executing the blocks with zktracer.

The subdivision of txs in bundles is only present when building a block, because after the block is built the bundle information is lost, and you only see a sequence of txs after the block is built, so when tracing an existing block, you there are no bundle to take care of, and you can simply execute the tx one at time.