Skip to content

BIP 181, 182, 183: BIPs for Utreexo#1923

Draft
kcalvinalvin wants to merge 4 commits into
bitcoin:masterfrom
kcalvinalvin:2025-08-10-utreexo-bips
Draft

BIP 181, 182, 183: BIPs for Utreexo#1923
kcalvinalvin wants to merge 4 commits into
bitcoin:masterfrom
kcalvinalvin:2025-08-10-utreexo-bips

Conversation

@kcalvinalvin
Copy link
Copy Markdown
Contributor

These are the 3 BIPs that describe Utreexo, a consensus-compatible (non-soft fork) way to send and verify transactions without storing the full UTXO set.

The 3 BIPs are for:

  1. The specification of the Utreexo accumulator.
  2. The specification of Bitcoin block and tx validation using the Utreexo accumulator.
  3. The peer to peer networking changes required to enable Utreexo nodes.

Mailing list post: https://groups.google.com/g/bitcoindev/c/W1lxBraKG_E

@kcalvinalvin kcalvinalvin force-pushed the 2025-08-10-utreexo-bips branch 2 times, most recently from 9b3eafb to a94f643 Compare August 10, 2025 07:09
Copy link
Copy Markdown

@jmoik jmoik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some typos

Comment thread utreexo-p2p-bip.md Outdated
Comment thread utreexo-p2p-bip.md Outdated
Comment thread utreexo-p2p-bip.md Outdated
Comment thread utreexo-p2p-bip.md Outdated
Comment thread utreexo-p2p-bip.md Outdated
Comment thread utreexo-p2p-bip.md Outdated
Comment thread utreexo-p2p-bip.md Outdated
Comment thread utreexo-p2p-bip.md
Copy link
Copy Markdown
Member

@jonatack jonatack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for proposing these drafts. They already look quite complete with respect to the editorial requirements (BIPs 2 and 3). I've done a cursory first pass. No immediate conceptual feedback. A few editorial comments follow; feel free to ignore them during conceptual review until they are applicable.

Comment thread utreexo-p2p-bip.md Outdated
Comment thread utreexo-validation-bip.md Outdated
Comment thread utreexo-accumulator-bip.md Outdated
Comment thread utreexo-accumulator-bip.md Outdated
Comment thread utreexo-validation-bip.md Outdated
Comment thread utreexo-accumulator-bip.md Outdated
Comment thread utreexo-accumulator-bip.md Outdated
Comment thread utreexo-accumulator-bip.md Outdated
@kcalvinalvin kcalvinalvin force-pushed the 2025-08-10-utreexo-bips branch 2 times, most recently from cb2993c to d1d0342 Compare August 12, 2025 06:23
@petertodd
Copy link
Copy Markdown
Contributor

You need to justify why you're using SHA-512/256 rather than SHA-256, like the rest of the Bitcoin protocol. Right now you just link to a paper from 2011. But that paper is out of date now that hardware support for SHA-256 has become common.

@ghost
Copy link
Copy Markdown

ghost commented Aug 12, 2025

I strongly recommend replacing SHA-256 with SHAKE256 (from the SHA-3 standard) for the following reasons:

1. Security Advantages

  • 🔒 Provides built-in protection against length-extension attacks
  • 📏 Offers flexible output lengths (supports 128-bit and 256-bit security levels)
  • ⚙️ Based on Keccak sponge construction (NIST FIPS 202 standard)
  • 🌐 Aligns with post-quantum cryptography standards

2. Comparative Analysis: SHA-256 vs SHAKE256

Characteristic SHA-256 SHAKE256
Algorithm Family SHA-2 SHA-3 (Keccak)
Output Flexibility Fixed 256-bit Arbitrary length
Security Properties Vulnerable to length-extension Resistant to length-extension
Internal Structure Merkle-Damgård Sponge function
Standardization NIST FIPS 180-4 NIST FIPS 202

3. Functional Example

Input: Bitcoin

SHAKE256 (512-bit output):
6beb0661ba1fa7289bf359fbb81550bd9641cf5abc62a14d466c421c8a86e528e027632ec0e7ceb994650566f3c8258af2240333b6d0e9186766fd2c1ebb763a

SHAKE256 (256-bit output):
6beb0661ba1fa7289bf359fbb81550bd9641cf5abc62a14d466c421c8a86e528

4. Implementation Benefits

  • ✅ Maintains 256-bit output compatibility where needed
  • ✅ Future-proofs against emerging cryptographic vulnerabilities
  • ✅ Reduces potential attack vectors through improved design
  • ✅ Supports Bitcoin's security evolution while maintaining performance

5. Technical Reference

For detailed cryptographic differences:
Cryptographic Comparison: SHA-2 vs SHA-3

@kcalvinalvin
Copy link
Copy Markdown
Contributor Author

You need to justify why you're using SHA-512/256 rather than SHA-256, like the rest of the Bitcoin protocol. Right now you just link to a paper from 2011. But that paper is out of date now that hardware support for SHA-256 has become common.

Sure we can update the accumulator BIP with benchmarks for SHA512/256 vs SHA256.

But could you link to the aforementioned justifications for the other parts of the Bitcoin protocol that use SHA512?

@kcalvinalvin
Copy link
Copy Markdown
Contributor Author

kcalvinalvin commented Aug 18, 2025

I strongly recommend replacing SHA-256 with SHAKE256 (from the SHA-3 standard) for the following reasons:

SHAKE256 is not used in Bitcoin and introduces a new hash which increases the trust-assumption. We do not want to do this.

@bitcoin bitcoin deleted a comment Aug 18, 2025
@bitcoin bitcoin deleted a comment from kcalvinalvin Aug 18, 2025
@ghost

This comment was marked as off-topic.

@bitcoin bitcoin deleted a comment from kcalvinalvin Aug 18, 2025
@bitcoin bitcoin deleted a comment Aug 18, 2025
@jonatack
Copy link
Copy Markdown
Member

Some friendly moderation to keep the discussion focused on technical review -- thanks.

@kcalvinalvin
Copy link
Copy Markdown
Contributor Author

kcalvinalvin commented Aug 18, 2025

The reliance of Bitcoin on SHA-2—a legacy hash function designed by the National Security Agency (NSA)—introduces non-trivial security risks, particularly when considering the often-dismissed threat posed by quantum adversaries.

SHA256 and SHA512 are quantum resistent.

Migrating to SHAKE256 (a variant of SHA-3) would represent a meaningful improvement, though such a change merely delays the inevitable: Bitcoin must eventually transition to a quantum-resistant cryptographic framework. When this occurs—and it will, regardless of opposition—SHA-2, along with ECDSA private keys, public keys, and signatures, will become obsolete.
See: Lenght extension attack (Bitcoin is vulnerable because it's using SHA-256)

Ok but this has nothing to do with this BIP.

@murchandamus
Copy link
Copy Markdown
Member

@1BitcoinBoWP1FZ4xwTNkq6XksKidmgYYw, please cut out the LLM generated comments. If any of us were interested in seeing an LLM’s prediction of what might be said about a topic, we could prompt one ourselves.

@petertodd
Copy link
Copy Markdown
Contributor

petertodd commented Aug 18, 2025 via email

@kcalvinalvin
Copy link
Copy Markdown
Contributor Author

On Mon, Aug 18, 2025 at 04:06:51AM -0700, Calvin Kim wrote: kcalvinalvin left a comment (bitcoin/bips#1923) > You need to justify why you're using SHA-512/256 rather than SHA-256, like the rest of the Bitcoin protocol. Right now you just link to a paper from 2011. But that paper is out of date now that hardware support for SHA-256 has become common. Sure we can update the accumulator BIP with benchmarks for SHA512/256 vs SHA256. But could you link to the aforementioned justifications for the other parts of the Bitcoin protocol that use SHA512?
No part of the Bitcoin consensus protocol uses SHA512.

Ok but you've stated in your previous comment "You need to justify why you're using SHA-512/256 rather than SHA-256, like the rest of the Bitcoin protocol". Would be very helpful to see what type of justifications the other protocols have made.

Second, I don't think it matters if SHA512 wasn't used in the Bitcoin consensus protocol. SHA512 is used in BIP32 and the argument that SHA512 is safe for generating private keys but not safe for Bitcoin consensus isn't sound.

I think our original justification (better performance with SHA512/256) mentioned in the BIP is sound. Happy to provide the benchmarks, they're being worked on at the moment.

@ghost

This comment was marked as abuse.

@kcalvinalvin

This comment was marked as off-topic.

@kcalvinalvin

This comment was marked as off-topic.

Comment thread utreexo-validation-bip.md Outdated
Comment on lines +63 to +66
| Name | Type | Description |
| ----------------- | ------------------------ | ----------------------------------------- |
| Utreexo_Tag_V1 | 64 byte array | The version tag to be prepended to the leafhash. |
| Utreexo_Tag_V1 | 64 byte array | The version tag to be prepended to the leafhash. |
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For clarification, is the Utreexo_Tag_V1 really used twice in preimage to the hash?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My guess would be that this duplication is unintended.

Suggested change
| Name | Type | Description |
| ----------------- | ------------------------ | ----------------------------------------- |
| Utreexo_Tag_V1 | 64 byte array | The version tag to be prepended to the leafhash. |
| Utreexo_Tag_V1 | 64 byte array | The version tag to be prepended to the leafhash. |
| Name | Type | Description |
| ----------------- | ------------------------ | ----------------------------------------- |
| Utreexo_Tag_V1 | 64 byte array | The version tag to be prepended to the leafhash. |

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh no the duplication is intended.

Since we use SHA512/256 as the hash function, each chunk is 128 bytes. Since the version tag is only 64 bytes, we need two of them.

@petertodd
Copy link
Copy Markdown
Contributor

On Mon, Aug 18, 2025 at 04:06:51AM -0700, Calvin Kim wrote: kcalvinalvin left a comment (bitcoin/bips#1923) > You need to justify why you're using SHA-512/256 rather than SHA-256, like the rest of the Bitcoin protocol. Right now you just link to a paper from 2011. But that paper is out of date now that hardware support for SHA-256 has become common. Sure we can update the accumulator BIP with benchmarks for SHA512/256 vs SHA256. But could you link to the aforementioned justifications for the other parts of the Bitcoin protocol that use SHA512?
No part of the Bitcoin consensus protocol uses SHA512.

Ok but you've stated in your previous comment "You need to justify why you're using SHA-512/256 rather than SHA-256, like the rest of the Bitcoin protocol". Would be very helpful to see what type of justifications the other protocols have made.

Second, I don't think it matters if SHA512 wasn't used in the Bitcoin consensus protocol. SHA512 is used in BIP32 and the argument that SHA512 is safe for generating private keys but not safe for Bitcoin consensus isn't sound.

I think our original justification (better performance with SHA512/256) mentioned in the BIP is sound. Happy to provide the benchmarks, they're being worked on at the moment.

The question is 1) why are we added one new dependency to consensus implementations, and 2) is this actually a performance increase, given that dedicated SHA256 hardware is becoming common?

Length-extension attacks are not relevant for this use-case as we are only committing to public data.

Copy link
Copy Markdown
Member

@murchandamus murchandamus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update.

I gave the diff a quick skim:

Comment thread bip-0182.md
The UTXO proof has 2 elements: the accumulator proof and the leaf data. The
leaf data provides the necessary UTXO data for block validation that would be
stored locally for non-Utreexo nodes. Non-Utreexo nodes store this data (under "chainstate/" for Bitcoin Core)
but since utreexo nodes don't this data, it must be provided.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing a word.

Suggested change
but since utreexo nodes don't this data, it must be provided.
but since utreexo nodes don't <missing word> this data, it must be provided.

Comment thread bip-0182.md

**Why use the Utreexo accumulator to keep track of UTXOs instead of a key-value database like leveldb?**

There's two main advantages to using the Utreexo accumulator instead of a key-value database like leveldb:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
There's two main advantages to using the Utreexo accumulator instead of a key-value database like leveldb:
There are two main advantages to using the Utreexo accumulator instead of a key-value database like leveldb:

Comment thread bip-0182.md
There's two main advantages to using the Utreexo accumulator instead of a key-value database like leveldb:

1. Puts a cap on the UTXO set growth.
3. Performance gains with the elimination of random reads/writes.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renders right of course, but still:

Suggested change
3. Performance gains with the elimination of random reads/writes.
2. Performance gains with the elimination of random reads/writes.

Comment thread bip-0182.md
Comment on lines +334 to +335
Currently, the UTXO set size is $O(log(N))$ where $N$ is the number of UTXOs.
By utilizing the Utreexo accumulator, we're able to cap the UTXO set growth at $O(log_2(N))$.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that you don’t store the UTXO set, but an accumulator that commits to the UTXO set, perhaps these two sentences should be amended?

@murchandamus
Copy link
Copy Markdown
Member

It would perhaps be good if one or two other people gave it also a read, but either way, it seems pretty complete to me. What’s the status on your end? Do you still have planned work, or are waiting for people to finish reviews?

Copy link
Copy Markdown
Contributor

@vostrnad vostrnad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would perhaps be good if one or two other people gave it also a read

Here's my read. I've suggested mainly formatting and capitalization changes, but at least two suggestions are quite important: the distinction between "varint" and "compact size", and the broken cross-BIP links.

Comment thread bip-0181.md
The Utreexo accumulator is based on an append-only Merkle tree design introduced in [^1],
which provides logarithmic-sized inclusion proofs. Utreexo extends this design to support dynamic updates,
specifically enabling deletions from the set—a requirement for tracking UTXO spends in Bitcoin.
To accommodate this, Utreexo changes the storage requirement from the accumulator design in [^1] to $O(log_2(N))$,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Specifying the logarithm base is redundant in big O notation, as changing the base is equivalent to multiplying by a constant factor.

Suggested change
To accommodate this, Utreexo changes the storage requirement from the accumulator design in [^1] to $O(log_2(N))$,
To accommodate this, Utreexo changes the storage requirement from the accumulator design in [^1] to $O(log(N))$,

Comment thread bip-0181.md
a 16-element tree ($2^4$), a 4-element tree ($2^2$), and a 1-element tree ($2^0$), with gaps at the 8-element ($2^3$)
and 2-element ($2^1$) positions.

Each of the hashes in the forest can be referred by an integer label. This labeling is a convention we find easiest
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Each of the hashes in the forest can be referred by an integer label. This labeling is a convention we find easiest
Each of the hashes in the forest can be referred to by an integer label. This labeling is a convention we find easiest

Comment thread bip-0181.md

**treerows(numleaves):** Returns the minimum number of bits required to represent `numleaves - 1`. This corresponds to the height of the largest tree in the forest. Returns `0` if `numleaves` is `0`.

The reason for taking the minimum number of bits required for `numleaves-1` and not `numleaves` is because when `numleaves` is a power of two, we'd get an off-by-one error.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to have consistent spacing around the minus sign, there's both numleaves - 1 and numleaves-1. Same goes for the equals sign below.

Comment thread bip-0181.md
The calculate roots algorithm is defined as `CalculateRoots(numleaves, []hash, proof) -> calculated_roots`:

- Check if length of `proof.targets` is equal to the length of `[]hash`. Return early if they're not equal.
- map `proof.targets` to their hash.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- map `proof.targets` to their hash.
- Map `proof.targets` to their hash.

Comment thread bip-0181.md
- Map parent hash to the parent position.
- Return calculated_roots

The algorithm implemented in python:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The algorithm implemented in python:
The algorithm implemented in Python:

Comment thread bip-0183.md
| length of the proof hashes | varint | The length of the proof hashes |
| proof hashes | vector of 32 byte hashes | The vector of the requested Utreexo summaries |
| length of the leafdatas | varint | The length of the leafdatas |
| leafdatas | vector of compact leafdatas | The preimage of the leafdatas referenced in the bitcoin transaction. MUST be in the order of the referenced inputs. Unconfirmed inputs do not have a corresponding leaf data. See compact leaf data for details |
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| leafdatas | vector of compact leafdatas | The preimage of the leafdatas referenced in the bitcoin transaction. MUST be in the order of the referenced inputs. Unconfirmed inputs do not have a corresponding leaf data. See compact leaf data for details |
| leafdatas | vector of compact leafdatas | The preimage of the leafdatas referenced in the Bitcoin transaction. MUST be in the order of the referenced inputs. Unconfirmed inputs do not have a corresponding leaf data. See compact leaf data for details |

Comment thread bip-0183.md

#### MSG_UTREEXO_ROOT

`MSG_UTREEXO_ROOT` is the utreexo accumulator state at a given height with a proof to a utreexo accumulator of the utreexo roots.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
`MSG_UTREEXO_ROOT` is the utreexo accumulator state at a given height with a proof to a utreexo accumulator of the utreexo roots.
`MSG_UTREEXO_ROOT` is the Utreexo accumulator state at a given height with a proof to a Utreexo accumulator of the Utreexo roots.

There are also several instances of lowercase "utreexo" in the table below.

Comment thread bip-0183.md
Comment on lines +403 to +405
For example, a computer could divide the task of validating 800,000 blocks into 100 tasks of 8,000 blocks each: blocks 1 through 800, 800 through 1600, 1600 through 2400, and so on.

In order start the 1600 through 2400 IBD task, however, the node should know what the state of the utxo set is at block 1600, so that it can validate and modify the accumulator.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
For example, a computer could divide the task of validating 800,000 blocks into 100 tasks of 8,000 blocks each: blocks 1 through 800, 800 through 1600, 1600 through 2400, and so on.
In order start the 1600 through 2400 IBD task, however, the node should know what the state of the utxo set is at block 1600, so that it can validate and modify the accumulator.
For example, a computer could divide the task of validating 800,000 blocks into 100 tasks of 8,000 blocks each: blocks 1 through 800, 801 through 1600, 1601 through 2400, and so on.
In order start the 1601 through 2400 IBD task, however, the node should know what the state of the UTXO set is at block 1600, so that it can validate and modify the accumulator.

Comment thread bip-0183.md

These hints are statements of fact that are hard-coded into the program itself, and if they are false all bets are off about the program.

Archive nodes create a forest of Linkup hints, so that they can prove, with respect to the Linkup forest roots in a node performing IBD, what their binary has claimed the utxo accumulator state to be at any block height.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Archive nodes create a forest of Linkup hints, so that they can prove, with respect to the Linkup forest roots in a node performing IBD, what their binary has claimed the utxo accumulator state to be at any block height.
Archive nodes create a forest of Linkup hints, so that they can prove, with respect to the Linkup forest roots in a node performing IBD, what their binary has claimed the UTXO accumulator state to be at any block height.

Comment thread bip-0183.md

#### MSG_GET_UTREEXO_ROOT

`MSG_GET_UTREEXO_ROOT` is used to request a utreexo accumulator state at a given height.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
`MSG_GET_UTREEXO_ROOT` is used to request a utreexo accumulator state at a given height.
`MSG_GET_UTREEXO_ROOT` is used to request a Utreexo accumulator state at a given height.

@luisschwab
Copy link
Copy Markdown

Some test vectors are in order as well.

Comment thread bip-0183.md

![Utreexo TX relay multiple Utreexo proof hash vectors](bip-0183/utreexo-tx-relay-with-multiple-proofhash-inventory-vectors.png)

It's possible to have an inv message with multiple txs as well.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
It's possible to have an inv message with multiple txs as well.
It's possible to have an `inv` message with multiple transactions as well.

Comment thread bip-0183.md

### Block Propagation

Legacy block propagation without Compact Blocks comprises of three steps:
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Legacy block propagation without Compact Blocks comprises of three steps:
Legacy block propagation without Compact Blocks is comprised of three steps:

Comment thread bip-0183.md
Comment on lines +150 to +151
1. Node A sends an inv message or a block header to Node B.
2. Node B makes a getdata request for the block.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. Node A sends an inv message or a block header to Node B.
2. Node B makes a getdata request for the block.
1. Node A sends an `inv` message or a block header to Node B.
2. Node B makes a `getdata` request for the block.

Comment thread bip-0183.md
2. Node B makes a getdata request for the block.
3. Node A sends the block data to Node B.

Below image illustrates how a non-Utreexo node would relay blocks without using Compact Blocks.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Below image illustrates how a non-Utreexo node would relay blocks without using Compact Blocks.
The image below illustrates how a non-Utreexo node would relay blocks without using Compact Blocks.

Comment thread bip-0183.md
Comment on lines +160 to +162
1. Node A sends an inv message or a block header to Node B.
2. Node B makes a getdata request for the block.
3. Node B makes a getutreexoproof request for the block.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. Node A sends an inv message or a block header to Node B.
2. Node B makes a getdata request for the block.
3. Node B makes a getutreexoproof request for the block.
1. Node A sends an `inv` message or a block header to Node B.
2. Node B makes a `getdata` request for the block.
3. Node B makes a `getutreexoproof` request for the block.

Comment thread bip-0183.md

### Commitment scheme for TTL messages

We choose an arbitrary height `X` and go through each of `TTL info` in all the the `Utreexo TTL` values up until that height.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
We choose an arbitrary height `X` and go through each of `TTL info` in all the the `Utreexo TTL` values up until that height.
We choose an arbitrary height `X` and go through each of `TTL Info`s in all of the `Utreexo TTL` values up until that height.

Comment thread bip-0183.md

We choose an arbitrary height `X` and go through each of `TTL info` in all the the `Utreexo TTL` values up until that height.

If the TTL in the `TTL info` is greater than the [numleaves](bip-0181.md#Definitions) value of the Utreexo accumulator at the chosen height `X`, we reset the `death position` and the `TTL` values to their default of 0.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If the TTL in the `TTL info` is greater than the [numleaves](bip-0181.md#Definitions) value of the Utreexo accumulator at the chosen height `X`, we reset the `death position` and the `TTL` values to their default of 0.
If the TTL in the `TTL Info` is greater than the [numleaves](bip-0181.md#Definitions) value of the Utreexo accumulator at the chosen height `X`, we reset the `death position` and the `TTL` values to their default of 0.

Comment thread bip-0183.md

**Why is there a separate NODE_UTREEXO_ARCHIVE service bit from the NODE_UTREEXO service bit?**

For archive nodes, we wanted the ability for a node to keep just the historical Utreexo proofs since the historical blocks can be served by any archival nodes.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
For archive nodes, we wanted the ability for a node to keep just the historical Utreexo proofs since the historical blocks can be served by any archival nodes.
For archival nodes, we wanted the ability for a node to keep just the historical Utreexo proofs since the historical blocks can be served by any archival node.

Comment thread bip-0183.md

We decided to communicate the positions in the Utreexo merkle forest by inventory vectors instead of a separate message to avoid an extra round trip during the transaction propagation.

As mentioned above in [Transaction Relay](#transaction-relay), non-Utreexo nodes propagate a transaction in these 3 steps:
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
As mentioned above in [Transaction Relay](#transaction-relay), non-Utreexo nodes propagate a transaction in these 3 steps:
As mentioned above in [Transaction Relay](#transaction-relay), non-Utreexo nodes propagate a transaction in 3 steps:

Comment thread bip-0183.md
Comment on lines +534 to +535
2. Send a message to get the positions in the Utreexo merkle forest for the transaction.
3. Receive the positions in the Utreexo merkle forest.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
2. Send a message to get the positions in the Utreexo merkle forest for the transaction.
3. Receive the positions in the Utreexo merkle forest.
2. Send a message to get the positions in the Utreexo Merkle forest for the transaction.
3. Receive the positions in the Utreexo Merkle forest.

Comment thread bip-0183.md
|----------------------------|-------------------------|------------------------------------------------------------------------------------------------------------------|
| blockhash | 32 byte vector | The hash of the block that the requested utreexo root message is for |

### New Inventory Types
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For all inventory types: be explicit about what needs to be provided and in what format (eg: blockhash, leaf positions, etc..).

Comment thread bip-0181.md
of the UTXO set. Since it can grow indefinitely, bounded only by block size, it represents a
long-term scalability concern.

Utreexo is a dynamic accumulator that enables the UTXO set to be represented in just a few kilobytes,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coming from https://github.com/cryptography-camp/workbook
Screenshot 2025-10-02 at 10 43 22

The defined accumulator in BIP 181 is positive because it supports membership proofs.

Suggested change
Utreexo is a dynamic accumulator that enables the UTXO set to be represented in just a few kilobytes,
Utreexo is a dynamic positive accumulator that enables the UTXO set to be represented in just a few kilobytes

Comment thread bip-0181.md
Comment on lines +53 to +55
The Utreexo accumulator is based on an append-only Merkle tree design introduced in [^1],
which provides logarithmic-sized inclusion proofs. Utreexo extends this design to support dynamic updates,
specifically enabling deletions from the set—a requirement for tracking UTXO spends in Bitcoin.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Parsing through the linked paper it claimed that the accumulator defined there is sound and strong.

With the extension here to make that accumulator dynamic, I suppose it is still correct, sound and strong?
Perhaps link to some resource on where that was explicitly studied

@ismaelsadeeq
Copy link
Copy Markdown
Member

I think our original justification (better performance with SHA512/256) mentioned in the BIP is sound. Happy to provide the benchmarks, they're being worked on at the moment.

This point should also be added in the rationale along with the benchmarks when available

@murchandamus murchandamus added the PR Author action required Needs updates, has unaddressed review comments, or is otherwise waiting for PR author label Oct 6, 2025
Copy link
Copy Markdown
Member

@murchandamus murchandamus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like there is still work in progress here. Please let me know when the review feedback has been resolved.

@jonatack
Copy link
Copy Markdown
Member

@kcalvinalvin there doesn't seem to be any response since late August -- do you plan to address the review and update here?

@kcalvinalvin
Copy link
Copy Markdown
Contributor Author

@kcalvinalvin there doesn't seem to be any response since late August -- do you plan to address the review and update here?

Yes. There have been multiple protocol changes to BIP183 since then and I've been busy with the implementations for those changes. I'll get to the reviews and update the BIP.

@kcalvinalvin kcalvinalvin marked this pull request as draft November 30, 2025 17:34
Comment thread bip-0183.md
`MSG_UTREEXO_PROOF` is all the data required for a CSN or archive node using the Utreexo accumulators to validate a Bitcoin block.

Its `cmdString` for P2PV1 is `uproof`.
Its [BIP324 P2PV2](https://github.com/bitcoin/bips/blob/master/bip-0324.mediawiki#user-content-v2_Bitcoin_P2P_message_structure) message type is `29`.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these values should be reserved in bip 324 before being added to other bips, to help avoid conflicts. (At worst, perhaps bip-324 could be updated in this PR)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point! I was also told ages ago that we should make a PR to Core reserving the service bits we are using. @kcalvinalvin I think we should also do that?

Comment thread bip-0183.md
To save that bandwidth, we only send a Compact Leaf Data, that contains all missing information for the receiving peer to reconstruct the full leaf data.
A compact leaf data is defined as:

| Field | type | Description |
Copy link
Copy Markdown

@rustaceanrob rustaceanrob Dec 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The leaf data is quite similar to CTxUndo in Bitcoin Core (de-serialized as a Coin) . I suggest this data be sent as a separate message, as non-Utreexo clients could also make use of this data, and verify its integrity to varying degrees:

  • The SwiftSync protocol requires this data to perform full validation and enable parallel block downloads. The integrity of the data is verified at a pre-determined block height. A client cannot be mislead into accepting an invalid state.
  • BIP-157 clients may audit the construction of a compact block filter when two peers disagree on the hash of a filter for a particular block. This assumes the client is not eclipsed. I realized after the fact this is not accurate. One could change the block undo data for trivially spendable scripts and still produce a valid block and undo data combination.
  • BIP-157 clients do not have a way to reasonably estimate fee rates without a third party oracle. This data would allow for block-level fee rate analysis by light clients, given the client audits the undo data indeed corresponds to the block filter they have received.
  • There is no protocol-native way for clients to scan for silent payments. This would allow a "light" client to download both the block undo data and compact block filter to check for potential payments by computing partial secrets locally.

If the current serialization format from Bitcoin Core is used, nodes serving this data can simply read the bytes from disk and send them directly over the wire without a de-serialization step. The trade-off here is of course the TTLs. To compensate, along with the length of the message, a header section may be included that specifies a height filter for coins that are assumed to already be in the client's cache. So a client requests "I would like all coins spent in this block, but I have the coins that were created in the last N blocks already," however this would required interpreting the undo-data from disk.

When I checked in October, undo data was approximately 90GB on disk. Is this even significant compared to the proofs?

@murchandamus
Copy link
Copy Markdown
Member

Hey, I just wanted to point out that we started collecting the one-byte identifiers used for messages on p2pv2 in an auxiliary file for BIP 324 and the identifiers you propose to use for Utreexo were added to the tracking there: #2092.

Should you should change your identifiers going forth, please remember to update them there as well.

Copy link
Copy Markdown
Member

@murchandamus murchandamus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, just thought I’d check on y’all. How is it coming along? When you make your updates, please also tweak the preamble to comply with BIP 3 which was deployed meanwhile.

Comment thread bip-0183.md
Comment on lines +2 to +13
BIP: 183
Layer: Peer Services
Title: Utreexo - Peer Services
Author: Tadge Dryja <[email protected]>
Calvin Kim <[email protected]>
Davidson Souza <[email protected]>
Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0183
Status: Draft
Type: Standards Track
Created: 2024-08-08
License: BSD-3-Clause
Requires: 181, 182
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
BIP: 183
Layer: Peer Services
Title: Utreexo - Peer Services
Author: Tadge Dryja <[email protected]>
Calvin Kim <[email protected]>
Davidson Souza <[email protected]>
Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0183
Status: Draft
Type: Standards Track
Created: 2024-08-08
License: BSD-3-Clause
Requires: 181, 182
BIP: 183
Layer: Peer Services
Title: Utreexo - Peer Services
Authors: Tadge Dryja <[email protected]>
Calvin Kim <[email protected]>
Davidson Souza <[email protected]>
Status: Draft
Type: Specification
Assigned: 2025-08-29
License: BSD-3-Clause
Requires: 181, 182

Comment thread README.mediawiki
| Peer Services
| Utreexo Accumulator Specification
| Tadge Dryja, Calvin Kim, Davidson Souza
| Standard
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| Standard
| Specification

Comment thread README.mediawiki
| Peer Services
| Utreexo - Transaction and block validation
| Tadge Dryja, Calvin Kim, Davidson Souza
| Standard
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| Standard
| Specification

Comment thread README.mediawiki
| Peer Services
| Utreexo - Peer Services
| Tadge Dryja, Calvin Kim, Davidson Souza
| Standard
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| Standard
| Specification

Comment thread bip-0181.md
Comment on lines +2 to +13
BIP: 181
Layer: Peer Services
Title: Utreexo Accumulator Specification
Author: Tadge Dryja <[email protected]>
Calvin Kim <[email protected]>
Davidson Souza <[email protected]>
Comments-Summary: No comments yet.
Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0181
Status: Draft
Type: Standards Track
Created: 2025-06-18
License: BSD-3-Clause
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
BIP: 181
Layer: Peer Services
Title: Utreexo Accumulator Specification
Author: Tadge Dryja <[email protected]>
Calvin Kim <[email protected]>
Davidson Souza <[email protected]>
Comments-Summary: No comments yet.
Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0181
Status: Draft
Type: Standards Track
Created: 2025-06-18
License: BSD-3-Clause
BIP: 181
Layer: Peer Services
Title: Utreexo Accumulator Specification
Authors: Tadge Dryja <[email protected]>
Calvin Kim <[email protected]>
Davidson Souza <[email protected]>
Status: Draft
Type: Specification
Assigned: 2025-08-29
License: BSD-3-Clause

Comment thread bip-0182.md
Comment on lines +2 to +13
BIP: 182
Layer: Peer Services
Title: Utreexo - Transaction and block validation
Author: Tadge Dryja <[email protected]>
Calvin Kim <[email protected]>
Davidson Souza <[email protected]>
Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0182
Status: Draft
Type: Standards Track
Created: 2023-10-01
License: BSD-3-Clause
Requires: 181
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
BIP: 182
Layer: Peer Services
Title: Utreexo - Transaction and block validation
Author: Tadge Dryja <[email protected]>
Calvin Kim <[email protected]>
Davidson Souza <[email protected]>
Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0182
Status: Draft
Type: Standards Track
Created: 2023-10-01
License: BSD-3-Clause
Requires: 181
BIP: 182
Layer: Peer Services
Title: Utreexo - Transaction and block validation
Authors: Tadge Dryja <[email protected]>
Calvin Kim <[email protected]>
Davidson Souza <[email protected]>
Status: Draft
Type: Specification
Assigned: 2025-08-29
License: BSD-3-Clause
Requires: 181

@jonatack
Copy link
Copy Markdown
Member

@kcalvinalvin there doesn't seem to be any response since late August -- do you plan to address the review and update here?

Yes. There have been multiple protocol changes to BIP183 since then and I've been busy with the implementations for those changes. I'll get to the reviews and update the BIP.

A few more months have gone by since #1923 (comment) -- are you still working on this?

jaoleal added a commit to jaoleal/FlorestaBA that referenced this pull request Apr 16, 2026
Mentio Utreexo BIPs from kcalvinalvin/bips since they are not yet merged into bitcoin/bips. Add a note marking them as work in progress and mention the PR  on bitcoin/bips#1923.
jaoleal added a commit to jaoleal/FlorestaBA that referenced this pull request Apr 16, 2026
Mentio Utreexo BIPs from kcalvinalvin/bips since they are not yet merged into bitcoin/bips. Add a note marking them as work in progress and mention the PR  on bitcoin/bips#1923.
Davidson-Souza added a commit to getfloresta/Floresta that referenced this pull request Apr 20, 2026
66f29cd docs: acknowledge utreexo BIP drafts and note WIP status (jaoleal)

Pull request description:

  ### What is the purpose of this pull request?

  - [ ] Bug fix
  - [X] Documentation update
  - [ ] New feature
  - [ ] Test
  - [ ] Other: <!-- Please describe it -->

  ### Which crates are being modified?

  - [ ] floresta-chain
  - [ ] floresta-cli
  - [ ] floresta-common
  - [ ] floresta-compact-filters
  - [ ] floresta-electrum
  - [ ] floresta-watch-only
  - [ ] floresta-wire
  - [ ] floresta
  - [ ] florestad
  - [x] Other: Readme
  ### Description and Notes

  To avoid discussions and misunderstandings about utreexo itself the bips were written and are, at the time Im writing this PR, being merged.

  This mentions bitcoin/bips#1923 to be enough to close #296

  draft until bips are merged.

  ### Contributor Checklist

  <!-- Feel free to remove this section once you've confirmed all items -->

  - [X] I've followed the [contribution guidelines](https://github.com/vinteumorg/Floresta/blob/master/CONTRIBUTING.md)
  - [X] I've verified one of the following:
    - Ran `just pcc` (recommended but slower)
    - Ran `just lint-features '-- -D warnings' && cargo test --release`
    - Confirmed CI passed on my fork
  - [X] I've linked any related issue(s) in the sections above

  Finally, you are encouraged to sign all your commits (it proves authorship and guards against tampering—see [How (and why) to sign Git commits](https://withblue.ink/2020/05/17/how-and-why-to-sign-git-commits.html) and [GitHub's guide to signing commits](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits)).

ACKs for top commit:
  moisesPompilio:
    ACK 66f29cd
  JoseSK999:
    ACK 66f29cd; links are working
  luisschwab:
    ACK 66f29cd

Tree-SHA512: ba7d34eac1235bfbd8bfc7720b40dcb15838c9ad404bfd91c079f6e1b8d174c07bd3ef50812d32b41665c5de1fc4ee293e81c471ac3d07d7e102a0f0d6664ead
@jonatack
Copy link
Copy Markdown
Member

Update, just ran into Tadge and the spec is being reworked and updated for the validation and p2p spec, maybe less change for the accumulator spec.

Comment thread bip-0182.md
## References

[^1]: https://groups.google.com/g/bitcoindev/c/qyId8Yto45M
[^2]: https://delvingbitcoin.org/t/great-consensus-cleanup-revival/710
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: i think it's preferable to link to BIP 54 directly nowadays

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

New BIP PR Author action required Needs updates, has unaddressed review comments, or is otherwise waiting for PR author

Projects

None yet

Development

Successfully merging this pull request may close these issues.