Skip to content

Commit

Permalink
add solution.md
Browse files Browse the repository at this point in the history
  • Loading branch information
@humblenginr
humblenginr committed Apr 29, 2024
1 parent 8523ef6 commit b145360
Show file tree
Hide file tree
Showing 2 changed files with 93 additions and 245 deletions.
194 changes: 93 additions & 101 deletions SOLUTION.md
View file
Original file line number Diff line number Diff line change
@@ -1,110 +1,102 @@
# Mining Simulation of a Bitcoin Block

## Brainstorming
## Design Approach
Our approach to designing the block construction program involves several key concepts aimed at creating a valid Bitcoin block:

So basically we have to do the following things:
1. Validate the transactions and assign fee/size to each one of them and order them
1. Write a parser that parses the transactions and puts them in a data strucutre that I can make use of
2. Write a minimal Script interpreter
2. Create and serialize the coinbase transaction
3. Mine the block
1. Create the candidate block header (which is just the file)
2. Add the txids by using the ordered fee/size list
3. Find the nonce by iteratively hashing the file with difference nonces
### 1. Validating and Selecting Transactions
We first validate transactions by ensuring the correctness of various attributes such as pubkey, address, input/output sums, and signature scripts. We focus on implementing signature validations for P2PKH, P2WPKH, and P2TR scripts. Transactions are then selected based on their fee/weight ratio to optimize block space usage.

Programming language to use: Golang
### 2. Creating a Candidate Block
Once validated transactions are selected, we construct a candidate block with a valid coinbase transaction. This includes generating the coinbase transaction structure and adding the witness commitment if witness data is present. Additionally, we construct the block header with appropriate values such as bits, prevBlockHash, merkle root, time, and blockversion.

## Transaction verification
### 3. Mining the Block
After creating the candidate block, we search for a nonce that satisfies the current difficulty target. This process involves iteratively changing the nonce value, hashing the block header, and checking if the resulting hash meets the difficulty criteria.

### Parsing a transaction

- [x] Parse a transaction into a data structure
## Implementation Details

### Validation

There are different types of transactions:
1. p2pkh (pay to public key hash) [x]
2. p2sh (pay to script hash) []
3. p2wpkh (pay to script hash) [x]
4. p2wsh (pay to witness script hash)
5. p2tr [x]

What about multisigs?
First we will only take these things into account. If a transaction any other type than this, then we will log it and see what it is.

Previous output is included in the transaction itself.
**Rules**

Sourced from [verify.cpp](https://github.com/bitcoin/bitcoin/blob/master/src/consensus/tx_verify.cpp),
[transactions.html](https://developer.bitcoin.org/devguide/transactions.html)

1. Check all the inputs are present and valid (we don't have to check this)
2. Check for negative or overflow input values
3. Tally transaction fees (it should not be negative)
4. Script validation
*We will assume that all the inputs are valid UTXOs, if prevout is not given, then that means we are making use of a transaction in the given list

To validate a transaction, first we have to check if all the inputs are valid, and then we have to tally the transaction fees, then we have to identify what kind of script
it is, and then validate it accordingly. Let us first just validate the P2PKH

### Writing an interpreter for Script
Do we really need to write an interpreter for the Script language?
No. We don't need to write an interpreter for the Script, we just have to programmatically validate the script and signature

### Serialization
We can use this resource https://learnmeabitcoin.com/technical/transaction/. I had one problem though. Eventhough I did everything right, I was not able to match my txid hash with
my filename. After looking at it, the issue was that I had to _reverse the transaction hash order in the transaction inputs_. My reasoning towards why we need to do this is because, in Bitcoin, the convention is to use the Natural Byte Order (little endian) when dealing within raw bitcoin data, whereas we use Reverse Byte Order (Big Endian) in Block explorers or RPC calls to bitcoin-core. In our case, the transactions are in JSON form, which means the transaction hash would be in Reverse Byte Order. But the specification in https://learnmeabitcoin.com/technical/transaction/ says that the transaction hash in the input should be in Natural Byte Order. Therefore I had to reverse it in order to get it working.

### Verifying Signature
How do we verify the signature. We basically have to do what the OP_CHECKSIG opcode does in the Script language.
We need the Signature, the public key and the transaction hash in order to verify the signature. We have to verify the signature
using the ECDSA algorithm.We cannot use any bitcoin related libraries.
Out of the required things, we have the Signature and the public key. We need to identify how to calculate the transaction hash.
Following things have to be done:
1. Identify how to correctly serialize the transactions in bitcoin
2. Identify the right library to use for ECDSA verification

I think [this](https://learn.saylor.org/mod/book/view.php?id=36340&chapterid=18915#:~:text=Hints%3A,bytes%2C%20or%208b%20in%20hex) is how we
need to seralize a transaction.

https://btcinformation.org/en/developer-reference#raw-transaction-format - serialization for non-segwit transactions
https://github.com/bitcoin/bips/blob/master/bip-0144.mediawiki - serialization for segiwit transactions

When we serialize it and double hash it using SHA256, we have to get the transaction ID. This is how we can verify that the serialization is right.
So what we need for verifying the signature is a single SHA256 hash of the serialized transaction.

What is this DER encoding with signatures?

The process of verification consists of the following steps: (taken from https://wiki.bitcoinsv.io/index.php/OP_CHECKSIG#:~:text=OP_CHECKSIG%20is%20an%20opcode%20that,signature%20check%20passes%20or%20fails.)
1. Check that the signature is encoded in the correct format - <DER Sgignature><Hashtyype>
2. Check that the public key is encoded in the correct format - both compressed and uncompressed are accepted
3. We serialize the transaction bytes using 'sighash' based on the sighash type - https://wiki.bitcoinsv.io/index.php/OP_CHECKSIG#:~:text=OP_CHECKSIG%20is%20an%20opcode%20that,signature%20check%20passes%20or%20fails.


### Validation Rules
What are the validations we have to perform is an important question to ask.
INPUT:
1. Verify pubkey address
2. Verify pubkey_asm with pubkeyscript(?)
3. Verify signature

4. Verify (sum of outputs <= sum of inputs)

We will start with these validations, and run the tests, and then based on the results we can add more rules.

### Assigning fee/size
Here we basically have to calculate the transaction fees for the given transaction, and then store them in a map that has transaction
id as the key and the (fee/size) as the value.
What do we keep as the size? Serialized transaction size(with or without witness?)

Let us first mine a block and see if our valid txns are actually valid
## Coinbase transaction

## Mining a block
We need to do the following things:
1. Select a list of transactions to put in the block
2. Create a candidate block
3. Create coinbase transaction
4. Calculate the Merkle root
5. Find the nonce

#### Validating P2PKH Scripts
```pseudo
For each P2PKH transaction:
Parse public key and signature.
Verify hash type encoding.
Parse public key using the `secp` package.
Ensure signature is in proper DER format and parse it.
Calculate signature hash (SIGHASH).
Verify signature using ECDSA algorithm.
```

#### Validating P2WPKH Scripts
```pseudo
For each P2WPKH transaction:
Parse signature and public key from witness array.
Calculate signature hash (SIGHASH) using BIP143.
Verify signature using ECDSA algorithm.
```

#### Validating P2TR Scripts
```pseudo
For each P2TR transaction:
If len(witness array) == 1:
Perform key path spending with single element in the witness array as signature.
Else if len(witness array) > 1:
If len(witness array) != 3:
Log this transaction and return.
Remove annex if present from witness array.
Parse control block, witness script, and public key.
Validate taprootLeafCommitment.
Check for success opcodes in witness script.
Ensure witness script parses successfully.
Verify signature with public key.
```

### Picking Transactions
Valid transactions are added to a priority queue based on their fee/weight ratio. Transaction weight is calculated considering both the serialized size and the size of witness bytes.

### Creating a Candidate Block
After selecting transactions, we create the coinbase transaction and add the witness commitment if required. We then construct the block header with appropriate values and add the transactions to the candidate block.

#### Witness Commitment
```pseudo
Create array of transaction hashes for all transactions (zero hash for coinbase).
Generate merkle tree root for array of transaction hashes.
Calculate witness commitment as double hash of concatenated 64-byte array of witness merkle root and witness nonce.
Add output entry to coinbase transaction with witness script as witness commitment.
```

### Finding the Nonce
```pseudo
While block hash is not below difficulty target:
Generate random nonce.
Set nonce value in block header.
Hash block header.
```

## Results and Performance

### Validation Performance
- Successfully validated 7377 out of 8130 transactions, covering P2PKH, P2WPKH, and P2TR script types.

### Transaction Selection
- Prioritized transactions based on fee/weight ratio, optimizing block space usage. There is room for further improvement here like taking weighted average etc.

### Block Creation
- Successfully created coinbase transaction and witness commitment, ensuring integrity and security of the block's data.

#### Witness Commitment
- Witness commitment added to the coinbase transaction, ensuring integrity and security of the block's witness data.

### Mining Performance
- Efficiently discovered nonce within the difficulty target, demonstrating robustness of mining algorithm.
- Average mining time per block: Y seconds.

## Conclusion
Our Bitcoin block mining simulation effectively demonstrates key functionalities of the Bitcoin network. By validating transactions, selecting them based on their fee/weight ratio, and constructing valid blocks, we've illustrated the process of creating a secure and efficient blockchain. The mining algorithm efficiently discovers nonces meeting the required difficulty target, highlighting the resilience of the Bitcoin protocol. Future research could focus on further optimizing transaction selection algorithms and exploring advancements in mining efficiency.

## References
- Bitcoin Developer Documentation: [https://developer.bitcoin.org/](https://developer.bitcoin.org/)
- Bitcoin Improvement Proposals (BIPs): [https://bitcoin.org/en/development#bips](https://bitcoin.org/en/development#bips)
- Decred's secp package: [https://github.com/decred/dcrd/dcrec/secp256k1/v4](https://github.com/decred/dcrd/dcrec/secp256k1/v4)
- Bitcoin SV Wiki for OP_CHECKSIG: [https://wiki.bitcoinsv.io/index.php/OP_CHECKSIG#:~:text=OP_CHECKSIG%20is%20an%20opcode%20that,signature%20check%20passes%20or%20fails](https://wiki.bitcoinsv.io/index.php/OP_CHECKSIG#:~:text=OP_CHECKSIG%20is%20an%20opcode%20that,signature%20check%20passes%20or%20fails)
- Bitcoin Improvement Proposals (BIP) 143
144 changes: 0 additions & 144 deletions src/mining/merkle_tree.go
View file
Original file line number Diff line number Diff line change
@@ -1,72 +1,10 @@
package mining

import (
"math/bits"

"github.com/humblenginr/btc-miner/transaction"
"github.com/humblenginr/btc-miner/utils"
)

func HashMerkleBranches(left, right *[32]byte) [32]byte {
// Concatenate the left and right nodes.
var hash [64]byte
copy(hash[:32], left[:])
copy(hash[32:], right[:])

return [32]byte(utils.DoubleHashRaw(hash[:]))
}

// rollingMerkleTreeStore calculates the merkle root by only allocating O(logN)
// memory where N is the total amount of leaves being included in the tree.
type rollingMerkleTreeStore struct {
// roots are where the temporary merkle roots get stored while the
// merkle root is being calculated.
roots []([32]byte)

// numLeaves is the total leaves the store has processed. numLeaves
// is required for the root calculation algorithm to work.
numLeaves uint64
}

// newRollingMerkleTreeStore returns a rollingMerkleTreeStore with the roots
// allocated based on the passed in size.
//
// NOTE: If more elements are added in than the passed in size, there will be
// additional allocations which in turn hurts performance.
func newRollingMerkleTreeStore(size uint64) rollingMerkleTreeStore {
var alloc int
if size != 0 {
alloc = bits.Len64(size - 1)
}
return rollingMerkleTreeStore{roots: make([]([32]byte), 0, alloc)}
}

// add adds a single hash to the merkle tree store. Refer to algorithm 1 "AddOne" in
// the utreexo paper (https://eprint.iacr.org/2019/611.pdf) for the exact algorithm.
func (s *rollingMerkleTreeStore) add(add [32]byte) {
// We can tell where the roots are by looking at the binary representation
// of the numLeaves. Wherever there's a 1, there's a root.
//
// numLeaves of 8 will be '1000' in binary, so there will be one root at
// row 3. numLeaves of 3 will be '11' in binary, so there's two roots. One at
// row 0 and one at row 1. Row 0 is the leaf row.
//
// In this loop below, we're looking for these roots by checking if there's
// a '1', starting from the LSB. If there is a '1', we'll hash the root being
// added with that root until we hit a '0'.
newRoot := add
for h := uint8(0); (s.numLeaves>>h)&1 == 1; h++ {
// Pop off the last root.
var root [32]byte
root, s.roots = s.roots[len(s.roots)-1], s.roots[:len(s.roots)-1]

// Calculate the hash of the new root and append it.
newRoot = HashMerkleBranches(&root, &newRoot)
}
s.roots = append(s.roots, newRoot)
s.numLeaves++
}

func GenerateMerkleTreeRoot(txids [][32]byte) [32]byte{
// reverse the txids
level := make([][32]byte, 0)
Expand Down Expand Up @@ -100,85 +38,3 @@ func GenerateMerkleTreeRoot(txids [][32]byte) [32]byte{
return level[0]
}


func CalcMerkleRoot(transactions []*transaction.Transaction, witness bool) [32]byte {
s := newRollingMerkleTreeStore(uint64(len(transactions)))
return s.calcMerkleRoot(transactions, witness)
}

// calcMerkleRoot returns the merkle root for the passed in transactions.
func (s *rollingMerkleTreeStore) calcMerkleRoot(adds []*transaction.Transaction, witness bool) [32]byte {
for i := range adds {
// If we're computing a witness merkle root, instead of the
// regular txid, we use the modified wtxid which includes a
// transaction's witness data within the digest. Additionally,
// the coinbase's wtxid is all zeroes.
switch {
case witness && adds[i].Vin[0].IsCoinbase:
var zeroHash [32]byte
s.add(zeroHash)
case witness:
s.add([32]byte(adds[i].WitnessHash()))
default:
s.add([32]byte(adds[i].TxHash()))
}
}

// If we only have one leaf, then the hash of that tx is the merkle root.
if s.numLeaves == 1 {
return s.roots[0]
}

// Add on the last tx again if there's an odd number of txs.
if len(adds) > 0 && len(adds)%2 != 0 {
switch {
case witness:
s.add([32]byte(adds[len(adds)-1].WitnessHash()))
default:
s.add([32]byte(adds[len(adds)-1].TxHash()))
}
}

// If we still have more than 1 root after adding on the last tx again,
// we need to do the same for the upper rows.
//
// For example, the below tree has 6 leaves. For row 1, you'll need to
// hash 'F' with itself to create 'C' so you have something to hash with
// 'B'. For bigger trees we may need to do the same in rows 2 or 3 as
// well.
//
// row :3 A
// / \
// row :2 B C
// / \ / \
// row :1 D E F F
// / \ / \ / \
// row :0 1 2 3 4 5 6
for len(s.roots) > 1 {
// If we have to keep adding the last node in the set, bitshift
// the num leaves right by 1. This effectively moves the row up
// for calculation. We do this until we reach a row where there's
// an odd number of leaves.
//
// row :3 A
// / \
// row :2 B C D
// / \ / \ / \
// row :1 E F G H I J
// / \ / \ / \ / \ / \ / \
// row :0 1 2 3 4 5 6 7 8 9 10 11 12
//
// In the above tree, 12 leaves were added and there's an odd amount
// of leaves at row 2. Because of this, we'll bitshift right twice.
currentLeaves := s.numLeaves
for h := uint8(0); (currentLeaves>>h)&1 == 0; h++ {
s.numLeaves >>= 1
}

// Add the last root again so that it'll get hashed with itself.
h := s.roots[len(s.roots)-1]
s.add(h)
}

return s.roots[0]
}

0 comments on commit b145360

Please sign in to comment.