Skip to main content

Validator Node

💜 Support the DIG Network

Help build the future of decentralized storage! The DIG Network is an open-source project that needs community support to continue development.

💜 Support the Project → - Donate crypto, buy NFTs, or sponsor development

Overview​

Validator Nodes are high-performance infrastructure responsible for collecting and validating WitnessCoins at epoch boundaries, then building RewardsPackages for submission to the rewards distributor. A RewardsPackage is simply a list of public keys and their weights, determining reward distribution for the next epoch for both DIG Nodes and Witness Miners. Additionally, validators create RewardsReportCoins as on-chain receipts for transparency and auditability of reward distributions.

System Requirements​

Core Architecture​

class ValidatorNode:
# Epoch Processing
epoch_processor: EpochProcessor # Manage epoch transitions
witness_coin_collector: WitnessCoinCollector # Collect WitnessCoins
proof_validator: ProofValidator # Validate all proofs

# Reward Calculation
weight_calculator: RewardWeightCalculator # Calculate node weights
package_builder: RewardsPackageBuilder # Build reward packages
distributor_interface: DistributorInterface # Submit to rewards distributor
report_coin_creator: RewardsReportCoinCreator # Create on-chain receipts

# Consensus Module
multisig_coordinator: MultisigCoordinator # Coordinate validator signatures
state_machine: ValidatorFSM # State transitions
epoch_timer: EpochTimer # Track epoch boundaries

# Blockchain Interface
chia_client: ChiaClient # Monitor blockchain
singleton_scanner: SingletonScanner # Find WitnessCoins
transaction_builder: TransactionBuilder # Create on-chain transactions

# High Availability
cluster_manager: ClusterManager # Multi-node coordination
failover_controller: FailoverEngine # Automatic recovery
backup_validator: BackupValidator # State replication

Validation Protocol​

Epoch-Based Reward Distribution​

EPOCH_BLOCKS = 37800  # ~7 days at 16s/block

class EpochProcessor:
def __init__(self, epoch_number):
self.epoch = epoch_number
self.start_height = epoch_number * EPOCH_BLOCKS
self.end_height = (epoch_number + 1) * EPOCH_BLOCKS - 1
self.witness_coins = []
self.rewards_package = None

async def process_epoch_end(self):
# Phase 1: Collect all WitnessCoins from the epoch
self.witness_coins = await self.collect_witness_coins()

# Phase 2: Validate all proofs in WitnessCoins
valid_coins = await self.validate_witness_coins()

# Phase 3: Calculate reward weights
reward_weights = await self.calculate_reward_weights(valid_coins)

# Phase 4: Build RewardsPackage (list of public keys and weights)
self.rewards_package = await self.build_rewards_package(reward_weights)

# Phase 5: Submit to rewards distributor and create RewardsReportCoin
submission_result = await self.submit_rewards_package()

return {
"rewards_package": self.rewards_package,
"submission_result": submission_result
}

WitnessCoin Collection​

class WitnessCoinCollector:
def __init__(self, epoch):
self.epoch = epoch
self.chia_client = ChiaClient()

async def collect_witness_coins(self):
# Query blockchain for all WitnessCoins in epoch range
start_height = self.epoch * EPOCH_BLOCKS
end_height = (self.epoch + 1) * EPOCH_BLOCKS - 1

witness_coins = []
for height in range(start_height, end_height + 1):
# Get all singleton spends at this height
coins = await self.chia_client.get_singleton_spends(height)

for coin in coins:
if self.is_witness_coin(coin):
witness_coins.append(self.parse_witness_coin(coin))

return witness_coins

def parse_witness_coin(self, coin_spend):
# Extract validation data from coin
return WitnessCoin(
coin_id=coin_spend.coin.name(),
witness_public_key=coin_spend.solution["witness_id"],
dig_node_public_key=coin_spend.solution["dig_node_public_key"],
physical_access_proof=coin_spend.solution["physical_access_proof"],
ownership_proof=coin_spend.solution["ownership_proof"],
inclusion_proof=coin_spend.solution["inclusion_proof"],
block_height=coin_spend.solution["block_height"],
timestamp=coin_spend.solution["timestamp"]
)

Proof Validation​

class ProofValidator:
async def validate_witness_coins(self, witness_coins):
valid_coins = []

for coin in witness_coins:
# Verify all three proofs
validations = await asyncio.gather(
self.verify_ownership_proof(coin.ownership_proof),
self.verify_inclusion_proof(coin.inclusion_proof),
self.verify_physical_access_proof(coin.physical_access_proof)
)

if all(validations):
# Verify entropy-based selection was correct
if await self.verify_selection_proof(coin):
valid_coins.append(coin)
else:
self.log_invalid_selection(coin)
else:
self.log_invalid_proofs(coin, validations)

return valid_coins

async def verify_selection_proof(self, coin):
# Recreate entropy seed from block hash
block_hash = await self.get_block_hash(coin.block_height)
witness_entropy = sha256(block_hash + coin.witness_public_key)

# Verify this witness should have validated this DIG node
expected_target = self.compute_expected_target(witness_entropy)
return expected_target.matches(coin.dig_node_public_key)

Reward Weight Calculation​

class RewardWeightCalculator:
def __init__(self):
# Weight factors will be published when fully articulated
self.weight_factors = {
"base_weight": 1.0,
# Handle tier multipliers (shorter handles = higher rewards)
"handle_tier_multipliers": {
3: 5.0, 4: 4.0, 5: 3.0, 6: 2.0, 7: 1.5, 8: 1.0
},
# Capsule size multipliers (larger capsules = higher rewards)
"capsule_size_multipliers": {
0.25: 1.0, 1.0: 1.2, 10.0: 1.5, 100.0: 2.0, 1000.0: 3.0
}
}

async def calculate_reward_weights(self, valid_coins):
# Group by node type
dig_nodes = {}
witness_nodes = {}

for coin in valid_coins:
# Track DIG node validations
if coin.dig_node_public_key not in dig_nodes:
dig_nodes[coin.dig_node_public_key] = []
dig_nodes[coin.dig_node_public_key].append(coin)

# Track witness validations
if coin.witness_public_key not in witness_nodes:
witness_nodes[coin.witness_public_key] = []
witness_nodes[coin.witness_public_key].append(coin)

# Calculate weights for each node
reward_weights = []

# DIG Node weights
for node_key, validations in dig_nodes.items():
weight = await self.calculate_dig_node_weight(node_key, validations)
reward_weights.append({
"node_type": "DIG",
"public_key": node_key,
"weight": weight,
"validation_count": len(validations)
})

# Witness Miner weights
for node_key, validations in witness_nodes.items():
weight = await self.calculate_witness_weight(node_key, validations)
reward_weights.append({
"node_type": "WITNESS",
"public_key": node_key,
"weight": weight,
"validation_count": len(validations)
})

return reward_weights

async def calculate_dig_node_weight(self, node_key, validations):
# Calculate weight based on validations and their associated content value
base_weight = self.weight_factors["base_weight"]
total_weight = 0

for validation in validations:
# Get handle tier and capsule size for this validation
handle_tier = await self.get_handle_tier(validation.datastore_id)
capsule_size_mb = await self.get_capsule_size(validation.capsule_id)

# Apply multipliers: shorter handles and larger capsules = higher rewards
handle_multiplier = self.weight_factors["handle_tier_multipliers"].get(handle_tier, 1.0)
size_multiplier = self.weight_factors["capsule_size_multipliers"].get(capsule_size_mb, 1.0)

validation_weight = base_weight * handle_multiplier * size_multiplier
total_weight += validation_weight

return total_weight

async def calculate_witness_weight(self, node_key, validations):
# Calculate weight based on validations and their associated content value
base_weight = self.weight_factors["base_weight"]
total_weight = 0

for validation in validations:
# Get handle tier and capsule size for this validation
handle_tier = await self.get_handle_tier(validation.datastore_id)
capsule_size_mb = await self.get_capsule_size(validation.capsule_id)

# Apply multipliers: shorter handles and larger capsules = higher rewards
handle_multiplier = self.weight_factors["handle_tier_multipliers"].get(handle_tier, 1.0)
size_multiplier = self.weight_factors["capsule_size_multipliers"].get(capsule_size_mb, 1.0)

validation_weight = base_weight * handle_multiplier * size_multiplier
total_weight += validation_weight

return total_weight

async def get_handle_tier(self, datastore_id: str) -> int:
"""Get the handle tier (character count) for a given datastore"""
# Query the DIG Handle registry to find handle for this datastore
handle = await self.query_handle_registry(datastore_id)
if handle:
return min(len(handle), 8) # 8+ chars all treated as tier 8
return 8 # Default to lowest tier if no handle found

async def get_capsule_size(self, capsule_id: str) -> float:
"""Get the capsule size in MB for reward calculation"""
# Query capsule metadata to get size
capsule_info = await self.query_capsule_info(capsule_id)
if capsule_info:
size_bytes = capsule_info.get('size', 0)
size_mb = size_bytes / (1024 * 1024) # Convert to MB

# Return standardized sizes for multiplier lookup
if size_mb <= 0.25:
return 0.25
elif size_mb <= 1.0:
return 1.0
elif size_mb <= 10.0:
return 10.0
elif size_mb <= 100.0:
return 100.0
else:
return 1000.0
return 0.25 # Default to smallest size if info not found

RewardsPackage Building​

class RewardsPackageBuilder:
def __init__(self, epoch):
self.epoch = epoch
self.next_epoch = epoch + 1

async def build_rewards_package(self, reward_weights):
# Simple list of public keys and weights
rewards_package = []

for entry in reward_weights:
rewards_package.append({
"public_key": entry["public_key"],
"weight": entry["weight"]
})

return rewards_package

async def submit_rewards_package(self, rewards_package):
# Phase 1: Submit to rewards distributor smart contract
distributor_tx = await self.create_rewards_transaction(
rewards_distributor_address=REWARDS_DISTRIBUTOR_ADDRESS,
rewards_list=rewards_package,
epoch=self.next_epoch
)

# Multisig signing for distributor
signed_distributor_tx = await self.multisig_sign(distributor_tx)
distributor_tx_id = await self.broadcast_transaction(signed_distributor_tx)

# Phase 2: Create RewardsReportCoin as on-chain receipt
report_coin_data = {
"epoch": self.next_epoch,
"rewards_package": rewards_package,
"recipient_count": len(rewards_package),
"total_weight": sum(r["weight"] for r in rewards_package),
"distributor_tx_id": distributor_tx_id,
"created_at": int(time.time()),
"validator_signatures": await self.get_multisig_signatures()
}

report_coin = await self.create_rewards_report_coin(report_coin_data)
report_coin_tx_id = await self.broadcast_transaction(report_coin)

return {
"distributor_tx_id": distributor_tx_id,
"report_coin_id": report_coin.name(),
"report_coin_tx_id": report_coin_tx_id,
"epoch": self.next_epoch,
"recipient_count": len(rewards_package)
}

RewardsReportCoin​

class RewardsReportCoin:
"""
On-chain receipt for epoch reward distributions.
Provides transparency and auditability for reward allocations.
"""

def __init__(self, epoch, rewards_package):
self.epoch = epoch
self.rewards_package = rewards_package
self.created_at = int(time.time())

async def create_rewards_report_coin(self, report_data):
# Create singleton coin for permanent record
report_coin = self.create_singleton(
parent_coin=self.get_parent_coin(),
inner_puzzle_hash=self.report_coin_puzzle_hash,
amount=1, # Singleton amount
metadata={
"epoch": report_data["epoch"],
"rewards_list": report_data["rewards_package"],
"recipient_count": report_data["recipient_count"],
"total_weight": report_data["total_weight"],
"distributor_tx_id": report_data["distributor_tx_id"],
"timestamp": report_data["created_at"],
"validator_signatures": report_data["validator_signatures"]
}
)

return report_coin

@staticmethod
async def query_report_coins(epoch=None):
"""Query historical RewardsReportCoins for transparency"""
if epoch:
# Get specific epoch report
return await query_singleton_by_epoch(epoch)
else:
# Get all reports
return await query_all_report_coins()

API Specifications​

Epoch Processing Endpoints​

GET /validator/status
Response: {
"validator_id": "val_abc123",
"status": "active",
"current_epoch": 42,
"next_epoch_start": "2024-01-02T00:00:00Z",
"uptime_percentage": 99.95,
"cluster_status": {
"primary": "val_abc123",
"secondaries": ["val_def456", "val_ghi789"],
"sync_status": "synchronized"
}
}

GET /epoch/{epoch}/status
Response: {
"epoch": 42,
"start_height": 193536,
"end_height": 198143,
"status": "processing",
"witness_coins_collected": 1250,
"valid_coins": 1230,
"processing_progress": 85
}

POST /epoch/process
Body: {
"epoch": 42,
"validator_signature": "0x..."
}
Response: {
"epoch": 42,
"witness_coins_processed": 1250,
"valid_coins": 1230,
"rewards_package_id": "pkg_123",
"distributor_tx_id": "tx_456",
"report_coin_id": "coin_789",
"report_coin_tx_id": "tx_790"
}

WitnessCoin Validation​

GET /witness-coins/{epoch}
Response: {
"epoch": 42,
"total_coins": 1250,
"coins": [
{
"coin_id": "coin_123",
"witness_public_key": "0x...",
"dig_node_public_key": "0x...",
"validation_status": "valid",
"block_height": 193600
},
...
]
}

POST /witness-coin/validate
Body: {
"coin_id": "coin_123",
"proofs": {...}
}
Response: {
"coin_id": "coin_123",
"validation_result": {
"ownership_proof": "valid",
"inclusion_proof": "valid",
"physical_access_proof": "valid",
"selection_proof": "valid"
},
"overall_status": "valid"
}

Rewards Package Management​

GET /rewards/package/{epoch}
Response: {
"epoch": 43,
"status": "submitted",
"distributor_tx_id": "tx_456",
"report_coin_id": "coin_789",
"report_coin_tx_id": "tx_790",
"recipient_count": 850
}

GET /rewards/recipients/{epoch}
Response: {
"epoch": 43,
"recipients": [
{
"public_key": "0x...",
"weight": 15.5
},
{
"public_key": "0x...",
"weight": 5.0
},
...
]
}

GET /rewards/report-coin/{epoch}
Response: {
"epoch": 43,
"report_coin_id": "coin_789",
"recipient_count": 850,
"total_weight": 12500.5,
"distributor_tx_id": "tx_456",
"created_at": "2024-01-02T00:00:00Z",
"blockchain_height": 198144,
"validator_signatures": [
"0xsig1...",
"0xsig2...",
"0xsig3..."
]
}

GET /rewards/report-coins
Response: {
"report_coins": [
{
"epoch": 43,
"report_coin_id": "coin_789",
"recipient_count": 850,
"total_weight": 12500.5,
"created_at": "2024-01-02T00:00:00Z"
},
{
"epoch": 42,
"report_coin_id": "coin_456",
"recipient_count": 823,
"total_weight": 12100.0,
"created_at": "2024-01-01T00:00:00Z"
},
...
]
}

Consensus Mechanism​

Multi-Validator Coordination​

class ValidatorCluster:
def __init__(self, validator_ids, threshold=0.67):
self.validators = validator_ids
self.threshold = threshold
self.current_epoch = None

async def process_epoch_end(self, epoch):
# Coordinate epoch processing among validators
self.current_epoch = epoch

# Phase 1: Agree on epoch boundaries
epoch_agreement = await self.reach_epoch_consensus(epoch)

# Phase 2: Collect WitnessCoins collaboratively
witness_coins = await self.collaborative_collection(epoch)

# Phase 3: Validate proofs with redundancy
valid_coins = await self.distributed_validation(witness_coins)

# Phase 4: Build rewards package with multisig
rewards_package = await self.build_consensus_package(valid_coins)

# Phase 5: Submit with threshold signatures
return await self.submit_with_multisig(rewards_package)

async def collaborative_collection(self, epoch):
# Each validator collects a range of blocks
blocks_per_validator = EPOCH_BLOCKS // len(self.validators)

collections = []
for i, validator in enumerate(self.validators):
start_block = epoch * EPOCH_BLOCKS + (i * blocks_per_validator)
end_block = start_block + blocks_per_validator - 1

coins = await validator.collect_witness_coins_range(start_block, end_block)
collections.append(coins)

# Merge and deduplicate
all_coins = self.merge_collections(collections)
return all_coins

State Synchronization​

class StateSyncManager:
def __init__(self):
self.state_db = StateDatabase()
self.sync_peers = []

async def sync_state(self):
# Get current state hash
local_state = self.state_db.get_state_hash()

# Query peers for their state
peer_states = await self.query_peer_states()

# Find majority state
majority_state = self.find_majority(peer_states)

if local_state != majority_state:
# Sync required
await self.download_state_diff(majority_state)
await self.apply_state_diff()

async def checkpoint_state(self, round_id):
# Create state checkpoint
checkpoint = StateCheckpoint(
round_id=round_id,
validator_states=self.get_all_validator_states(),
reward_distributions=self.get_pending_rewards(),
network_metrics=self.get_network_snapshot()
)

# Store locally
await self.state_db.store_checkpoint(checkpoint)

# Broadcast to peers
await self.broadcast_checkpoint(checkpoint)

Multi-Signature Validation​

class MultiSigValidator:
def __init__(self, threshold=2, total=3):
self.threshold = threshold
self.total = total
self.signers = []

async def validate_round(self, round_data):
# Each validator independently validates
signatures = []
for i in range(self.total):
sig = await self.validators[i].sign_round(round_data)
signatures.append(sig)

# Combine signatures
if len(signatures) >= self.threshold:
combined_sig = self.combine_signatures(signatures)
return ValidationResult(
round_id=round_data.round_id,
signature=combined_sig,
signers=self.signers[:self.threshold]
)
else:
raise InsufficientSignatures()

Validator Accountability​

class ValidatorAccountability:
def __init__(self):
self.performance_metrics = {
"epochs_processed": 0,
"witness_coins_validated": 0,
"rewards_packages_created": 0,
"processing_errors": 0,
"missed_epochs": 0
}

async def track_epoch_processing(self, validator_id, epoch_result):
# Track validator performance
if epoch_result.success:
self.performance_metrics["epochs_processed"] += 1
self.performance_metrics["witness_coins_validated"] += epoch_result.coins_processed
self.performance_metrics["rewards_packages_created"] += 1
else:
self.performance_metrics["processing_errors"] += 1

# Log for transparency
await self.publish_performance_report(validator_id, epoch_result)

async def check_epoch_completion(self, epoch):
# Ensure validators process epochs on time
epoch_end_height = (epoch + 1) * EPOCH_BLOCKS
current_height = await self.get_current_height()

if current_height > epoch_end_height + GRACE_PERIOD_BLOCKS:
if not await self.is_epoch_processed(epoch):
self.performance_metrics["missed_epochs"] += 1
await self.alert_community(f"Epoch {epoch} processing delayed")

async def verify_reward_distribution(self, epoch):
# RewardsReportCoins provide full transparency
report_coin = await self.query_report_coin(epoch)
distributor_state = await self.query_distributor_state(epoch)

# Anyone can verify reward calculations
return {
"report_coin_id": report_coin.id,
"recipients_in_report": report_coin.recipient_count,
"recipients_paid": distributor_state.paid_count,
"discrepancies": self.find_discrepancies(report_coin, distributor_state)
}

Monitoring and Metrics​

Performance Monitoring​

VALIDATOR_METRICS = {
# Epoch Processing metrics
"epochs_processed": Counter("epochs successfully processed"),
"witness_coins_collected": Counter("total WitnessCoins collected"),
"witness_coins_validated": Counter("WitnessCoins validated"),
"rewards_packages_created": Counter("RewardsPackages created"),
"report_coins_created": Counter("RewardsReportCoins created"),
"epoch_processing_time": Histogram("time to process epoch"),

# Validation metrics
"proof_validation_time": Histogram("time to validate proofs"),
"selection_proof_failures": Counter("invalid selection proofs"),
"physical_access_failures": Counter("invalid physical access proofs"),

# Network metrics
"peer_validators": Gauge("connected validator peers"),
"blockchain_sync_lag": Gauge("blocks behind network"),
"witness_coin_scan_time": Histogram("time to scan for WitnessCoins"),

# Resource metrics
"cpu_usage": Gauge("CPU utilization %"),
"memory_usage": Gauge("Memory utilization %"),
"disk_io": Gauge("Disk IOPS"),
"network_bandwidth": Gauge("Mbps in/out"),

# Economic metrics
"total_reward_weight": Gauge("sum of all reward weights"),
"unique_dig_nodes": Gauge("unique DIG nodes in rewards"),
"unique_witnesses": Gauge("unique witnesses in rewards"),
}

Configuration​

# validator.yaml
validator:
id: "val_abc123"
cluster:
size: 3
threshold: 0.67

epoch_processing:
epoch_duration_blocks: 4608 # ~24 hours
processing_grace_period: 288 # ~1.5 hours after epoch end
witness_coin_batch_size: 100
parallel_validation_threads: 8

blockchain:
chia_rpc_endpoint: "https://localhost:8555"
chia_certificate_path: "/certs/chia.crt"
chia_key_path: "/certs/chia.key"
scan_batch_size: 100

rewards:
distributor_address: "0x..."
report_coin_puzzle_hash: "0x..."
weight_factors_config: "/config/weight_factors.yaml"
min_witnesses_per_epoch: 10
max_reward_recipients: 10000

network:
p2p_port: 8446
rpc_port: 8547
bootstrap_validators:
- "val1.dig.net:8446"
- "val2.dig.net:8446"

security:
multisig_threshold: 2
multisig_signers: 3
tls_cert_path: "/certs/validator.crt"
tls_key_path: "/certs/validator.key"
enable_rate_limiting: true
max_requests_per_minute: 1000

monitoring:
metrics_port: 9092
enable_profiling: true
log_level: "info"
alert_webhook: "https://alerts.dig.net/webhook"

high_availability:
enable_failover: true
health_check_interval: 5
state_backup_interval: 300
backup_locations:
- "s3://dig-backups/validator/"
- "/backup/validator/"

Future DAO Integration​

Transition Plan​

class DAOTransition:
"""
Phase 1: Current (Multisig)
- Curated validator selection
- Manual weight factor updates
- Community oversight of reward distribution

Phase 2: DAO Launch (Post-Chia DAO)
- On-chain validator election
- DAO-governed weight factors
- Automated epoch processing

Phase 3: Full Automation
- Smart contract-based WitnessCoin validation
- Autonomous reward distribution
- Minimal governance needed
"""

async def phase2_integration(self):
# DAO-based validator selection
validators = await self.dao.get_elected_validators()

# Weight factors via governance
weight_factors = await self.dao.get_reward_weight_factors()

# Automated epoch processing
if self.is_epoch_boundary():
epoch = self.get_current_epoch()
await self.process_epoch_with_dao_oversight(epoch, validators)

# Performance monitoring
validator_performance = await self.monitor_epoch_processing()
if validator_performance.below_threshold():
await self.dao.propose_validator_replacement(validator_id)

On-Chain Artifacts Created​

  • WitnessCoins - Collected from witness miners (input)
  • RewardsPackages - Submitted to rewards distributor (output)
  • RewardsReportCoins - On-chain receipts for transparency (audit trail)

Data Flow​

Epoch N: Witness Miners → Create WitnessCoins
↓
Epoch N+1 Start: Validators → Collect WitnessCoins
↓
Validate Proofs
↓
Calculate Weights
↓
Create RewardsPackage
↓
├→ Submit to Rewards Distributor
└→ Create RewardsReportCoin (on-chain receipt)
↓
Community can verify distributions