Validator Node
💜 Support the DIG Network
Help build the future of decentralized storage! The DIG Network is an open-source project that needs community support to continue development.
💜 Support the Project → - Donate crypto, buy NFTs, or sponsor development
Overview​
Validator Nodes are high-performance infrastructure responsible for collecting and validating WitnessCoins at epoch boundaries, then building RewardsPackages for submission to the rewards distributor. A RewardsPackage is simply a list of public keys and their weights, determining reward distribution for the next epoch for both DIG Nodes and Witness Miners. Additionally, validators create RewardsReportCoins as on-chain receipts for transparency and auditability of reward distributions.
System Requirements​
Core Architecture​
class ValidatorNode:
# Epoch Processing
epoch_processor: EpochProcessor # Manage epoch transitions
witness_coin_collector: WitnessCoinCollector # Collect WitnessCoins
proof_validator: ProofValidator # Validate all proofs
# Reward Calculation
weight_calculator: RewardWeightCalculator # Calculate node weights
package_builder: RewardsPackageBuilder # Build reward packages
distributor_interface: DistributorInterface # Submit to rewards distributor
report_coin_creator: RewardsReportCoinCreator # Create on-chain receipts
# Consensus Module
multisig_coordinator: MultisigCoordinator # Coordinate validator signatures
state_machine: ValidatorFSM # State transitions
epoch_timer: EpochTimer # Track epoch boundaries
# Blockchain Interface
chia_client: ChiaClient # Monitor blockchain
singleton_scanner: SingletonScanner # Find WitnessCoins
transaction_builder: TransactionBuilder # Create on-chain transactions
# High Availability
cluster_manager: ClusterManager # Multi-node coordination
failover_controller: FailoverEngine # Automatic recovery
backup_validator: BackupValidator # State replication
Validation Protocol​
Epoch-Based Reward Distribution​
EPOCH_BLOCKS = 37800 # ~7 days at 16s/block
class EpochProcessor:
def __init__(self, epoch_number):
self.epoch = epoch_number
self.start_height = epoch_number * EPOCH_BLOCKS
self.end_height = (epoch_number + 1) * EPOCH_BLOCKS - 1
self.witness_coins = []
self.rewards_package = None
async def process_epoch_end(self):
# Phase 1: Collect all WitnessCoins from the epoch
self.witness_coins = await self.collect_witness_coins()
# Phase 2: Validate all proofs in WitnessCoins
valid_coins = await self.validate_witness_coins()
# Phase 3: Calculate reward weights
reward_weights = await self.calculate_reward_weights(valid_coins)
# Phase 4: Build RewardsPackage (list of public keys and weights)
self.rewards_package = await self.build_rewards_package(reward_weights)
# Phase 5: Submit to rewards distributor and create RewardsReportCoin
submission_result = await self.submit_rewards_package()
return {
"rewards_package": self.rewards_package,
"submission_result": submission_result
}
WitnessCoin Collection​
class WitnessCoinCollector:
def __init__(self, epoch):
self.epoch = epoch
self.chia_client = ChiaClient()
async def collect_witness_coins(self):
# Query blockchain for all WitnessCoins in epoch range
start_height = self.epoch * EPOCH_BLOCKS
end_height = (self.epoch + 1) * EPOCH_BLOCKS - 1
witness_coins = []
for height in range(start_height, end_height + 1):
# Get all singleton spends at this height
coins = await self.chia_client.get_singleton_spends(height)
for coin in coins:
if self.is_witness_coin(coin):
witness_coins.append(self.parse_witness_coin(coin))
return witness_coins
def parse_witness_coin(self, coin_spend):
# Extract validation data from coin
return WitnessCoin(
coin_id=coin_spend.coin.name(),
witness_public_key=coin_spend.solution["witness_id"],
dig_node_public_key=coin_spend.solution["dig_node_public_key"],
physical_access_proof=coin_spend.solution["physical_access_proof"],
ownership_proof=coin_spend.solution["ownership_proof"],
inclusion_proof=coin_spend.solution["inclusion_proof"],
block_height=coin_spend.solution["block_height"],
timestamp=coin_spend.solution["timestamp"]
)
Proof Validation​
class ProofValidator:
async def validate_witness_coins(self, witness_coins):
valid_coins = []
for coin in witness_coins:
# Verify all three proofs
validations = await asyncio.gather(
self.verify_ownership_proof(coin.ownership_proof),
self.verify_inclusion_proof(coin.inclusion_proof),
self.verify_physical_access_proof(coin.physical_access_proof)
)
if all(validations):
# Verify entropy-based selection was correct
if await self.verify_selection_proof(coin):
valid_coins.append(coin)
else:
self.log_invalid_selection(coin)
else:
self.log_invalid_proofs(coin, validations)
return valid_coins
async def verify_selection_proof(self, coin):
# Recreate entropy seed from block hash
block_hash = await self.get_block_hash(coin.block_height)
witness_entropy = sha256(block_hash + coin.witness_public_key)
# Verify this witness should have validated this DIG node
expected_target = self.compute_expected_target(witness_entropy)
return expected_target.matches(coin.dig_node_public_key)
Reward Weight Calculation​
class RewardWeightCalculator:
def __init__(self):
# Weight factors will be published when fully articulated
self.weight_factors = {
"base_weight": 1.0,
# Handle tier multipliers (shorter handles = higher rewards)
"handle_tier_multipliers": {
3: 5.0, 4: 4.0, 5: 3.0, 6: 2.0, 7: 1.5, 8: 1.0
},
# Capsule size multipliers (larger capsules = higher rewards)
"capsule_size_multipliers": {
0.25: 1.0, 1.0: 1.2, 10.0: 1.5, 100.0: 2.0, 1000.0: 3.0
}
}
async def calculate_reward_weights(self, valid_coins):
# Group by node type
dig_nodes = {}
witness_nodes = {}
for coin in valid_coins:
# Track DIG node validations
if coin.dig_node_public_key not in dig_nodes:
dig_nodes[coin.dig_node_public_key] = []
dig_nodes[coin.dig_node_public_key].append(coin)
# Track witness validations
if coin.witness_public_key not in witness_nodes:
witness_nodes[coin.witness_public_key] = []
witness_nodes[coin.witness_public_key].append(coin)
# Calculate weights for each node
reward_weights = []
# DIG Node weights
for node_key, validations in dig_nodes.items():
weight = await self.calculate_dig_node_weight(node_key, validations)
reward_weights.append({
"node_type": "DIG",
"public_key": node_key,
"weight": weight,
"validation_count": len(validations)
})
# Witness Miner weights
for node_key, validations in witness_nodes.items():
weight = await self.calculate_witness_weight(node_key, validations)
reward_weights.append({
"node_type": "WITNESS",
"public_key": node_key,
"weight": weight,
"validation_count": len(validations)
})
return reward_weights
async def calculate_dig_node_weight(self, node_key, validations):
# Calculate weight based on validations and their associated content value
base_weight = self.weight_factors["base_weight"]
total_weight = 0
for validation in validations:
# Get handle tier and capsule size for this validation
handle_tier = await self.get_handle_tier(validation.datastore_id)
capsule_size_mb = await self.get_capsule_size(validation.capsule_id)
# Apply multipliers: shorter handles and larger capsules = higher rewards
handle_multiplier = self.weight_factors["handle_tier_multipliers"].get(handle_tier, 1.0)
size_multiplier = self.weight_factors["capsule_size_multipliers"].get(capsule_size_mb, 1.0)
validation_weight = base_weight * handle_multiplier * size_multiplier
total_weight += validation_weight
return total_weight
async def calculate_witness_weight(self, node_key, validations):
# Calculate weight based on validations and their associated content value
base_weight = self.weight_factors["base_weight"]
total_weight = 0
for validation in validations:
# Get handle tier and capsule size for this validation
handle_tier = await self.get_handle_tier(validation.datastore_id)
capsule_size_mb = await self.get_capsule_size(validation.capsule_id)
# Apply multipliers: shorter handles and larger capsules = higher rewards
handle_multiplier = self.weight_factors["handle_tier_multipliers"].get(handle_tier, 1.0)
size_multiplier = self.weight_factors["capsule_size_multipliers"].get(capsule_size_mb, 1.0)
validation_weight = base_weight * handle_multiplier * size_multiplier
total_weight += validation_weight
return total_weight
async def get_handle_tier(self, datastore_id: str) -> int:
"""Get the handle tier (character count) for a given datastore"""
# Query the DIG Handle registry to find handle for this datastore
handle = await self.query_handle_registry(datastore_id)
if handle:
return min(len(handle), 8) # 8+ chars all treated as tier 8
return 8 # Default to lowest tier if no handle found
async def get_capsule_size(self, capsule_id: str) -> float:
"""Get the capsule size in MB for reward calculation"""
# Query capsule metadata to get size
capsule_info = await self.query_capsule_info(capsule_id)
if capsule_info:
size_bytes = capsule_info.get('size', 0)
size_mb = size_bytes / (1024 * 1024) # Convert to MB
# Return standardized sizes for multiplier lookup
if size_mb <= 0.25:
return 0.25
elif size_mb <= 1.0:
return 1.0
elif size_mb <= 10.0:
return 10.0
elif size_mb <= 100.0:
return 100.0
else:
return 1000.0
return 0.25 # Default to smallest size if info not found
RewardsPackage Building​
class RewardsPackageBuilder:
def __init__(self, epoch):
self.epoch = epoch
self.next_epoch = epoch + 1
async def build_rewards_package(self, reward_weights):
# Simple list of public keys and weights
rewards_package = []
for entry in reward_weights:
rewards_package.append({
"public_key": entry["public_key"],
"weight": entry["weight"]
})
return rewards_package
async def submit_rewards_package(self, rewards_package):
# Phase 1: Submit to rewards distributor smart contract
distributor_tx = await self.create_rewards_transaction(
rewards_distributor_address=REWARDS_DISTRIBUTOR_ADDRESS,
rewards_list=rewards_package,
epoch=self.next_epoch
)
# Multisig signing for distributor
signed_distributor_tx = await self.multisig_sign(distributor_tx)
distributor_tx_id = await self.broadcast_transaction(signed_distributor_tx)
# Phase 2: Create RewardsReportCoin as on-chain receipt
report_coin_data = {
"epoch": self.next_epoch,
"rewards_package": rewards_package,
"recipient_count": len(rewards_package),
"total_weight": sum(r["weight"] for r in rewards_package),
"distributor_tx_id": distributor_tx_id,
"created_at": int(time.time()),
"validator_signatures": await self.get_multisig_signatures()
}
report_coin = await self.create_rewards_report_coin(report_coin_data)
report_coin_tx_id = await self.broadcast_transaction(report_coin)
return {
"distributor_tx_id": distributor_tx_id,
"report_coin_id": report_coin.name(),
"report_coin_tx_id": report_coin_tx_id,
"epoch": self.next_epoch,
"recipient_count": len(rewards_package)
}
RewardsReportCoin​
class RewardsReportCoin:
"""
On-chain receipt for epoch reward distributions.
Provides transparency and auditability for reward allocations.
"""
def __init__(self, epoch, rewards_package):
self.epoch = epoch
self.rewards_package = rewards_package
self.created_at = int(time.time())
async def create_rewards_report_coin(self, report_data):
# Create singleton coin for permanent record
report_coin = self.create_singleton(
parent_coin=self.get_parent_coin(),
inner_puzzle_hash=self.report_coin_puzzle_hash,
amount=1, # Singleton amount
metadata={
"epoch": report_data["epoch"],
"rewards_list": report_data["rewards_package"],
"recipient_count": report_data["recipient_count"],
"total_weight": report_data["total_weight"],
"distributor_tx_id": report_data["distributor_tx_id"],
"timestamp": report_data["created_at"],
"validator_signatures": report_data["validator_signatures"]
}
)
return report_coin
@staticmethod
async def query_report_coins(epoch=None):
"""Query historical RewardsReportCoins for transparency"""
if epoch:
# Get specific epoch report
return await query_singleton_by_epoch(epoch)
else:
# Get all reports
return await query_all_report_coins()
API Specifications​
Epoch Processing Endpoints​
GET /validator/status
Response: {
"validator_id": "val_abc123",
"status": "active",
"current_epoch": 42,
"next_epoch_start": "2024-01-02T00:00:00Z",
"uptime_percentage": 99.95,
"cluster_status": {
"primary": "val_abc123",
"secondaries": ["val_def456", "val_ghi789"],
"sync_status": "synchronized"
}
}
GET /epoch/{epoch}/status
Response: {
"epoch": 42,
"start_height": 193536,
"end_height": 198143,
"status": "processing",
"witness_coins_collected": 1250,
"valid_coins": 1230,
"processing_progress": 85
}
POST /epoch/process
Body: {
"epoch": 42,
"validator_signature": "0x..."
}
Response: {
"epoch": 42,
"witness_coins_processed": 1250,
"valid_coins": 1230,
"rewards_package_id": "pkg_123",
"distributor_tx_id": "tx_456",
"report_coin_id": "coin_789",
"report_coin_tx_id": "tx_790"
}
WitnessCoin Validation​
GET /witness-coins/{epoch}
Response: {
"epoch": 42,
"total_coins": 1250,
"coins": [
{
"coin_id": "coin_123",
"witness_public_key": "0x...",
"dig_node_public_key": "0x...",
"validation_status": "valid",
"block_height": 193600
},
...
]
}
POST /witness-coin/validate
Body: {
"coin_id": "coin_123",
"proofs": {...}
}
Response: {
"coin_id": "coin_123",
"validation_result": {
"ownership_proof": "valid",
"inclusion_proof": "valid",
"physical_access_proof": "valid",
"selection_proof": "valid"
},
"overall_status": "valid"
}
Rewards Package Management​
GET /rewards/package/{epoch}
Response: {
"epoch": 43,
"status": "submitted",
"distributor_tx_id": "tx_456",
"report_coin_id": "coin_789",
"report_coin_tx_id": "tx_790",
"recipient_count": 850
}
GET /rewards/recipients/{epoch}
Response: {
"epoch": 43,
"recipients": [
{
"public_key": "0x...",
"weight": 15.5
},
{
"public_key": "0x...",
"weight": 5.0
},
...
]
}
GET /rewards/report-coin/{epoch}
Response: {
"epoch": 43,
"report_coin_id": "coin_789",
"recipient_count": 850,
"total_weight": 12500.5,
"distributor_tx_id": "tx_456",
"created_at": "2024-01-02T00:00:00Z",
"blockchain_height": 198144,
"validator_signatures": [
"0xsig1...",
"0xsig2...",
"0xsig3..."
]
}
GET /rewards/report-coins
Response: {
"report_coins": [
{
"epoch": 43,
"report_coin_id": "coin_789",
"recipient_count": 850,
"total_weight": 12500.5,
"created_at": "2024-01-02T00:00:00Z"
},
{
"epoch": 42,
"report_coin_id": "coin_456",
"recipient_count": 823,
"total_weight": 12100.0,
"created_at": "2024-01-01T00:00:00Z"
},
...
]
}
Consensus Mechanism​
Multi-Validator Coordination​
class ValidatorCluster:
def __init__(self, validator_ids, threshold=0.67):
self.validators = validator_ids
self.threshold = threshold
self.current_epoch = None
async def process_epoch_end(self, epoch):
# Coordinate epoch processing among validators
self.current_epoch = epoch
# Phase 1: Agree on epoch boundaries
epoch_agreement = await self.reach_epoch_consensus(epoch)
# Phase 2: Collect WitnessCoins collaboratively
witness_coins = await self.collaborative_collection(epoch)
# Phase 3: Validate proofs with redundancy
valid_coins = await self.distributed_validation(witness_coins)
# Phase 4: Build rewards package with multisig
rewards_package = await self.build_consensus_package(valid_coins)
# Phase 5: Submit with threshold signatures
return await self.submit_with_multisig(rewards_package)
async def collaborative_collection(self, epoch):
# Each validator collects a range of blocks
blocks_per_validator = EPOCH_BLOCKS // len(self.validators)
collections = []
for i, validator in enumerate(self.validators):
start_block = epoch * EPOCH_BLOCKS + (i * blocks_per_validator)
end_block = start_block + blocks_per_validator - 1
coins = await validator.collect_witness_coins_range(start_block, end_block)
collections.append(coins)
# Merge and deduplicate
all_coins = self.merge_collections(collections)
return all_coins
State Synchronization​
class StateSyncManager:
def __init__(self):
self.state_db = StateDatabase()
self.sync_peers = []
async def sync_state(self):
# Get current state hash
local_state = self.state_db.get_state_hash()
# Query peers for their state
peer_states = await self.query_peer_states()
# Find majority state
majority_state = self.find_majority(peer_states)
if local_state != majority_state:
# Sync required
await self.download_state_diff(majority_state)
await self.apply_state_diff()
async def checkpoint_state(self, round_id):
# Create state checkpoint
checkpoint = StateCheckpoint(
round_id=round_id,
validator_states=self.get_all_validator_states(),
reward_distributions=self.get_pending_rewards(),
network_metrics=self.get_network_snapshot()
)
# Store locally
await self.state_db.store_checkpoint(checkpoint)
# Broadcast to peers
await self.broadcast_checkpoint(checkpoint)
Multi-Signature Validation​
class MultiSigValidator:
def __init__(self, threshold=2, total=3):
self.threshold = threshold
self.total = total
self.signers = []
async def validate_round(self, round_data):
# Each validator independently validates
signatures = []
for i in range(self.total):
sig = await self.validators[i].sign_round(round_data)
signatures.append(sig)
# Combine signatures
if len(signatures) >= self.threshold:
combined_sig = self.combine_signatures(signatures)
return ValidationResult(
round_id=round_data.round_id,
signature=combined_sig,
signers=self.signers[:self.threshold]
)
else:
raise InsufficientSignatures()
Validator Accountability​
class ValidatorAccountability:
def __init__(self):
self.performance_metrics = {
"epochs_processed": 0,
"witness_coins_validated": 0,
"rewards_packages_created": 0,
"processing_errors": 0,
"missed_epochs": 0
}
async def track_epoch_processing(self, validator_id, epoch_result):
# Track validator performance
if epoch_result.success:
self.performance_metrics["epochs_processed"] += 1
self.performance_metrics["witness_coins_validated"] += epoch_result.coins_processed
self.performance_metrics["rewards_packages_created"] += 1
else:
self.performance_metrics["processing_errors"] += 1
# Log for transparency
await self.publish_performance_report(validator_id, epoch_result)
async def check_epoch_completion(self, epoch):
# Ensure validators process epochs on time
epoch_end_height = (epoch + 1) * EPOCH_BLOCKS
current_height = await self.get_current_height()
if current_height > epoch_end_height + GRACE_PERIOD_BLOCKS:
if not await self.is_epoch_processed(epoch):
self.performance_metrics["missed_epochs"] += 1
await self.alert_community(f"Epoch {epoch} processing delayed")
async def verify_reward_distribution(self, epoch):
# RewardsReportCoins provide full transparency
report_coin = await self.query_report_coin(epoch)
distributor_state = await self.query_distributor_state(epoch)
# Anyone can verify reward calculations
return {
"report_coin_id": report_coin.id,
"recipients_in_report": report_coin.recipient_count,
"recipients_paid": distributor_state.paid_count,
"discrepancies": self.find_discrepancies(report_coin, distributor_state)
}
Monitoring and Metrics​
Performance Monitoring​
VALIDATOR_METRICS = {
# Epoch Processing metrics
"epochs_processed": Counter("epochs successfully processed"),
"witness_coins_collected": Counter("total WitnessCoins collected"),
"witness_coins_validated": Counter("WitnessCoins validated"),
"rewards_packages_created": Counter("RewardsPackages created"),
"report_coins_created": Counter("RewardsReportCoins created"),
"epoch_processing_time": Histogram("time to process epoch"),
# Validation metrics
"proof_validation_time": Histogram("time to validate proofs"),
"selection_proof_failures": Counter("invalid selection proofs"),
"physical_access_failures": Counter("invalid physical access proofs"),
# Network metrics
"peer_validators": Gauge("connected validator peers"),
"blockchain_sync_lag": Gauge("blocks behind network"),
"witness_coin_scan_time": Histogram("time to scan for WitnessCoins"),
# Resource metrics
"cpu_usage": Gauge("CPU utilization %"),
"memory_usage": Gauge("Memory utilization %"),
"disk_io": Gauge("Disk IOPS"),
"network_bandwidth": Gauge("Mbps in/out"),
# Economic metrics
"total_reward_weight": Gauge("sum of all reward weights"),
"unique_dig_nodes": Gauge("unique DIG nodes in rewards"),
"unique_witnesses": Gauge("unique witnesses in rewards"),
}
Configuration​
# validator.yaml
validator:
id: "val_abc123"
cluster:
size: 3
threshold: 0.67
epoch_processing:
epoch_duration_blocks: 4608 # ~24 hours
processing_grace_period: 288 # ~1.5 hours after epoch end
witness_coin_batch_size: 100
parallel_validation_threads: 8
blockchain:
chia_rpc_endpoint: "https://localhost:8555"
chia_certificate_path: "/certs/chia.crt"
chia_key_path: "/certs/chia.key"
scan_batch_size: 100
rewards:
distributor_address: "0x..."
report_coin_puzzle_hash: "0x..."
weight_factors_config: "/config/weight_factors.yaml"
min_witnesses_per_epoch: 10
max_reward_recipients: 10000
network:
p2p_port: 8446
rpc_port: 8547
bootstrap_validators:
- "val1.dig.net:8446"
- "val2.dig.net:8446"
security:
multisig_threshold: 2
multisig_signers: 3
tls_cert_path: "/certs/validator.crt"
tls_key_path: "/certs/validator.key"
enable_rate_limiting: true
max_requests_per_minute: 1000
monitoring:
metrics_port: 9092
enable_profiling: true
log_level: "info"
alert_webhook: "https://alerts.dig.net/webhook"
high_availability:
enable_failover: true
health_check_interval: 5
state_backup_interval: 300
backup_locations:
- "s3://dig-backups/validator/"
- "/backup/validator/"
Future DAO Integration​
Transition Plan​
class DAOTransition:
"""
Phase 1: Current (Multisig)
- Curated validator selection
- Manual weight factor updates
- Community oversight of reward distribution
Phase 2: DAO Launch (Post-Chia DAO)
- On-chain validator election
- DAO-governed weight factors
- Automated epoch processing
Phase 3: Full Automation
- Smart contract-based WitnessCoin validation
- Autonomous reward distribution
- Minimal governance needed
"""
async def phase2_integration(self):
# DAO-based validator selection
validators = await self.dao.get_elected_validators()
# Weight factors via governance
weight_factors = await self.dao.get_reward_weight_factors()
# Automated epoch processing
if self.is_epoch_boundary():
epoch = self.get_current_epoch()
await self.process_epoch_with_dao_oversight(epoch, validators)
# Performance monitoring
validator_performance = await self.monitor_epoch_processing()
if validator_performance.below_threshold():
await self.dao.propose_validator_replacement(validator_id)
Related Documentation​
- DIG Node - Storage providers validated by witnesses
- Witness Miner - Creates WitnessCoins for validator processing
- Rewards Distributor - Receives RewardsPackages
- DAO Governance - Future governance model
On-Chain Artifacts Created​
- WitnessCoins - Collected from witness miners (input)
- RewardsPackages - Submitted to rewards distributor (output)
- RewardsReportCoins - On-chain receipts for transparency (audit trail)
Data Flow​
Epoch N: Witness Miners → Create WitnessCoins
↓
Epoch N+1 Start: Validators → Collect WitnessCoins
↓
Validate Proofs
↓
Calculate Weights
↓
Create RewardsPackage
↓
├→ Submit to Rewards Distributor
└→ Create RewardsReportCoin (on-chain receipt)
↓
Community can verify distributions