Plugin Framework
Variable Reference Guides
State Management (state, global_state)

State Management Guide

When to Use State Management

State management is essential for:

  • Tracking plugin progress and counters
  • Caching expensive computations or API responses
  • Sharing data between plugin executions
  • Coordinating between multiple plugins (global_state)
  • Implementing rate limiting and throttling
  • Managing distributed locks and flags

Key Difference: state is private to each plugin, while global_state is shared across all plugins.

Quick Start

Basic Operations

// MVEL - Store and retrieve plugin state
state.put("last_processed_item", itemId);
state.put("request_count", 1000);
 
lastItem = state.get("last_processed_item");
if (state.containsKey("request_count")) {
    count = state.get("request_count");
}

Atomic Counter

// JavaScript - Thread-safe counter
const requestCount = state.increment("api_requests");
if (requestCount > 1000) {
    console.log("Milestone: 1000 API requests processed!");
}

Core Concepts

state vs global_state

Featurestateglobal_state
ScopePlugin-specificShared across all plugins
IsolationComplete - other plugins can't accessNone - all plugins can read/write
Use CasesPlugin config, progress trackingCross-plugin coordination, shared cache
NamespaceAutomatic (plugin name)Global namespace

Operation Categories

  1. Basic Operations - Simple key-value storage
  2. Atomic Counters - Thread-safe numeric operations
  3. Set Operations - Manage collections atomically
  4. Compute Operations - Update values based on current state
  5. Compare-and-Set - Conditional updates for coordination

API Reference

Basic Operations

// Store values
state.put("key", "value");
state.put("counter", 42);
state.put("data", complexObject);
 
// Retrieve values
value = state.get("key");
counter = state.get("counter", 0); // With default
 
// Check existence
if (state.containsKey("key")) {
    // Key exists
}
 
// Remove values
state.remove("key");
 
// Clear all state (use with caution!)
state.clear();

Atomic Counter Operations

These operations are thread-safe and perfect for concurrent environments:

// Increment operations
count = state.increment("counter");        // +1, returns new value
count = state.increment("counter", 5);     // +5, returns new value
 
// Decrement operations
count = state.decrement("counter");        // -1, returns new value
count = state.decrement("counter", 3);     // -3, returns new value
 
// Add and get variations
newVal = state.addAndGet("counter", 10);   // Add 10, return new value
oldVal = state.getAndAdd("counter", 10);   // Return old value, then add 10

Set Operations

Manage collections of unique elements atomically:

// Add elements to set
added = state.addToSet("processed_items", itemId);  // Returns true if new
 
// Remove elements
removed = state.removeFromSet("processed_items", itemId);
 
// Check membership
exists = state.containsInSet("processed_items", itemId);
 
// Get entire set (immutable view)
allItems = state.getSet("processed_items");
 
// Get set size
size = state.setSize("processed_items");

Advanced Compute Operations

Update values based on their current state:

// Compute - update based on current value
state.compute("max_amount", (key, current) -> {
    return Math.max(current ?: 0, newAmount);
});
 
// Merge - combine new value with existing
state.merge("total_volume", amount, (existing, new) -> {
    return (existing ?: 0) + new;
});
 
// Compare and set - atomic conditional update
success = state.compareAndSet("status", "pending", "processing");
 
// Get and set - return old value, set new
oldValue = state.getAndSet("flag", true);

Common Patterns

1. Progress Tracking

// JavaScript - Track processing progress
function processBlocks(blocks) {
    // Get last processed block
    let lastProcessed = state.get("last_block") || 0;
 
    blocks.forEach(block => {
        if (block.number <= lastProcessed) {
            return; // Skip already processed
        }
 
        // Process block
        processBlock(block);
 
        // Update progress atomically
        state.put("last_block", block.number);
        state.increment("blocks_processed");
    });
 
    // Log statistics
    const total = state.get("blocks_processed");
    console.log(`Total blocks processed: ${total}`);
}

2. Rate Limiting

// MVEL - Implement rate limiting
function checkRateLimit(clientId) {
    key = "rate_limit:" + clientId;
    windowKey = "rate_window:" + clientId;
 
    currentWindow = System.currentTimeMillis() / 60000; // 1-minute windows
    lastWindow = state.get(windowKey, 0);
 
    if (currentWindow != lastWindow) {
        // New time window, reset counter
        state.put(windowKey, currentWindow);
        state.put(key, 1);
        return true;
    }
 
    // Same window, check limit
    requests = state.increment(key);
    return requests <= 100; // 100 requests per minute
}

3. Deduplication

// JavaScript - Prevent duplicate processing
function processTransaction(tx) {
    const txHash = tx.getTxHash();
 
    // Atomic check-and-add
    if (!state.addToSet("processed_transactions", txHash)) {
        console.log(`Transaction ${txHash} already processed, skipping`);
        return false;
    }
 
    try {
        // Process the transaction
        const result = doProcessing(tx);
 
        // Track successful processing
        state.increment("successful_txs");
        return true;
 
    } catch (error) {
        // Remove from set on failure to allow retry
        state.removeFromSet("processed_transactions", txHash);
        state.increment("failed_txs");
        throw error;
    }
}

4. Distributed Locking

// MVEL - Simple distributed lock using compare-and-set
function acquireLock(lockName, timeout) {
    lockKey = "lock:" + lockName;
    lockHolder = Thread.currentThread().getName() + ":" + System.currentTimeMillis();
 
    // Try to acquire lock
    if (global_state.compareAndSet(lockKey, null, lockHolder)) {
        return lockHolder; // Lock acquired
    }
 
    // Check if existing lock is expired
    existing = global_state.get(lockKey);
    if (existing != null) {
        parts = existing.split(":");
        lockTime = Long.parseLong(parts[1]);
 
        if (System.currentTimeMillis() - lockTime > timeout) {
            // Lock expired, try to take over
            if (global_state.compareAndSet(lockKey, existing, lockHolder)) {
                return lockHolder;
            }
        }
    }
 
    return null; // Failed to acquire
}
 
function releaseLock(lockName, lockHolder) {
    lockKey = "lock:" + lockName;
    return global_state.compareAndSet(lockKey, lockHolder, null);
}

5. Caching with Expiry

# Python - Cache with TTL
def get_cached_data(key, fetch_function, ttl_seconds=300):
    cache_key = f"cache:{key}"
    timestamp_key = f"cache_time:{key}"
 
    # Check cache validity
    cached_time = state.get(timestamp_key, 0)
    current_time = System.currentTimeMillis() / 1000
 
    if current_time - cached_time < ttl_seconds:
        # Cache is valid
        cached_data = state.get(cache_key)
        if cached_data:
            return cached_data
 
    # Fetch fresh data
    fresh_data = fetch_function()
 
    # Update cache atomically
    state.put(cache_key, fresh_data)
    state.put(timestamp_key, current_time)
 
    return fresh_data

6. Metrics Aggregation

// JavaScript - Aggregate metrics across events
function updateMetrics(event) {
    const amount = event.getAmount();
 
    // Update various metrics atomically
    state.increment("event_count");
    state.merge("total_amount", amount, (old, val) => (old || 0) + val);
 
    // Track maximum
    state.compute("max_amount", (key, current) => {
        return Math.max(current || 0, amount);
    });
 
    // Track unique addresses
    state.addToSet("unique_addresses", event.getAddress());
 
    // Update hourly buckets
    const hour = Math.floor(Date.now() / 3600000);
    state.increment(`hourly_count:${hour}`);
}
 
function getMetricsSummary() {
    return {
        total_events: state.get("event_count", 0),
        total_amount: state.get("total_amount", 0),
        max_amount: state.get("max_amount", 0),
        unique_addresses: state.setSize("unique_addresses")
    };
}

Best Practices

1. Use Atomic Operations for Counters

⚠️

Always use atomic operations (increment, decrement) for counters instead of get/put to avoid race conditions.

// ❌ BAD - Race condition risk
let count = state.get("counter") || 0;
state.put("counter", count + 1);
 
// ✅ GOOD - Atomic operation
const count = state.increment("counter");

2. Namespace Your Keys

// Good key naming prevents collisions
const keys = {
    lastBlock: "processor:last_block",
    apiCache: "cache:api:response",
    rateLimit: `rate:${clientId}:${window}`
};

3. Handle Missing Keys Gracefully

// Always provide defaults or check existence
const config = state.get("config") || getDefaultConfig();
const retries = state.get("retry_count", 0);
 
if (state.containsKey("important_flag")) {
    // Process only if flag exists
}

4. Clean Up Stale Data

// Periodically clean up old data
function cleanupOldData() {
    const cutoff = Date.now() - (24 * 60 * 60 * 1000); // 24 hours ago
 
    // Remove old cache entries
    const cacheKeys = state.getKeys().filter(k => k.startsWith("cache:"));
    cacheKeys.forEach(key => {
        const timestamp = state.get(key + ":time");
        if (timestamp && timestamp < cutoff) {
            state.remove(key);
            state.remove(key + ":time");
        }
    });
}

Performance Considerations

  • Atomic operations are fast and lock-free
  • Set operations maintain uniqueness automatically
  • Large objects in state can impact memory - consider storing IDs and fetching from database
File Operations (files)Configuration Access (env)