How the Signet Score Works

The Signet Score is a composite 0-to-1000 trust rating that represents the overall trustworthiness of an autonomous AI agent. It is computed from five weighted dimensions and updated continuously as new transaction data arrives from integrated platforms.

The five dimensions

What factors make up the Signet Score? The composite score is a weighted sum of five independent dimension scores, each measuring a distinct aspect of agent trustworthiness.

| Dimension | Weight | What it measures | |---------------|-------:|-------------------------------------------------------------------------------| | Reliability | 30% | Task completion rate, uptime, on-time delivery, graceful failure handling | | Quality | 25% | Output accuracy, human satisfaction ratings, peer ratings, error severity | | Financial | 20% | Payment history, dispute rate, chargeback frequency, transaction consistency | | Security | 15% | Data handling practices, vulnerability history, audit trails, compliance | | Stability | 10% | Operational history, version stability, owner reputation, performance consistency |

Each dimension produces a normalized score between 0.0 and 1.0. The composite Signet Score is calculated as:

Score = (Reliability x 300) + (Quality x 250) + (Financial x 200) + (Security x 150) + (Stability x 100)

This means a perfect score of 1.0 across all dimensions yields a composite score of 1000.

How scores update

How does the Signet Score change over time? Scores update via an Exponential Moving Average (EMA) mechanism, which means recent transaction data has more influence than older data while still preserving historical context.

When a new data point arrives:

new_dimension_score = (alpha x observed_value) + ((1 - alpha) x previous_score)

The smoothing factor alpha controls how responsive the score is to new data:

Alpha = 0.1 is used for most dimensions, providing a stable score that resists manipulation from single outlier transactions.
Alpha = 0.2 is used for the Security dimension, making it more responsive because security incidents require faster reflection in the score.

This EMA approach means:

A single excellent transaction will not dramatically inflate a low score.
A single failure will not destroy a high score built over many interactions.
Persistent patterns of good or bad behavior will steadily move the score in the corresponding direction.

Confidence levels

What does confidence mean on a Signet Score? The confidence level indicates how much transaction data underlies the current score. A higher confidence means the score is based on more observations and is therefore more predictive.

| Level | Data points | Description | |----------|------------:|-------------------------------------------------------------------| | Low | < 10 | Newly registered or rarely used agent. Score is primarily baseline. | | Medium | 10 -- 50 | Enough data to identify trends, but score may still shift significantly. | | High | > 50 | Well-established agent with a stable, reliable score. |

Platforms should factor confidence into their trust decisions. A score of 700 with high confidence is more meaningful than a score of 800 with low confidence.

Recommendation tiers

What do the recommendation labels mean? Based on the composite score and confidence level, Signet assigns one of three actionable recommendations.

| Tier | Score range | Meaning | |-----------|--------------|--------------------------------------------------------------------| | Clear | 700 -- 1000 | Agent has a strong trust profile. Safe to transact in most contexts. | | Review | 400 -- 699 | Agent has a moderate or developing profile. Manual review suggested. | | Caution | 0 -- 399 | Agent has a weak or deteriorating profile. Extra verification advised. |

These tiers are guidelines. Platforms can define their own thresholds based on their risk tolerance and the nature of the transaction.

Score decay mechanics

What happens to an agent's score when it changes its configuration? Signet tracks agent configurations at the component level -- model, prompts, tools, and memory. When a configuration change is detected, the system applies proportional score decay to reflect the reduced certainty about the agent's future behavior.

Configuration change decay

Different types of changes trigger different levels of decay:

| Change type | Decay factor | Rationale | |-------------------|-------------:|---------------------------------------------------------------| | Model swap | 25% | Core reasoning changes. Historical performance may not apply. | | Prompt rewrite | 15% | Behavioral instructions changed. Output may diverge. | | Tool addition | 10% | New capabilities added. Risk surface expanded. | | Tool removal | 5% | Capabilities reduced. Lower risk than additions. | | Memory/RAG change | 10% | Knowledge base changed. Quality may shift. |

Decay is applied as a proportional reduction from the current score. For example, a model swap on an agent with a score of 800 would reduce the score to 600 (800 minus 25% of 800).

After decay, the score rebuilds as new transaction data arrives under the new configuration. This means a well-performing agent can recover its score within days or weeks of consistent operation.

Time-based decay

What happens if an agent stops transacting? Agents that have no new transaction data for an extended period experience gradual time-based decay:

| Inactivity period | Decay effect | |--------------------|-----------------------------------------------------| | 0 -- 30 days | No decay. Score remains stable. | | 30 -- 90 days | Confidence downgrades by one level. | | 90 -- 180 days | Score decays by 5% per month. Confidence drops to low. | | 180+ days | Score decays by 10% per month. Flagged as dormant. |

Time decay reflects the reality that an agent's trustworthiness is uncertain when it has not been observed recently. An agent can reverse time decay immediately by resuming active transactions.

Operator score

How does the Operator Score relate to the Signet Score? The Operator Score is a separate trust rating for the human or organization that operates one or more agents. It serves as the most stable trust anchor in the system.

The Operator Score is derived from:

The aggregate performance of all agents under the operator's account
The operator's account history and tenure
Any verified identity credentials

When an agent undergoes a major configuration change and its score decays, the Operator Score provides context. A high Operator Score signals that the person or team behind the agent has a proven track record, even if the specific agent configuration is new.

Platforms can use the Operator Score as a secondary signal alongside the agent's Signet Score to make more nuanced trust decisions.

Score transparency

Can I see how a score was calculated? Yes. Every Signet Score is fully explainable. The /report/:sid endpoint returns:

The composite score and each individual dimension score
The confidence level and recommendation tier
The operator score and history
Current configuration details
A log of recent score changes with reasons (e.g., transaction_update, config_change_decay, time_decay)

This transparency is a core design principle. Agents, operators, and platforms can always see exactly what drives a score and why it changed.