Scoring Methodology — Obsidian Plugin Observer

Each plugin is classified into a privacy level based on its data-handling patterns, then scored within that level based on five factors.

Privacy Levels

Privacy levels describe a plugin's overall data-handling posture. The level is determined by what the plugin does with data — whether it stays local, goes to known services, or reaches unrecognized destinations.

Level	Label	Score	What Triggers It
0	Private	90–100	No network activity detected, or only fetches data from recognized hosts
1	Connected	75–89	Fetches data from unrecognized hosts
2	Shares Data	60–74	Sends data to recognized hosts (CDN, code hosting, cloud services)
3	Sends to Unknown	30–59	Sends data to unrecognized hosts
4	Obscured	0–29	Code obfuscation techniques (hex-encoded strings, base64 patterns) combined with network activity, OR eval()/Function() with unverified input sources combined with network activity. eval() used for user-written queries (e.g., DataviewJS) does not trigger this level.

Scoring Factors

Within each privacy level, five factors determine the exact score. Each factor has a weight and deducts from the top of the level's score range based on severity. Click a factor to see how severity is determined.

How broadly the plugin reads vault data, modulated by whether data flows outbound

Confidence that vault data flows to network

How often network calls are made

Number of distinct unrecognized external hosts contacted

Number of npm dependencies

How Findings Are Weighted

Each finding is classified as a behavior (the plugin demonstrably does something) or a capability (the plugin could do something). Behaviors weigh more heavily in scoring. Behaviors include sending data to hardcoded URLs, path traversal patterns, and taint-confirmed exfiltration chains. Capabilities include user-configured endpoints, eval() for user-written code, and filesystem imports without dangerous patterns. Only behavior findings count toward the Outbound Destinations scoring factor.

Confidence

The analysis tracks whether vault data actually flows to network endpoints using taint tracking. The confidence of that determination affects how much the scoring factors can deduct:

ConfirmedDefinite data flow detected. Full deduction budget applies.

LikelyProbable data flow. Deduction budget reduced to 50%.

PossibleSpeculative data flow. Deduction budget reduced to 33%.

The more certain the analysis is that vault data reaches a network endpoint, the more it affects the final score. Speculative findings have less impact than confirmed ones.

Build Verification

We reproduce each plugin's build from source and compare the output to the published release. The result affects the score:

VerifiedBuild output matches the release exactly. No score change.

UnverifiableNo score change — we can't determine whether divergence is from dependency drift or tampering. The plugin has no lockfile, so dependency versions may have changed since the release was built.

DivergedBuild output differs from the release and the plugin has a lockfile. This is a strong signal that the published artifact was not built from the published source.