One rogue AI command or prompt injection can wipe a filesystem or leak your API keys in seconds. I audited 11,000+ community skills, scrapped the 49% that were dangerous traps, and hooked hard OS firewalls directly into three agent harnesses—blocking rogue execution with zero breaches across 241 production tools.
Giving autonomous LLMs raw terminal access and API keys without guardrails is a recipe for disaster. The industry standard of 'model self-policing'—putting "Please don't delete files" into a system prompt—is a complete joke. Hidden prompt injections in untrusted web pages, sneaky tracking URLs in open-source tools, and model hallucinations easily bypass prompt instructions. Real security requires deterministic, out-of-band execution firewalls that kill dangerous commands before they ever reach the operating system.
Hallucinated or poisoned models run destructive shell commands like rm -rf /, del /s /q, git push --force, or unconstrained database drops. A single bad command wipes uncommitted engineering work or corrupts remote repositories.
Agents reading project directories vacuum up .env files, id_rsa SSH keys, and cloud tokens, leaking private credentials into external third-party LLM inference logs or outbound HTTP endpoints.
Malicious instructions hidden in fetched web pages, scraped HTML comments, or dependency READMEs hijack the agent's reasoning loop, tricking the model into executing secondary unauthorized actions under elevated developer permissions.
Third-party Model Context Protocol (MCP) servers and external GitHub repos push silent upstream updates, injecting unauthorized background daemons, crypto-miners, or telemetry beacons into previously trusted tools.
| Security Dimension | Conventional "Prompt-Guarded" Agents | Karol's Zero-Trust Architecture |
|---|---|---|
| Enforcement Layer | System prompt instructions ("Please do not delete files") | Out-of-band PreToolUse hooks compiled in native PowerShell / OS |
| Failure Mode | Probabilistic; fails under jailbreaks, long context, or role-play | Deterministic; hard-blocked at exit code 2 before shell executes |
| Secret Protection | Agent is asked politely to ignore secrets; files remain readable | Filesystem regex deny interceptor hard-blocks all .env*, ~/.ssh/ access |
| Subagent Tool Scoping | Monolithic context where every subagent gets full bash/write access | Strict Role-Based Access Control (RBAC); research agents have zero shell tools |
| Supply-Chain Upgrades | Blind auto-pulls and untracked git submodules from external repos | Radar v2: Automated AST delta briefs, immutable staging, and human sign-off |
Pre-commit token checks, mojibake audits, hardcoded URL scrapers, and frontmatter schema linters. Unapproved tools never enter the library.
Zero vendor lock-in. Single source of truth (SKILL.md + _arsenal.yml sidecar) rendered deterministically for Claude Code, DSH, and Antigravity.
Native OS hooks intercept tool calls out-of-band: security-gate.ps1 kills destructive shell commands; secrets-guard.ps1 guards keys.
Monitors 25+ upstream GitHub repos, flags unauthorized changes via AST delta briefs, and prevents unvetted code from polluting production systems.
Before any skill, agent, or MCP server is allowed into the production library, it must survive a multi-stage static analysis pipeline. We scan for unpinned dependencies, suspicious dynamic evals, hardcoded URLs, and broken mojibake encoding. Out of 11,000+ candidate files audited, 5,363 were permanently rejected. Here is the automated taxonomy that caught them.
| Threat Classification | Rejection Share | Automated Trigger Condition | Defect Rationale & Exploit Vector |
|---|---|---|---|
| Hardcoded Runtime URLs | 41% (2,198 files) | WebFetch, curl, wget, Invoke-WebRequest to hardcoded domains | Deferred attack vector: External domain owner can redirect endpoints post-install to exfiltrate private corporate data or inject rogue instructions. |
| Arbitrary Dynamic Execution | 28% (1,501 files) | eval(), Invoke-Expression, $(...), backtick command interpolation | Remote code execution: Constructs and executes arbitrary shell commands from unvalidated model strings at runtime. |
| Secret & Credential Probing | 16% (858 files) | Explicit regex matches for .env, ~/.ssh/, ~/.aws/, ~/.claude.json | Credential harvesting: Ingests infrastructure secrets and transmits them via prompt context or outbound webhooks. |
| Token Bloat & Toy Prompts | 11% (590 files) | Unconstrained prose >4000 tokens, conversational banter, zero tests | Operational degradation: Dilutes attention context, triggers model hallucinations, and burns unnecessary inference tokens with zero automated tests. |
| Mojibake & Encoding Defects | 4% (216 files) | Malformed UTF-8 bytes, double-encoded cp1252 byte sequences | Syntax corruption: Mangled quotation marks and dashes corrupt shell arguments and parse trees silently during execution. |
Production PowerShell linter executed before any candidate skill or agent is staged into the vault. Hard-fails on encoding corruption or harness-specific leaks.
Execution Directive: Scans candidate packages for harness-specific token leaks and double-encoded cp1252 mojibake artifacts. Immediately halts ingest pipeline with exit code 1 if defects are found.
# Extract from scripts/audit-arsenal.ps1 (Universal Harness-Neutrality Gate & Mojibake Check)
$Script:CcTokens = @('Read','Edit','NotebookEdit','Write','Glob','Grep','Bash','Agent','Task','WebSearch','WebFetch','AskUser')
$Script:AgTokens = @('read_file','edit_file','write_file','run_command','execute_script','search_web','ask_question','fetch_web_page')
# 1. Harness Neutrality Linter: Artifacts must never leak vendor-specific tokens
function Test-HarnessNeutrality {
param([string]$FilePath, [string]$Content)
foreach ($token in ($Script:CcTokens + $Script:AgTokens)) {
if ($Content -match "\b$token\b" -and $FilePath -notmatch "(_arsenal\.yml|references/)") {
Write-Error "VIOLATION: Vendor token '$token' leaked into canonical artifact: $FilePath"
return $false
}
}
return $true
}
# 2. Mojibake Gate: Catches double-encoded cp1252 byte corruption before commit
function Test-MojibakePurity {
param([string]$FilePath, [string]$Content)
# Detects common cp1252 byte misinterpretations (’, “, â€, é)
$mojibakePattern = "[\xC2-\xDF][\x80-\xBF]|â€|ÃÂ"
if ($Content -match $mojibakePattern) {
Write-Error "ENCODING DEFECT: Mojibake byte artifacts detected in $FilePath"
return $false
}
return $true
}
# Gate mode enforces exit 1 on any frontmatter error or encoding defect
if (-not (Test-HarnessNeutrality $TargetFile $FileContent) -or -not (Test-MojibakePurity $TargetFile $FileContent)) {
exit 1
}
The visual disparity between naive industry prompts and production engineering: a fragile 20-line single-file prompt versus an unabridged 46-file deterministic state machine with automated linters, language playbooks, and 11 specialist checklists.
How Most Teams Do It: Drop a single loose 20-line markdown file into a prompt directory. Give the LLM full terminal access and ask it nicely to "Review code and fix all bugs."
# Generic Toy Skill (Single File: prompts/code-reviewer.md) Role: You are an expert code reviewer. Instructions: - Read all files in the current repository. - Identify all security vulnerabilities, performance issues, and bugs. - Fix all issues directly in place using your tools. - Ensure the code passes all tests. - Commit and push your changes when done. - If anything fails, try again until it succeeds. # Why This Fails in Production: # 1. Zero state tracking; models loop infinitely on syntax errors. # 2. No regression tests; fixes silently break other modules. # 3. Model self-policing; easily tricked by malicious comments. # 4. Unbounded bash writes wipe uncommitted developer files.
How Karol Builds It: An unabridged 46-file orchestrated engine pairing a 13-stage deterministic state machine with isolated subagents, test suites, and audit logs. Every file serves an explicit operational purpose across root analysis, language patterns, reproducible test replays, and 11 domain specialist review checklists.
review-fix-technical-code/ (Unabridged 46-File Production Subsystem Tree)
├── SKILL.md # 634-line core engine: 6 operational modes & state machine orchestrator
├── _arsenal.yml # Package metadata, semver, provenance & multi-harness target mappings
├── changelog.md # Immutable audit ledger of subsystem modifications across releases
├── hitl-loop.template.ps1 # Human-in-the-loop checkpoint runner (PowerShell / Windows environment)
├── hitl-loop.template.sh # Human-in-the-loop checkpoint runner (POSIX Bash / Unix fallback)
├── analysis/ # Deterministic Diagnostics & Fault Isolation Engines (2 files):
│ ├── root-cause.md # 5-Whys systematic failure hypothesis builder and diagnostic matrix
│ └── stack-trace.md # Exception frame demangling & source line correlation analyzer
├── patterns/ # Technology-Specific Audit & Fix Playbooks (9 files):
│ ├── database.md # Connection pooling, transaction boundaries & query deadlock recovery
│ ├── docker.md # Multi-stage cache invalidation, signal traps (SIGTERM) & hygiene
│ ├── git.md # Merge conflicts, rebase branch drift & detached HEAD state recovery
│ ├── go.md # Goroutine leaks, channel deadlocks & memory profiling playbooks
│ ├── network.md # DNS resolution timeouts, TLS handshake drops & socket exhaustion
│ ├── nodejs.md # Event-loop lag, unhandled promise rejections & stream backpressure
│ ├── python.md # GIL bottlenecks, async event-loop blocking & type drift mitigations
│ ├── react.md # Hook dependency cycles, hydration mismatches & re-render churn
│ └── rust.md # Lifetime bounds, borrow checker conflicts & unsafe block audits
├── replay/ # Regression Prevention & Solution Harvesting (2 files):
│ ├── README.md # Replay engine contract & fixture reproduction execution guide
│ └── solution-template.yaml # Declarative failure replay & regression test specification fixture
├── scripts/ # Automated Verification & Static Metric Tooling (2 files):
│ ├── lint_runner.py # Multi-linter execution pipeline with unified exit code aggregation
│ └── type_coverage.py # Strict TypeScript / MyPy type coverage metric enforcement gate
├── sections/ # Modular Operational Procedures & Triage Gates (14 files):
│ ├── app-qa.md # Full-stack manual & automated verification runbooks
│ ├── checklist.md # Universal pre-fix diagnostic and post-fix validation gate
│ ├── ci-failure-playbook.md # Deterministic CI runner log triage & failure isolation procedure
│ ├── cold-recheck-prompts.md # Zero-context adversarial prompts for cold audit reviewer agents
│ ├── design-checklist.md # UI visual regression, WCAG accessibility & token fidelity checks
│ ├── detect-base-branch.md # Git branch topology detection & merge-base identification
│ ├── diff-analysis.md # AST-aware diff inspection for unintended side effects
│ ├── greptile-triage.md # External AI code-review comment parsing & normalization
│ ├── live-bug.md # Emergency production defect stabilization & hotfix protocol
│ ├── pre-merge-gate.md # 4-stage blocking gate before branch merge authorization
│ ├── qa-health-score.md # Quantitative code health & risk scoring algorithm
│ ├── regression-test-procedure.md # Regression suite execution & boundary assertion runbook
│ ├── test-bootstrap.md # Test harness environment initialization & container orchestration
│ └── ticket-format.md # Standardized durable issue schema for issue trackers
├── sections/specialists/ # Deep Technical Domain Audit Checklists (11 files):
│ ├── api-contract.md # OpenAPI schema drift, backward compatibility & breaking change guard
│ ├── comments.md # Documentation integrity, stale docstrings & comment drift
│ ├── data-migration.md # Zero-downtime database migrations, rollbacks & lock safety
│ ├── error-handling.md # Exhaustive error propagation, typed errors & fault isolation
│ ├── maintainability.md # Cyclomatic complexity, code duplication & modular separation
│ ├── performance.md # N+1 query detection, memory allocation & hot-path optimization
│ ├── red-team.md # Adversarial injection, privilege escalation & attack vectors
│ ├── security.md # Secret leaks, OWASP Top 10, CORS & input sanitization
│ ├── test-coverage.md # Strict branch & condition coverage verification
│ ├── testing.md # Flaky test mitigation, mock boundaries & assertion quality
│ └── type-design.md # Sound typing, discriminated unions & any/unknown defense
└── templates/ # Audit Documentation & Evidence Output Schemas (1 file):
└── qa-report-template.md # Formal cryptographic evidence & sign-off audit report
evidence.md and immutable test logs verifying every code fix
The most critical layer in the zero-trust architecture operates at runtime. Whenever an agent decides to run a shell command, read a file, or spawn a subagent, the harness triggers a PreToolUse hook. The hook runs out-of-band in the native operating system shell. If the hook returns exit code 2 or a deny decision, execution is immediately aborted before the shell ever touches the filesystem.
Emits structured decision payloads compatible with Google Antigravity, Claude Code, and DeepSeek Harness formats simultaneously.
Execution Directive: Formats intercept decisions into all three major vendor schemas simultaneously. Enables a single PowerShell script to govern Claude Code, Antigravity, and DeepSeek Harness without code duplication.
# Multi-Dialect Hook Output Adapter Function (Production Code: library/hooks/secrets-guard/secrets-guard.ps1)
function Write-Decision {
param([string]$Decision, [string]$Reason)
$hookEventName = if ($payload.hook_event_name) { $payload.hook_event_name } else { "PreToolUse" }
@{
# Dialect 1: Google Antigravity Format
decision = $Decision
reason = $Reason
# Dialect 2: Claude Code Standard Format
permissionDecision = $Decision
permissionDecisionReason = $Reason
# Dialect 3: DeepSeek Harness & Claude Code Nested Spec
hookSpecificOutput = @{
hookEventName = $hookEventName
permissionDecision = $Decision
permissionDecisionReason = $Reason
}
} | ConvertTo-Json -Compress
}
Inspects shell command payloads before execution. Hard-blocks destructive filesystem operations, unconstrained database drops, and forced git branch overwrites.
Execution Directive: Matches proposed shell execution strings against a rigid cross-platform regex denylist. Emits a hard deny and exits with code 2 if any forbidden destructive pattern is detected.
# Production Code: library/hooks/security-gate/security-gate.ps1 (Dual-OS Denylist)
$patterns = @(
# Unix & Cross-Platform Destructive Patterns
'rm -rf', 'rm -fr', 'sudo rm', 'chmod -R 777', 'mkfs', 'dd if=', ':\(\)\{:\|:&\};:',
# Windows PowerShell Destructive Patterns
'Remove-Item\s+(-Recurse|-Force)', 'ri\s+.*(-r|-fo)', 'rd\s+/s\s+/q', 'del\s+/s\s+/q',
'Clear-Disk', 'Format-Volume',
# Git Destructive History & Branch Overwrites
'git push .*(\+|--force|-f\b|--force-with-lease)',
'git reset --hard', 'git clean -fd', 'git clean -f', 'git branch -D',
'git checkout \.', 'git restore \.',
# Database Unconstrained Deletion Patterns
'DROP TABLE', 'DROP DATABASE', 'TRUNCATE TABLE', 'DELETE FROM .* WHERE 1=1'
)
foreach ($pat in $patterns) {
if ($cmd -match "(?i)$pat") {
$msg = "BLOCKED by security-gate: destructive pattern detected ($pat)."
[Console]::Error.WriteLine($msg)
[Console]::Error.WriteLine(" If this is intentional, run it manually in your terminal.")
Write-Decision -Decision "deny" -Reason "$msg Command: $cmd"
if (-not $isAntigravity) { exit 2 }
exit 0
}
}
Prevents autonomous agents from reading or modifying environment files, SSH keys, cloud provider tokens, or harness configuration databases.
Execution Directive: Intercepts file read and write operations. Grants access to harmless documentation templates (.env.example) while hard-blocking all live credential paths.
# Production Code: library/hooks/secrets-guard/secrets-guard.ps1
$envExclusions = @(".env.example", ".env.sample", ".env.template")
# Safe template files are permitted for developer onboarding
if (-not ($envExclusions -contains $basenameFull)) {
# Tier 1 Hard Blocks: Live credentials and cloud secrets
if ($basenameFull -match '^\.env(\..+)?$' -or
$filePath -match '(\.ssh[/\]|\.aws[/\]|\.claude\.json$|id_rsa|id_ed25519)') {
$msg = "BLOCKED by secrets-guard: Sensitive credential path detected ($filePath)."
[Console]::Error.WriteLine($msg)
[Console]::Error.WriteLine(" Never expose infrastructure secrets to LLM context.")
Write-Decision -Decision "deny" -Reason "$msg Path: $filePath"
if (-not $isAntigravity) { exit 2 }
exit 0
}
}
Classifies tool operations into read-only commands (instantly approved for maximum agent velocity) and mutation commands (gated for human confirmation).
Execution Directive: Auto-approves safe filesystem inspection tools across all harnesses while preventing unapproved background file mutations.
# Production Code: library/hooks/smart-permission-filter/smart-permission-filter.ps1
$readonlyTools = @('Read', 'Glob', 'Grep', 'LS', 'read_file', 'list_dir', 'search_files', 'view_file', 'search_web')
# Safe read-only operations receive instantaneous out-of-band approval
if ($readonlyTools -contains $toolName) {
Write-Decision -Decision "allow" -Reason "Read-only exploration tool pre-approved"
exit 0
}
# Any tool attempting disk mutations falls back to human oversight gate
Write-Decision -Decision "ask" -Reason "Write/Execution operation requires human confirmation"
exit 0
In a multi-agent system, giving every agent the full toolkit is an architectural anti-pattern. A codebase researcher has zero business writing to disk or running shell commands. An implementation subagent should be confined to its assigned branch and barred from pushing code or deploying. We enforce a strict Role-Based Access Control (RBAC) matrix that defines exactly what tools each agent is permitted to touch.
| Subagent Role | Permitted Tool Domain | Hard-Stripped / Forbidden Tools | Workspace Boundary & Confinement Standard |
|---|---|---|---|
| researcher | Read-Only | write_file, edit_file, run_command, Bash | Triage sandbox in research/; evaluates external candidate repos without write privileges to production library/. |
| librarian | Vault-Ops | WebSearch, WebFetch, external network endpoints | Strictly local vault operations (library/, index.md, log.md); physically blocked from outbound HTTP calls. |
| code-builder | Task-Scoped Write | git push, git commit, git merge, deploy scripts | Confined to declared task files in isolated git branch; forbidden from staging, committing, or pushing to remote. |
| code-reviewer | Read-Only QA | write_file, edit_file, sed, in-place file mutations | Cold second-opinion reviewer; inspects git diffs and outputs structured findings without authority to touch source code. |
| security-gate | Runtime Hook | Self-mutation, bypassing exit code 2 contract | Out-of-band OS execution interceptor; evaluates shell string syntax before process spawning occurs. |
Open-source AI tools move fast. When an upstream author updates a repository you rely on, pulling changes blindly is how backdoors and prompt injections enter production. Karol built Radar v2—an automated drift-monitoring engine that tracks 25+ upstream GitHub repositories, generates structured AST delta briefs, and requires an explicit human security verdict before code can ever be staged.
Nightly cron runs radar.ps1 scan across all monitored upstream repos, computing cryptographic SHA-256 hashes against stored baselines.
Generates human-readable delta briefs categorizing upstream additions into new capabilities, behavioral modifications, or potential attack surfaces.
The operator reviews the brief and assigns a permanent recorded verdict: clean, skip, absorb, or ticket.
Absorptions stage updates in isolated temporary directories. Full linter suites run before atomically updating the production library.
Production CLI commands for scanning upstream drift, inspecting delta briefs, recording security verdicts, and executing crash-safe atomic absorptions.
Execution Directive: CLI workflow executed to detect, review, and absorb upstream repository changes with complete cryptographic verification and rollback safety.
# 1. Scan monitored upstream repository watch-universe for drift .\scripts\radar.ps1 scan -Repo "https://github.com/DietrichGebert/ponytail" # 2. View generated delta brief and AST changes for a modified skill .\scripts\radar.ps1 diff -Item "ponytail-review" -Verbose # 3. Record human verdict following security review (clean | skip | absorb | ticket) .\scripts\radar.ps1 verdict -Item "ponytail-review" -Action "absorb" -Rationale "Reviewed AST diff; no hardcoded URLs" # 4. Verify staged content against static linters and test suites .\scripts\radar.ps1 verify -Item "ponytail-review" # 5. Finalize promotion, update changelog.md, and bump semver in _arsenal.yml .\scripts\radar.ps1 promote -Item "ponytail-review"
Security isn't hypothetical; it's proven in the trenches. Over months of auditing candidate packages, our static linters and PreToolUse hooks caught real vulnerabilities in highly popular open-source packages before they could touch our developer environments.
Defect Caught: External plugin contained unvetted background webhooks phoning home on every model invocation, transmitting user query metadata to unauthorized telemetry servers.
Zero-Trust Action: Hard-blocked at Static Linter Gate (Ring 01). Telemetry code was permanently purged before the tool was approved.
Defect Caught: Third-party scraper parsed webpage scripts using eval(), creating an immediate indirect prompt-injection exploit allowing malicious websites to run arbitrary local shell commands.
Zero-Trust Action: Permanently rejected into rejected/ with forensic rationale recorded.
| Operational Risk Category | Unconstrained Agent Installation | Karol's Zero-Trust Control Plane |
|---|---|---|
| Engineering Data & Secrets | High Risk (Unvetted .env & SSH key access) |
Zero Exposure (Regex denylists hard-block access) |
| System Integrity | Critical Risk (Unrestricted destructive shell commands) | 100% Protected (Dual-OS denylists kill destructive syntax) |
| Supply-Chain Upgrades | Unmonitored (Silent background updates introduce vulnerabilities) | Cryptographically Audited (Radar v2 AST delta briefs & human gate) |
| Audit Trail & Compliance | None (Ephemeral conversational sessions) | Immutable (50,787 vault files versioned in Git with changelogs) |
System prompts are advisory; operating system hooks are mandatory. Never trust an LLM to follow "security guidelines" when a single line of out-of-band PowerShell regex can prove compliance deterministically.
If an agent is meant to research code or read documentation, don't give it bash tools. If an agent is meant to edit files, don't give it permission to push to production. Least privilege prevents 99% of accidents.
Open-source AI packages change overnight. Blindly running git pull on external agent repositories is an open invitation for supply-chain attacks. Always inspect AST delta briefs before promoting code.