disler-observability-hooks
Provides real-time monitoring and tracking capabilities for multi-agent AI systems through event capture, session management, automated summaries, and cost analysis using Python hook scripts.
71
/ 100 · Grade D
D = 60–69
“I need to monitor and track the behavior, costs, and performance of my multi-agent AI systems in real-time with automated insights and session management.”
disler-observability-hooks earned Verified status with a trust score of 71/100 (Grade D). Adversarial testing produced 9 findings (1 critical, 7 high, 1 medium). Security scan flagged 0 findings. Tier is Verified rather than Certified due to unmitigated findings above severity thresholds.
Trust Score Breakdown
Eight weighted signals composing the aggregate trust score
Scheme v2.0 · Weights provisional · Consumer confirmations and uptime use pipeline-derived baselines.
Findings
Security scan results, adversarial testing, and pipeline review
Security Scan — Cisco Skill Scanner
Adversarial Testing — 6 categories, 9 findings
The skill's hooks execute bash commands that incorporate user-provided content without proper sanitization. Multiple hook scripts use subprocess.run() and similar methods to execute shell commands where user input could be interpolated. The most concerning example is in the TTS functionality where user messages are passed directly to shell commands. Additionally, the pre_tool_use.py script processes bash commands from user input and could allow injection of additional shell metacharacters or command chaining.
The skill's hook scripts contain multiple instances of bash commands that use force flags to bypass safety checks and confirmations. Specifically, the pre_tool_use.py script checks for dangerous rm commands but allows them in certain directories, and multiple scripts execute system commands without user confirmation. The notification.py script executes TTS commands with subprocess calls that could potentially be exploited.
The skill's hook scripts access environment variables (ELEVENLABS_API_KEY, OPENAI_API_KEY, ENGINEER_NAME) and execute system-level commands through subprocess calls. The notification.py script determines which TTS system to use based on available API keys and executes external scripts. The pre_compact.py script loads environment variables using dotenv and can backup files to system directories.
The skill implements comprehensive logging hooks that capture and store tool inputs, outputs, session data, and user prompts to local JSON files. While this appears to be for observability purposes, the logging is extensive and could capture sensitive data that passes through tool calls, including API responses, file contents, command outputs, and user inputs.
The skill includes multiple hooks that call 'send_event.py' with various event types and data summarization flags. This script is referenced extensively but not provided in the skill content, making it impossible to verify where this data is being sent. The --source-app parameter suggests data is being transmitted to an external observability system.
The pre_tool_use.py script includes logic to detect and potentially block access to .env files, but the detection logic has gaps and the blocking mechanism uses a 'deny' response that could still log the attempted credential access. The script specifically checks for .env file patterns but allows .env.sample files.
The skill contains multiple conditional branches that depend on environment variables (ELEVENLABS_API_KEY, OPENAI_API_KEY, ENGINEER_NAME) and system state (file existence, random number generation) that users would not anticipate. The TTS selection logic chooses different audio providers based on API keys, and notification behavior varies based on engineer name presence and random chance. These conditionals are not documented in the skill's behavioral guarantees.
The skill declares empty scope and dependencies but contains 11 substantial companion scripts (notification.py, permission_request.py, post_tool_use.py, etc.) totaling thousands of lines of code. These scripts perform significant functionality including TTS audio generation, file system operations, transcript backups, and external API calls that are not declared in the skill metadata.
The skill executes Python scripts using 'uv run' commands that contain dependency declarations in script headers (/// script blocks). These dependencies include python-dotenv and other packages that are not listed in the skill's declared dependencies field, which is empty. This creates implicit package installations that bypass dependency review.
Methodology v1.0 · 6 categories · ~55 attack patterns
Interface
Skill triggers and instruction summary
Activation
Binds to lifecycle events: PreToolUse, PostToolUse, Notification, Stop, SubagentStop, PreCompact, UserPromptSubmit, SessionStart, SessionEnd, PermissionRequest, PostToolUseFailure, SubagentStart
Hook configuration with 24 handler(s)
Scope & Permissions
What this capability can and cannot access — derived from pipeline analysis
no
no
no
yes
yes
no
Badge & Integration
Embed certification status in your README, docs, or CI pipeline
Certification Notes
Provenance observations from the pipeline
Publisher "disler" is not verified — first certification from this publisher
No license file found in repository
No SECURITY.md or SECURITY.txt file found — no published vulnerability reporting process
Single contributor — no peer review evidence in commit history
Package description appears to be boilerplate or template text
Signed Artifact
Certification provenance and verification metadata
Pipeline Artifacts
Raw data files from this certification run — downloadable for independent verification
contract.json
Full unsigned contract
stage1-ingest.json
Ingest stage output
stage2a-sbom.json
SBOM generation results
stage2a-vulns.json
Vulnerability scan results
stage2b-security.json
Security scan results
stage3a-functional.json
Functional test results
stage3b-adversarial.json
Adversarial test results
stage3c-fingerprint.json
Behavioral fingerprint
stage4-certify.json
Certification decision + trust score
stage3a-measurements.json
Raw functional test measurements
stage3b-measurements.json
Raw adversarial test measurements
run-log.json
Pipeline execution log
Not all files may be present for every certification.