Error pattern tracking for AI agents. Detects corrections, escalates recurring mistakes, learns mitigations. The 'something's off' detector from the AI Brain series.
Security Analysis
medium confidenceThe skill's code and instructions match an error‑tracking purpose, but it reads session transcripts and will invoke arbitrary CLI commands (via ACC_MODELS) which could leak conversation data — and it uses environment variables not declared in the registry metadata.
The scripts implement error detection, logging, escalation, calibration, and resolution exactly as the description claims (reading transcripts, pattern matching, LLM screening, state files under ~/.openclaw/workspace/memory). This functionality is coherent with the skill name and description. Minor mismatch: registry metadata requires 'jq' but most shipped scripts use python3; jq is not obviously needed in the visible files.
Runtime instructions and scripts read session transcripts (~/.openclaw/agents/.../sessions and ~/.openclaw/sessions) and send extracted exchanges to model CLIs configured by ACC_MODELS for classification. That networked LLM invocation is consistent with purpose but is a privacy/exfiltration vector: ACC_MODELS is treated as a comma-separated list of arbitrary CLI commands and each command is invoked with the conversation prompt appended. If ACC_MODELS points to a network-capable CLI (curl, http client, or any custom program), transcripts (user messages) can be sent to external endpoints. The skill also writes persistent state files and logs in the workspace, which is expected behavior.
There is no remote install/download step — the repo provides an install.sh and multiple local scripts that create and update state files. No external archive or IP/shortened URL downloads are used in the provided files. install.sh only creates workspace files and prints a cron command for the user to add; it does not automatically register a system cron by itself.
Registry metadata lists no required env vars, but the runtime expects and uses several environment variables (ACC_MODELS, WORKSPACE, AGENT_ID, possibly others). In particular, ACC_MODELS (not declared in requires.env) controls which CLI commands are invoked with user transcripts. That is a significant discrepancy: an undeclared env var determines external endpoints (via whatever CLI is supplied). The skill requests no credentials, which is appropriate, but the undeclared ACC_MODELS and the ability to call arbitrary CLIs is a proportionality and transparency concern.
The skill writes persistent state (acc-state.json, learned-patterns.json, brain-events.jsonl, watermark files) into the user's workspace and can be scheduled via a cron job (the installer prints the cron command but does not add it silently). It does not request always:true or modify other skills. Autonomous invocation (disable-model-invocation=false) is the platform default — combined with the above ACC_MODELS issue, this increases risk because scheduled/automated runs could repeatedly send transcripts to configured CLIs without per-run approval.
Guidance
This skill generally does what it says (tracks and escalates recurring error patterns), but review these points before installing: - ACC_MODELS risk: The scripts call whatever CLI commands you list in ACC_MODELS and append conversation text as an argument. If ACC_MODELS contains networked tools (curl, http clients) or untrusted binaries, your transcripts can be exfiltrated. Before running, set ACC_MODELS to only trusted, vetted model CLIs (e.g., a local Ollama command or an official provider CLI you control). - Undeclared env var: ACC_MODELS is used by the code but not declared in the registry metadata. Also be aware WORKSPACE and AGENT_ID affect where files are read/written. - Data exposure: The skill reads raw session transcripts (~/.openclaw/agents/.../sessions). If those transcripts contain sensitive user data, the calibration/screening steps will send that content to configured models. Don’t enable cron or run the pipeline on sensitive data until you’re comfortable with the configured models. - Inspect the code & datasets: Look through learned-patterns.json, acc-state.json, and brain-events.jsonl after running to ensure nothing unexpected is recorded. Consider running the scripts in a sandboxed account or with a local-only model configuration first. - jq mismatch: The metadata indicates jq is required but most logic runs in python3; confirm jq is actually needed for your environment. If you want to proceed: set ACC_MODELS explicitly to trusted commands, run encode/quick-check on non-sensitive data, and avoid enabling the cron job until you’ve validated behavior. If you’re unsure, err on the side of not installing or running with production transcripts.
Latest Release
v1.0.0
Initial release: watermark-based error detection, 3-tier cost optimization (regex→Haiku→Opus), self-improving regex calibration, model-agnostic config
More by @ImpKind
Published by @ImpKind on ClawHub
