Skip to main content
OrchestKit v6.7.1 — 67 skills, 38 agents, 77 hooks with Opus 4.6 support
OrchestKit
Skills

Feedback

Manages OrchestKit usage analytics, learning preferences, and privacy settings. Use when reviewing patterns, pausing learning, or managing consent.

Command low

Feedback - Manage Learning System

View and manage the OrchestKit feedback system that learns from your usage.

Overview

  • Checking feedback system status
  • Pausing/resuming learning
  • Resetting learned patterns
  • Exporting feedback data
  • Managing privacy settings
  • Enabling/disabling anonymous analytics sharing
  • Viewing privacy policy

Usage

/ork:feedback                    # Same as status
/ork:feedback status             # Show current state
/ork:feedback pause              # Pause learning
/ork:feedback resume             # Resume learning
/ork:feedback reset              # Clear learned patterns
/ork:feedback export             # Export feedback data
/ork:feedback settings           # Show/edit settings
/ork:feedback opt-in             # Enable anonymous sharing
/ork:feedback opt-out            # Disable anonymous sharing
/ork:feedback privacy            # View privacy policy
/ork:feedback export-analytics   # Export anonymous analytics for review

Subcommands

SubcommandDescription
status (default)Show current feedback system state, learned patterns, agent performance
pauseTemporarily pause learning without clearing data
resumeResume paused learning
resetClear all learned patterns (requires "RESET" confirmation)
exportExport all feedback data to .claude/feedback/export-\{date\}.json
settingsShow/edit settings (usage: /ork:feedback settings <key> <value>)
opt-inEnable anonymous analytics sharing (GDPR-compliant consent)
opt-outDisable anonymous analytics sharing (revokes consent)
privacyDisplay the full privacy policy
export-analyticsExport anonymous analytics to file for review before sharing

Output: Each subcommand displays formatted status, confirmation prompts, or exported file paths. See Subcommand Reference for detailed actions and expected output for each subcommand.

See Consent and Security Rules for GDPR consent management, security restrictions, and analytics data sharing policies.

  • ork:skill-evolution: Evolve skills based on feedback

File Locations

See File Locations for storage details.

See Privacy Policy for full privacy documentation.


Rules (1)

Consent Management and Security Rules

Consent is managed per GDPR requirements:

  1. Explicit opt-in required - No data shared until you actively consent
  2. Audit trail - All consent actions logged in consent-log.json
  3. Easy revocation - Opt-out is as easy as opt-in
  4. Version tracking - Consent version tracked for policy changes

Security Note

The following commands are NEVER auto-approved regardless of learning:

  • rm -rf, sudo, chmod 777
  • Commands with --force or --no-verify
  • Commands involving passwords, secrets, or credentials

Analytics Data Sharing

Anonymous analytics include only aggregated, non-identifiable data:

Data TypeWhat's IncludedWhat's NOT Included
SkillsUsage counts, success ratesFile paths, code content
AgentsSpawn counts, success ratesProject names, decisions
HooksTrigger/block countsCommand content, paths

The following are blocked by design and never included in analytics:

  • Project names, paths, or directory structure
  • File contents, code, or diffs
  • Decision content or context
  • User identity, email, or credentials
  • Memory data
  • URLs, IP addresses, or hostnames
  • Any strings that could identify the project or user

Incorrect — Auto-approving dangerous commands:

# Pre-push hook auto-approves all commands
git push --force origin main  # DANGEROUS
rm -rf /important/directory    # DESTRUCTIVE

Correct — Blocking dangerous commands from auto-approval:

# Security check in hook
if [[ "$cmd" =~ (rm -rf|sudo|chmod 777|--force|--no-verify) ]]; then
  echo "BLOCKED: Command requires manual approval"
  exit 1
fi

References (3)

File Locations

Feedback File Locations

All feedback data is stored locally in the project.

Storage Paths

.claude/feedback/
├── preferences.json       # User preferences and settings
├── metrics.json           # Skill and agent usage metrics
├── learned-patterns.json  # Auto-approve patterns and code style
├── satisfaction.json      # Session satisfaction tracking
├── consent-log.json       # GDPR consent audit trail (#59)
├── analytics-sent.json    # Transmission history (if enabled)
└── analytics-exports/     # Anonymous analytics exports for review
    └── analytics-export-*.json

File Descriptions

preferences.json

{
  "version": "1.0",
  "enabled": true,
  "learnFromEdits": true,
  "learnFromApprovals": true,
  "learnFromAgentOutcomes": true,
  "shareAnonymized": false,
  "consentedAt": null,
  "consentVersion": null,
  "syncGlobalPatterns": true,
  "retentionDays": 90,
  "pausedUntil": null
}

Records all consent actions for GDPR compliance. Created by consent-manager.sh.

{
  "version": "1.0",
  "events": [
    {
      "action": "granted",
      "version": "1.0",
      "timestamp": "2026-01-14T10:00:00Z"
    },
    {
      "action": "revoked",
      "timestamp": "2026-01-15T14:00:00Z"
    }
  ]
}

Event Actions:

  • granted - User opted in (includes policy version)
  • declined - User said "No Thanks" on first prompt
  • revoked - User opted out after previously opting in

metrics.json

{
  "version": "1.0",
  "updated": "2026-01-14T10:00:00Z",
  "skills": { ... },
  "hooks": { ... },
  "agents": { ... }
}

learned-patterns.json

{
  "version": "1.0",
  "updated": "2026-01-14T10:00:00Z",
  "permissions": { ... },
  "codeStyle": { ... }
}

analytics-export-*.json

Anonymous analytics exports for user review before sharing.

{
  "timestamp": "2026-01-14",
  "plugin_version": "4.12.0",
  "skill_usage": {
    "<skill-id>": { "uses": 12, "success_rate": 0.92 }
  },
  "agent_performance": {
    "<agent-id>": { "spawns": 8, "success_rate": 0.88 }
  },
  "hook_metrics": {
    "<hook-name>": { "triggered": 45, "blocked": 3 }
  },
  "summary": {
    "unique_skills_used": 8,
    "unique_agents_used": 3,
    "hooks_configured": 5,
    "total_skill_invocations": 45,
    "total_agent_spawns": 12
  },
  "metadata": {
    "exported_at": "2026-01-14T12:00:00Z",
    "format_version": "1.0",
    "note": "Review before sharing"
  }
}

Gitignore

All files in .claude/feedback/ are gitignored by default.

Add to your .gitignore:

.claude/feedback/

Global Patterns

Cross-project patterns are stored at:

~/.claude/global-patterns.json

This file is shared across all projects when syncGlobalPatterns is enabled.

Scripts

Manages GDPR-compliant consent for anonymous analytics.

.claude/scripts/consent-manager.sh

Key functions:

  • has_consent() - Check if user has consented
  • has_been_asked() - Check if user was ever prompted
  • record_consent() - Record opt-in with timestamp
  • record_decline() - Record "No Thanks" response
  • revoke_consent() - Record opt-out (revocation)
  • get_consent_status() - Get full consent state as JSON
  • show_opt_in_prompt() - Display interactive prompt
  • show_privacy_policy() - Display full privacy policy
  • show_consent_status() - Display human-readable status

analytics-lib.sh

Prepares anonymous analytics data.

.claude/scripts/analytics-lib.sh

Key functions:

  • prepare_anonymous_report() - Aggregate anonymized data
  • get_shareable_metrics() - Get only safe-to-share metrics
  • validate_no_pii(data) - Verify no PII in data
  • export_analytics(filepath) - Export to file for user review
  • opt_in_analytics() - Enable anonymous sharing
  • opt_out_analytics() - Disable anonymous sharing

analytics-sender.sh (#59 - Optional)

Handles optional network transmission (disabled by default).

.claude/scripts/analytics-sender.sh

Key functions:

  • is_transmission_enabled() - Check if network send is enabled
  • send_analytics_report() - Send to configured endpoint
  • show_transmission_status() - Show send history

Note: Network transmission requires:

  1. User consent (has_consent() = true)
  2. Endpoint configured (ORCHESTKIT_ANALYTICS_ENDPOINT)
  3. Network enabled (enableNetworkTransmission in preferences)

Schemas

.claude/schemas/
├── feedback.schema.json   # Metrics, patterns, preferences schemas
└── consent.schema.json    # Consent log schema (#59)

Privacy Policy

Privacy Policy Reference

Anonymous analytics privacy documentation for OrchestKit Claude Plugin

Overview

OrchestKit offers optional anonymous analytics to help improve the plugin. This is strictly opt-in - no data is collected without explicit user consent.

What We Collect

When you opt in, we collect only aggregated, non-identifiable metrics:

Data TypeExamplePurpose
Skill usage counts"api-design: 45 uses"Identify popular skills
Skill success rates"92% success"Improve underperforming skills
Agent spawn counts"backend-architect: 8 spawns"Agent utilization
Agent success rates"88% tasks completed"Agent effectiveness
Hook trigger counts"git-branch-protection: 120 triggers"Hook tuning
Hook block counts"5 commands blocked"Security effectiveness
Plugin version"4.12.0"Version adoption
Report date"2026-01-14"Trend analysis

What We Never Collect

The following are explicitly blocked from collection:

  • Your code or file contents
  • Project names, paths, or directory structure
  • User names, emails, or personal information
  • IP addresses (stripped at network layer)
  • Memory data or conversation history
  • Architecture decisions or design documents
  • API keys, tokens, or credentials
  • Git history or commit messages
  • Session IDs or user identifiers
  • Any data that could identify you or your projects

PII Detection

Before any data export, we scan for:

# Blocked patterns (excerpt from analytics-lib.sh)
/Users/, /home/, /var/, /tmp/    # File paths
@                                 # Email indicators
http://, https://                 # URLs
password, secret, token, api_key  # Credentials
username, user_id, email          # Identifiers
[0-9]{1,3}\.[0-9]{1,3}...        # IP addresses

If any PII pattern is detected, the export is aborted.

# Via command
/ork:feedback opt-in

# What happens:
# 1. consent-log.json records: {"action": "granted", "version": "1.0", "timestamp": "..."}
# 2. preferences.json sets: shareAnonymized = true
# Via command
/ork:feedback opt-out

# What happens:
# 1. consent-log.json records: {"action": "revoked", "timestamp": "..."}
# 2. preferences.json sets: shareAnonymized = false
# 3. No further data collection occurs

Checking Status

/ork:feedback status
# Shows: consent state, timestamp, policy version

Data Flow

Session Activity


┌──────────────┐
│ Local Metrics│ ← Stored in .claude/feedback/metrics.json
│  (always)    │
└──────┬───────┘


┌──────────────────┐
│ Consent Check    │ ← has_consent() must return true
└──────┬───────────┘

       ▼ (only if consented)
┌──────────────────┐
│ PII Validation   │ ← validate_no_pii() scans all data
└──────┬───────────┘

       ├── PII Found → ABORT

       └── Clean → Export/Send

Transmission (Optional)

If network transmission is enabled:

  • HTTPS only - encrypted in transit
  • No cookies - no session tracking
  • No IP logging - server strips client IP
  • Best effort - no retries on failure
  • Open source server - auditable code

Default behavior is local export only with no network calls.

Data Retention

StageRetention
Local metricsUser-controlled (default 90 days)
Server raw reportsDeleted after aggregation (max 30 days)
Aggregate statisticsIndefinite (no PII)

GDPR Compliance

This implementation follows GDPR requirements:

  1. Explicit consent - opt-in only, no pre-checked boxes
  2. Informed consent - clear disclosure of what's collected
  3. Easy withdrawal - opt-out as easy as opt-in
  4. Audit trail - consent-log.json records all actions
  5. Data minimization - only collect what's needed
  6. Purpose limitation - only used for plugin improvement

Files

FilePurpose
.claude/feedback/preferences.jsonContains shareAnonymized flag
.claude/feedback/consent-log.jsonAudit trail of consent actions
.claude/feedback/metrics.jsonLocal metrics (always stored)
.claude/feedback/analytics-exports/Exported reports for review

Commands Reference

CommandDescription
/ork:feedback opt-inEnable anonymous sharing
/ork:feedback opt-outDisable sharing (revoke consent)
/ork:feedback statusShow current consent status
/ork:feedback exportExport data for review
/ork:feedback privacyDisplay full privacy policy

Contact

Subcommand Reference

Feedback Subcommand Reference

Detailed actions and expected output for each /ork:feedback subcommand.

status (default)

Show the current feedback system state.

Output:

Feedback System Status
-----------------------------
Learning: Enabled
Anonymous sharing: Disabled
Data retention: 90 days

Learned Patterns:
- Auto-approves: npm install, npm test, git push (3 commands)
- Code style: async/await preferred, TypeScript strict mode

Agent Performance:
- backend-architect: 94% success (28 spawns) [improving]
- test-generator: 72% success (18 spawns) [declining]

Context Savings: ~8k tokens/session (estimated)

Storage: .claude/feedback/ (45 KB)

pause

Temporarily pause all learning without clearing data.

Action:

  1. Set pausedUntil in preferences to a far future date
  2. Confirm to user

Output:

Feedback learning paused

Your existing patterns are preserved.
Resume with: /ork:feedback resume

resume

Resume paused learning.

Action:

  1. Clear pausedUntil in preferences
  2. Confirm to user

Output:

Feedback learning resumed

The system will continue learning from your usage.

reset

Clear all learned patterns (requires confirmation).

Action:

  1. Show what will be deleted
  2. Ask for confirmation (user must type "RESET")
  3. If confirmed, clear patterns file but keep preferences

Output (before confirmation):

WARNING: This will clear all learned patterns:

- 5 auto-approve permission rules
- 3 code style preferences
- Agent performance history

Your preferences (enabled, sharing, retention) will be kept.

To confirm, respond with exactly: RESET
To cancel, respond with anything else.

Output (after confirmation):

Feedback data reset

- Cleared 5 permission patterns
- Cleared 3 style preferences
- Cleared agent metrics

Learning will start fresh from now.

export

Export all feedback data to a JSON file.

Action:

  1. Read all feedback files
  2. Combine into single export
  3. Write to .claude/feedback/export-\{date\}.json

Output:

Exported feedback data to:
   .claude/feedback/export-2026-01-14.json

Contains:
- 5 learned permission patterns
- 3 code style preferences
- 8 skill usage metrics
- 4 agent performance records

File size: 12 KB

settings

Show current settings with option to change.

Output:

Feedback Settings
-----------------------------
enabled:              true   (master switch)
learnFromEdits:       true   (learn from code edits)
learnFromApprovals:   true   (learn from permissions)
learnFromAgentOutcomes: true (track agent success)
shareAnonymized:      false  (share anonymous stats)
retentionDays:        90     (data retention period)

To change a setting, use:
  /ork:feedback settings <key> <value>

Example:
  /ork:feedback settings retentionDays 30

opt-in

Enable anonymous analytics sharing. Records GDPR-compliant consent.

Action:

  1. Record consent in consent-log.json with timestamp and policy version
  2. Set shareAnonymized = true in preferences
  3. Confirm to user

Output:

Anonymous analytics sharing enabled.

What we share (anonymized):
  - Skill usage counts and success rates
  - Agent performance metrics
  - Hook trigger counts

What we NEVER share:
  - Your code or file contents
  - Project names or paths
  - Personal information
  - Memory data

Disable anytime: /ork:feedback opt-out

opt-out

Disable anonymous analytics sharing. Revokes consent.

Action:

  1. Record revocation in consent-log.json with timestamp
  2. Set shareAnonymized = false in preferences
  3. Confirm to user

Output:

Anonymous analytics sharing disabled.

Your feedback data stays completely local.
No usage data is shared.

Re-enable anytime: /ork:feedback opt-in

privacy

Display the full privacy policy for anonymous analytics.

Action:

  1. Display comprehensive privacy documentation
  2. Show what's collected, what's never collected
  3. Explain data protection measures

Output:

═══════════════════════════════════════════════════════════════════
              ORCHESTKIT ANONYMOUS ANALYTICS PRIVACY POLICY
═══════════════════════════════════════════════════════════════════

WHAT WE COLLECT (only with your consent)
────────────────────────────────────────────────────────────────────

  ✓ Skill usage counts        - e.g., "api-design used 45 times"
  ✓ Skill success rates       - e.g., "92% success rate"
  ✓ Agent spawn counts        - e.g., "backend-architect spawned 8 times"
  ✓ Agent success rates       - e.g., "88% tasks completed successfully"
  ✓ Hook trigger counts       - e.g., "git-branch-protection triggered 120 times"
  ✓ Hook block counts         - e.g., "blocked 5 potentially unsafe commands"
  ✓ Plugin version            - e.g., "4.12.0"
  ✓ Report date               - e.g., "2026-01-14" (date only, no time)


WHAT WE NEVER COLLECT
────────────────────────────────────────────────────────────────────

  ✗ Your code or file contents
  ✗ Project names, paths, or directory structure
  ✗ User names, emails, or any personal information
  ✗ IP addresses (stripped at network layer)
  ✗ Memory data or conversation history
  ✗ Architecture decisions or design documents
  ✗ API keys, tokens, or credentials
  ✗ Git history or commit messages
  ✗ Any data that could identify you or your projects


YOUR RIGHTS
────────────────────────────────────────────────────────────────────

  • Opt-out anytime:     /ork:feedback opt-out
  • View your data:      /ork:feedback export-analytics
  • Check status:        /ork:feedback status
  • View this policy:    /ork:feedback privacy

export-analytics

Export anonymous analytics data to a file for review before sharing.

Usage:

/ork:feedback export-analytics [path]

If no path is provided, exports to .claude/feedback/analytics-exports/ with a timestamp.

Output:

Analytics exported to: .claude/feedback/analytics-exports/analytics-export-20260114-120000.json

Contents preview:
-----------------
Date: 2026-01-14
Plugin Version: 4.12.0

Summary:
  Skills used: 8
  Skill invocations: 45
  Agents used: 3
  Agent spawns: 12
  Hooks configured: 5

Please review the exported file before sharing.

Export Format:

{
  "timestamp": "2026-01-14",
  "plugin_version": "4.12.0",
  "skill_usage": {
    "api-design-framework": { "uses": 12, "success_rate": 0.92 }
  },
  "agent_performance": {
    "backend-system-architect": { "spawns": 8, "success_rate": 0.88 }
  },
  "hook_metrics": {
    "git-branch-protection": { "triggered": 45, "blocked": 3 }
  },
  "summary": {
    "unique_skills_used": 8,
    "unique_agents_used": 3,
    "hooks_configured": 5,
    "total_skill_invocations": 45,
    "total_agent_spawns": 12
  },
  "metadata": {
    "exported_at": "2026-01-14T12:00:00Z",
    "format_version": "1.0",
    "note": "Review before sharing"
  }
}
Edit on GitHub

Last updated on