audit-skills

Scans all src/skills/*/SKILL.md files and reports compliance with OrchestKit authoring standards. Each category has individual files in rules/ and references/ loaded on-demand.

Quick Reference

Total: 2 rules across 2 categories

Workflow

1. Discover — Glob src/skills/*/SKILL.md to get full skill list
2. Check each skill — Run all checks from rules/audit-checks.md in parallel
3. Classify — Apply status rules from rules/audit-status.md
4. Render — Output table using format from references/output-format.md
5. Totals — Show X pass, Y warn, Z fail at bottom

Quick Start

bash src/skills/audit-skills/scripts/run-audit.sh

Or invoke manually — Claude scans src/skills/, applies checks, and renders the summary table.

Key Decisions

Related Skills

• ork:skill-evolution — Guidance on iterating and improving skills
• ork:quality-gates — Broader codebase quality checks

Rules (2)

Run all per-skill audit checks to produce complete and accurate quality assessments — HIGH

Per-Skill Audit Checks

Run these six checks for every src/skills/<name>/SKILL.md. Use parallel Glob/Grep calls where possible.

Check 1: Line Count

Count lines in SKILL.md.

Incorrect:

# Skipping line count — misses the Anthropic 500-line limit

Correct:

wc -l src/skills/<name>/SKILL.md
# Flag FAIL if count > 500
# Flag WARN if count > 400 (approaching limit)

Check 2: Required Frontmatter Fields

Parse the --- block at the top of SKILL.md. Required fields:

name, description, tags, version, author, user-invocable, complexity

Incorrect:

# Missing version, author — produces incomplete metadata
name: my-skill
description: Does something
tags: [tag]
user-invocable: true
complexity: low

Correct:

name: my-skill
description: Does something. Use when X.
tags: [tag1, tag2]
version: 2.0.0
author: OrchestKit
user-invocable: true
complexity: low

Flag WARN for each missing required field. List the missing fields.

Check 3: Rules Count

Glob src/skills/<name>/rules/*.md, excluding _sections.md and _template.md.

# Count rule files (non-underscore-prefixed)
ls src/skills/<name>/rules/*.md 2>/dev/null | grep -v '^_'

Store as rules_count. Used in Check 5 combined with refs count.

Check 4: References Count

Glob src/skills/<name>/references/*.md.

ls src/skills/<name>/references/*.md 2>/dev/null

Store as refs_count. Used in Check 5 combined with rules count.

Check 5: No Supporting Files

Flag WARN if rules_count == 0 AND refs_count == 0.

Exception: orchestration skills (implement, explore, verify, brainstorming, etc.) with inline workflow detail are acceptable with 0 rules — only flag if BOTH are zero.

Check 6: Manifest Registration

Check whether the skill appears in manifests/ork.json or manifests/orkl.json.

Incorrect:

# Checking only orkl.json — misses skills covered by ork.json "all"

Correct:

with open("manifests/ork.json") as f:
    ork = json.load(f)

# ork.json "skills": "all" means every src/skills/* is included
if ork.get("skills") == "all":
    in_ork = True
else:
    in_ork = skill_name in [s if isinstance(s,str) else s["name"]
                             for s in ork.get("skills", [])]

Flag FAIL if skill is absent from both manifests.

Key rules:

• Run Checks 3 and 4 in parallel (independent Glob calls)
• Check 6 must handle "skills": "all" shorthand in ork.json
• Checks 1 and 2 together form the SKILL.md header validation
• Never skip Check 6 — unregistered skills are dead weight

Classify audit status with correct severity mapping to avoid misleading results — MEDIUM

Audit Status Classification

Maps check results from audit-checks.md to a PASS / WARN / FAIL status per skill.

Severity Table

Classification Logic

Incorrect:

# Treating all issues as WARN — masks real failures
status = "WARN" if any_issue else "PASS"

Correct:

fails = []
warns = []

if line_count > 500:
    fails.append(f"lines={line_count}>500")
elif line_count > 400:
    warns.append(f"lines={line_count}>400 (approaching limit)")

if missing_fm:
    warns.append(f"missing_fm:{','.join(sorted(missing_fm))}")

if rules_count == 0 and refs_count == 0:
    warns.append("no_rules_or_refs")

if not in_manifest:
    fails.append("not_in_manifest")

if broken_refs:
    for ref in broken_refs:
        warns.append(f"broken_ref:{ref}")

# Final status: FAIL > WARN > PASS
if fails:
    status = "FAIL"
elif warns:
    status = "WARN"
else:
    status = "PASS"

Output Detail Lines

Print WARN/FAIL details indented under the skill row:

my-skill                                   510   OK     3    0   YES FAIL
  FAIL: lines=510>500
my-skill-2                                 210  WARN     0    0   YES WARN
  WARN: missing_fm:version,author
  WARN: no_rules_or_refs

Key rules:

• FAIL takes precedence over WARN — a skill with both is FAIL
• Always print detail lines for non-PASS skills for actionability
• Treat each missing frontmatter field as a separate WARN item

References (2)

Edge Cases

Audit Edge Cases

Manifest "all" Shorthand

manifests/ork.json uses "skills": "all" to include every skill in src/skills/. When this is present, every discovered skill is automatically manifest-registered — mark Mfst as YES without listing individual names.

// manifests/ork.json — "all" means everything in src/skills/ qualifies
{ "skills": "all" }

manifests/orkl.json lists skills explicitly. A skill absent from orkl.json but covered by ork.json's "all" is still YES.

Orchestration Skills with 0 Rules

Orchestration/workflow skills describe multi-phase processes rather than prescriptive code patterns. They legitimately have 0 rule files but use references/ for workflow templates, rubrics, and report formats.

Skills in this category (not exhaustive):

• implement, explore, verify, brainstorm
• review-pr, assess, fix-issue
• plan-viz, configure, remember

These should only receive WARN: no_rules_or_refs if BOTH rules/ AND references/ are empty.

Required Frontmatter Fields

The authoring standard (src/skills/CONTRIBUTING-SKILLS.md) lists these as required:

name, description, tags, version, author, user-invocable, complexity

Older skills may be missing version and author — flag as WARN, not FAIL, since the skill still functions.

Broken Reference Detection

Scan SKILL.md body for patterns like:

rules/some-file.md
references/some-file.md

For each mentioned path, verify the file exists under src/skills/<name>/. Flag WARN if missing.

False positive: generic markdown links to external URLs — skip http:// and https:// patterns.

Project-Level Skills

Skills under .claude/skills/ (project-level) are not registered in manifests/. Exclude them from manifest checks or mark Mfst as N/A.

Output Format

Audit Output Format

Summary Table

Skill                                    Lines   FM Rules Refs  Mfst Status
---------------------------------------------------------------------------
<skill-name>                              NNN   OK     N    N   YES  PASS
<skill-name>                              NNN  WARN     N    N   YES  WARN
  WARN: missing_fm:version,author
<skill-name>                              NNN   OK     0    0    NO  FAIL
  FAIL: not_in_manifest
  WARN: no_rules_or_refs
---------------------------------------------------------------------------
Total: N skills | PASS: X | WARN: Y | FAIL: Z

Column Definitions

Detail Lines

For each non-PASS skill, print indented detail lines directly below its table row:

  FAIL: <reason>
  WARN: <reason>

Multiple conditions each get their own line.

Totals Line

Always show at the end:

Total: N skills | PASS: X | WARN: Y | FAIL: Z

Where N = X + Y + Z.