os.ReadDir returns name-sorted entries while JS readdirSync (skill.ts:169) does not, so
the Go and JS ports already emit different <skill_files> blocks today for any skill
with more than limit siblings — a live parity violation, not a latent one. Promote K1 to
an OpenSpec change. K2 waits for a consumer that needs structured frontmatter. K3 stays a
recorded non-goal.js/src/skill.ts, mirrored in golang/skill.go + python/java/csharp;
SPEC.md §3) against VS Code's prompt-file/instructions pipeline
(promptFileParser.ts, automaticInstructionsCollector.ts). toolnexus's skill design is
genuinely good — real-YAML frontmatter, progressive disclosure via one skill tool, a typed
skip inventory, symlink-cycle guards, data-sourced skills. Two things, though, quietly undercut
its own headline guarantee ("byte-identical across five ports") and throw away metadata it will
want later. This ADR records exactly those two, plus one deliberately-deferred design question.SPEC.md §3 deltas
(K1 and K2 both move the observable contract) and a shared examples/ fixture where the emitted
block must match across ports. All proposals are additive and backward-compatible.loadSkills discovers SKILL.md files, parses frontmatter, dedupes first-wins, and builds one
skill tool whose execute — at invoke time — emits the skill body plus a sampled list of
sibling files inside a <skill_files> block (skill.ts loadSkills, sample via
sampleSiblingFiles :161). Frontmatter is parsed with a real YAML parser
(parseFrontmatter :90) but then flattened: only scalar values survive
(data[key] = String(value).trim() :107, Go strings.TrimSpace(fmt.Sprintf(...))
skill.go:127). The sibling sample walks readdirSync order with no sort
(skill.ts:169, Go os.ReadDir skill.go:190). Priority order: K1 deterministic sampling →
K2 lossless frontmatter.
sampleSiblingFiles collects the first limit (default 10) files by DFS over raw directory-read
order — readdirSync(cur, ...) (skill.ts:169), os.ReadDir(cur) (Go skill.go:190) — and
never sorts. readdir order is filesystem- and OS-dependent (ext4 hash order ≠ APFS ≠ what Go
happens to return sorted-by-name). So the <skill_files> block a skill emits differs by machine
and by language port for any skill with more than limit files — directly contradicting the
"byte-identical across five ports" guarantee the source claims for that block. It is a silent
divergence: nothing errors, the sampled set just isn't the same set.
Note VS Code sidesteps this class of bug because it resolves referenced files by explicit link,
not by sampling directory order (AutomaticInstructionsCollector._addReferencedInstructions,
extensions/copilot/src/platform/promptFiles/node/automaticInstructionsCollector.ts:584).
Sort entries by name (byte order, case-sensitive) before DFS descent and before taking the first
limit, in every port. Directories and files sorted by the same key so traversal order is
identical everywhere. No API surface changes; the emitted block simply becomes stable.
// sampleSiblingFiles: sort each directory listing before use
const entries = readdirSync(cur, { withFileTypes: true })
.sort((a, b) => (a.name < b.name ? -1 : a.name > b.name ? 1 : 0))
Optional follow-on (separate change if wanted): expose sampleLimit ordering as documented
"lexicographic, breadth-then-name" so authors can predict which 10 files show.
<file> entries in the same order on
APFS, ext4, and across js/go/python/java/csharp (golden fixture).This is the parity fix — SPEC.md §3 must state "listings are name-sorted before sampling" as a
normative rule, and examples/skill-sampling/ carries the 25-file golden.
parseFrontmatter coerces every value with String(value).trim() and keeps only
string/number/boolean (skill.ts:107, Go skill.go:127), so a header like:
---
name: pdf-fill
allowed-tools: [read_file, write_file]
applyTo: "**/*.pdf"
model: [opus, sonnet]
---
loses allowed-tools and model entirely — arrays vanish at parse time. Today nothing consumes
them, so nothing breaks; but the moment toolnexus wants per-skill tool-gating (the natural
sibling of the MCP per-server tools allowlist and the skills filter that already exist) or
applyTo auto-attach (VS Code's PromptHeader keeps exactly this structured metadata —
src/vs/workbench/contrib/chat/common/promptSyntax/promptFileParser.ts:96-138), the data has
already been thrown away and every author's file has to be re-parsed. Preserve it now; consume it
later.
interface SkillInfo {
name: string
description?: string
content: string
location: string
// NEW — the full parsed frontmatter, structure preserved (arrays, maps, scalars).
// `description`/`name` continue to be surfaced as today for back-compat.
meta?: Record<string, unknown>
}
meta is populated straight from the YAML parse (before the scalar coercion), so allowed-tools
stays an array. No behavior change to the prompt catalog or the skill tool output; this is
purely retaining what was parsed. A future change can read meta["allowed-tools"] to scope the
toolkit per skill — but that is a separate ADR, not this one.
info.meta; name/description
unchanged.prompt() catalog output and skill-tool output are byte-identical to today (meta is carried,
not emitted).malformed-frontmatter skip (K2 doesn't touch the skip
path).Moves SPEC.md §3: SkillInfo.meta and the rule "preserve parsed structure; surface name +
description as before." Each port maps to its native any/object type (map[string]any,
Map<String,Object>, dict). Fixture examples/skill-frontmatter/.
VS Code's instructions pipeline follows [..](./ref.md) links and pulls referenced content in
(_addReferencedInstructions, automaticInstructionsCollector.ts:584), and auto-attaches
instructions to files matching an applyTo glob. toolnexus deliberately does not — a skill is
pulled on demand through the skill tool (progressive disclosure), and that is the correct
minimalism for a library: the model decides when to load, the host doesn't front-load a content
graph. This is recorded as an explicit non-goal so a future reader doesn't "fix" it by accident.
Revisit only if a consumer files a concrete need (e.g. a skill whose body is useless without a
referenced spec). K2's meta["applyTo"] would be the hook if we ever do.
meta retain all keys or only a documented allowlist? Retaining all is
simpler and future-proof; an allowlist avoids surprising an author whose stray YAML key becomes
load-bearing later. Leaning "retain all, document the reserved ones."Can you improve this documentation?Edit on GitHub
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |