Skip to content

matchPromptWithReason: phrases/allOf/anyOf use unbounded .includes() while noneOf uses \b — short phrases substring-match inside unrelated words #51

@kaiohenricunha

Description

@kaiohenricunha

Summary

hooks/src/prompt-patterns.mts scores skills with an asymmetric matcher: the noneOf branch uses a word-boundary regex, but the positive branches (phrases, allOf, anyOf) use plain String.prototype.includes(). Any skill declaring a short phrase ("PPR", "stale", "use", "deploy", etc.) force-injects itself whenever the prompt contains a larger English word that happens to embed that substring.

This is a narrow, mechanical bug. It's distinct from (though aggravated by) the architectural issue in #38 and the over-broad-patterns issue in #19: even if both of those ship, "ppr" will still match "appropriate" inside a legitimate Vercel repo because the matcher itself is wrong.

Plugin version: 0.32.5.

Minimal repro

echo '{"prompt":"please plan a fix to avoid this if appropriate","cwd":"/tmp","session_id":"repro","hook_event_name":"UserPromptSubmit"}' \
  | node hooks/user-prompt-submit-skill-inject.mjs

Actual:

[vercel-plugin] Best practices auto-suggested based on prompt analysis:
  - "next-cache-components" matched: phrase "ppr" +6
…
You must run the Skill(next-cache-components) tool.

Expected: no match. The prompt has nothing to do with Next.js or Partial Prerendering. "ppr" matched the substring appropriate. skills/next-cache-components/SKILL.md declares phrases: ["PPR"], which normalizes to "ppr".

The same class of false positive triggers on any short phrase embedded in an ordinary word — approach, approval, appropriate, etc.

Root cause

hooks/src/prompt-patterns.mts, matchPromptWithReason:

// noneOf — CORRECT (uses word boundaries)
for (const term of compiled.noneOf) {
  const escaped = term.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
  const re = new RegExp(`(?:^|\\b|\\s)${escaped}(?:\\b|\\s|$)`);
  if (re.test(normalizedPrompt)) { /* suppress */ }
}

// phrases — BROKEN (plain substring)
for (const phrase of compiled.phrases) {
  if (normalizedPrompt.includes(phrase)) {        // ← no word boundary
    score += 6;
  }
}

// allOf — BROKEN
for (const group of compiled.allOf) {
  const allMatch = group.every(t => normalizedPrompt.includes(t));  // ←
  if (allMatch) score += 4;
}

// anyOf — BROKEN
for (const term of compiled.anyOf) {
  if (normalizedPrompt.includes(term)) anyOfScore += 1;              // ←
}

findMatchedPhrases (same file) has the same bug.

The asymmetry — noneOf using \b, everything else using .includes() — looks like an oversight rather than an intentional design choice.

Proposed fix

Extract the noneOf regex into a small helper and use it for all four sites. Multi-word phrases like "partial prerendering" and "use cache" still match, because the boundaries sit on the outer edges only, not between interior words.

function phraseMatchesWordBoundary(normalizedPrompt: string, phrase: string): boolean {
  if (!phrase) return false;
  const escaped = phrase.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
  return new RegExp(`(?:^|\\b|\\s)${escaped}(?:\\b|\\s|$)`).test(normalizedPrompt);
}

Then replace the four .includes() sites in matchPromptWithReason and findMatchedPhrases.

Verification

Using a locally patched prompt-patterns.mjs:

Prompt Before After
please plan a fix to avoid this if appropriate phrase "ppr" +6 → injected no match
i need to approach this approval process with appropriate care phrase "ppr" +6 × 3 → injected no match
help me add partial prerendering to this next.js 16 app injected (legit) still injected via "partial prerendering" and "next.js"
how do I use cacheTag in next.js 16 injected (legit) still injected

No regressions on legitimate multi-word phrase matches.

Related

Happy to open a PR with the helper plus a unit test in hooks/prompt-patterns.test.ts if that's useful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions