Add typescript-compiler-blindspots skill - #85
Closed
MajorLift wants to merge 4 commits into
Closed
Conversation
A hand-written TypeScript type is a claim about a value's shape, and it compiles whether or not the claim is true — so a green build is not evidence that the types are correct. In a partially-migrated repo the compiler often cannot tell you at all: with `checkJs` off, a type written for a function whose callers are still `.js` is checked against nothing. This skill makes the claim falsifiable with a two-arm substitution proof at a fixed commit: Arm A is the PR as written and must be silent, Arm B swaps the hand-written type for the derived one and exercises it as the real call site does. Each new diagnostic is a disagreement the hand-written type concealed. It also audits the second axis a migration can fail on — typing edits that quietly change runtime behavior: stripped `| undefined`, deleted default parameters, literals swapped for runtime enum lookups, and calls made optional so a throw becomes a silent no-op. Includes a runner script, a worked example (12 hand-written types, 5 confirmed divergences, 5 cleared falsifiers), and extension-specific notes. Review-side companion to the "derive types from authoritative sources" guideline in contributor-docs.
MajorLift
force-pushed
the
add-skill/typescript-type-proof
branch
from
July 30, 2026 13:21
c5cd833 to
b138a8d
Compare
…d add the false-negative catalog The old name described what `tsc` already does. The skill's subject is the complement: defects the compiler is structurally unable to report, because it checks declarations for internal consistency and never for correspondence to the source they restate. Adds `references/false-negatives.md` covering the class that needs no PR to find — unchecked index access, optional-vs-undefined, bivariant method parameters, covariant arrays, excess-property checks that only fire on fresh literals, structural erasure of domain types, `any` absorption, unverified `declare module`, asserted external data, and the config-level gaps (`skipLibCheck`, `exclude`, `checkJs`, and `noEmit` meaning a type error breaks a CI job rather than the build). Verified against `metamask-extension` at `7fafda0` with a demonstration file whose ten blocks are all wrong and on which `tsc` reports zero errors.
typescript-compiler-blindspots skill
Six distributional claims across the skill and its references were asserted from a single observed migration: "nearly every real finding", "most types have a source", "the escape hatch usually exists to service the type", "often deletes the reason a guard existed", "the single highest-frequency false negative in most codebases", and "the one most likely to ship a real defect". Where a count exists it is now cited (9 of 12 hand-written types had a source; all five divergences fit the four shapes, stated as a small sample rather than a partition). Where none exists the sentence is rewritten as an instruction to check or as the mechanism itself, which is what each was doing anyway.
MajorLift
marked this pull request as draft
July 30, 2026 14:03
A heading is read out of order, linked to directly, and shown in outlines and search results, so it has no antecedent available: "any of this" and "use this in a review" resolve only for a reader who arrived from the line above.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a
typescript-compiler-blindspotsskill underdomains/testing.The problem
A hand-written TypeScript type is a claim about a value's shape, and it compiles whether or not the claim is true. So a green
tsctells you the code is well-formed, not that the types are correct — and in a partially-migrated repo it often cannot tell you at all: withcheckJsoff, a type written for a function whose callers are still.jsis checked against nothing and can drift indefinitely.That is exactly the gap JS→TS migration PRs live in, and reviewing them by eye does not close it.
What the skill does
@typeon a JS caller, which sometimes already names it),SomeController['method'], a package already imported in the file, the sender of a message,ReturnType<typeof selector>, the@types/*declaration. Plus the case that matters for honesty: when no source exists, hand-writing is correct and gets recorded as a cleared falsifier.Record<string, unknown>).catch { captureException }can outrank a likely one in a loud path, because nothing goes red.Contents
skill.mdscripts/substitution-ab.shreferences/false-negatives.mdreferences/worked-example.mdreferences/metamask-extension.mdcheckJsoff, authoritative-source lookup tableAdded since first review: the standing blind spots
The skill originally covered one class — types hand-written to restate a source that already has one, found by substituting the derived type and diffing
tscoutput.references/false-negatives.mdadds the class that needs no PR to find, because it is a property of the language and the config rather than of any diff:?:accepting an explicitundefined, bivariant method parameters, covariant arrays, excess-property checks that fire only on fresh literals, structural typing erasing domain distinctions.anyabsorbing any annotation, ambientdeclare modulebelieved unconditionally, external data asserted rather than validated, JS callers checked not at all.skipLibCheck,exclude,include,checkJs— and the one worth saying out loud, thatnoEmitplus a transpiling bundler means a type error breaks a CI job, not the build.Each is demonstrated, not asserted. The reference carries a file whose ten blocks are all genuinely wrong; typechecked against
metamask-extensionat7fafda0with that repo's owntsconfig.json,tscreports zero errors on every one.The rename follows from this:
type-proofdescribed whattscalready does. The subject is the complement — what it cannot do, because it checks declarations for internal consistency and never for correspondence to the source they restate.Notes for reviewers
maturity: experimental— the method is proven on one PR (metamask-extension#44397), not yet across repos.references/, notrepos/, deliberately: arepos/subdir containing only an extension overlay would make the skill skip installs for mobile and core. The technique applies to all three.yarn testpasses (41/41) andmetamask-skills list --domain testing --maturity experimentaldiscovers the skill.The part I would push back on hardest
The worked example includes a claim the probes refuted — a provider return type I first read as unsound, which turned out fine once an unrelated nullability error one property ahead of it was isolated. It is in there on purpose: a substitution sweep that only ever confirms is indistinguishable from one that never isolated anything, so the skill requires reporting clearances alongside findings.