HTML API: Implement adoption agency algorithm and active format reconstruction

[sirreal]

Implements two of the HTML Processor's longest-standing unsupported features: reconstruction of active formatting elements and the adoption agency algorithm. The processor no longer bails on misnested or implicitly-closed formatting elements, e.g. <b>bold <i>both</b> italic</i> or <p><em>One<p><em>Two.

What's here

Three commits, intended to be reviewed in order:

1. Stack helpers — positional primitives on WP_HTML_Active_Formatting_Elements (position_of, remove_at, insert_at, replace_node) which the adoption agency algorithm requires, and WP_HTML_Open_Elements::has_node_in_scope() for the spec steps which ask whether a specific node is in scope rather than any element with a given tag name. The "element in scope" termination list moves to a shared class constant.

2. Adoption agency + active format reconstruction — removes the bail() calls in reconstruct_active_formatting_elements() and run_adoption_agency_algorithm() and implements both algorithms fully, including the "Noah's Ark clause" (push_onto_active_formatting_elements()) with real attribute comparison. Real/virtual stack-event provenance is rewritten to use token identity instead of tag-name matching, which the adoption agency algorithm demands (a single tag closer may pop several same-named elements; only one of them is the tag found in the input).

3. FORM end tags — </form> no longer bails when other elements remain open inside the form.

Design: single-pass constraints

A streaming parser visits each node once and cannot move nodes it has already reported. This drives every deviation from browser behavior, and each one is documented at its site:

• Already-visited nodes are not re-parented. Where a browser's adoption agency algorithm would relocate them, this parser reports them where they were originally found.
• Everything visited afterwards gets browser-accurate breadcrumbs. The algorithm's random-access rearrangement of the stack of open elements is expressed as a properly-nested sequence of virtual close/open events (reconcile_stack_of_open_elements()), so the token stream never misnests.
• Reconstructed and cloned formatting elements are virtual nodes which report their source tag's attributes. get_attribute(), get_attribute_names_with_prefix(), get_qualified_attribute_name(), has_class(), and class_list() read the attributes of the tag which created the original element (via a lazily-created, cached reader over the source span). Modification (set_attribute(), remove_attribute(), etc.) remains refused, since there is no tag in the input HTML to modify.
• A FORM closed while its descendants remain open stays in a browser's DOM as the ancestor of following content even though it leaves the stack of open elements. A properly-nested token stream cannot express that; following content is reported outside the FORM, mirroring the stack of open elements.

Testing

• New test class Tests_HtmlApi_WpHtmlProcessorActiveFormattingElements covers: reconstruction across paragraphs and on text nodes, attribute reads on reconstructed elements (including write refusal), the Noah's Ark clause with and without attribute comparison, adoption with and without a furthest block, deep misnesting, the "any other end tag" fallback, A-implicitly-closes-A, a pinned event stream for an adoption case, and seek() stability across an adoption boundary.
• html5lib tree-construction suite: 1,123 passing vs 1,088 on trunk (+35); runtime bails drop from 420 to 340. 45 cases whose expected trees require re-parenting already-visited nodes (or holding a closed FORM open) are statically skipped with documented reasons (SKIP_HTML_PARSER_REPARENTS_VISITED_NODES, SKIP_HTML_PARSER_CANNOT_HOLD_FORM_OPEN).
• phpunit --group html-api: 1,506 tests pass. PHPCS clean on all changed files.

Trac ticket: https://core.trac.wordpress.org/ticket/65383

Use of AI Tools

AI assistance: Yes
Tool(s): Claude Code
Model(s): Fable 5
Used for: Implementation.

---

This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.

[sirreal]

Agent's report @ 764adbd:

The goal is achieved. Both suites pass, the implementation is verified against the HTML spec, browsers, and a 20,000-case oracle sweep. All changes are uncommitted on html-apl/implement-aaa-afr, ready for review.

What was implemented

Adoption agency algorithm (run_adoption_agency_algorithm) — the complete spec algorithm: the furthest-block path with the full inner loop (cloning, AFE bookmark repositioning, counter-based eviction), the 8-iteration outer loop, an identity-based scope check (new WP_HTML_Open_Elements::has_node_in_scope), and the "any other end tag" fallback that the -A…-U handlers now take. All three former bail() sites are gone.

Active format reconstruction (reconstruct_active_formatting_elements) — the full rewind/advance/create algorithm. Reconstructed elements appear as virtual openers whose attributes read from the original tag (a clone token carries a fresh bookmark over the source tag's span; a lazily-created reader serves get_attribute, has_class, class_list, etc., including quirks-mode class matching). Clones are read-only; set_attribute on them refuses.

Noah's Ark clause — implemented in the processor at AFE-push time, comparing tag name, namespace, and decoded attribute sets; attribute signatures are computed only when three same-name candidates already exist, so well-formed documents pay nothing. This also bounds reconstruction memory on adversarial input.

The core design problem

AAA rearranges the stack of open elements in a random-access way, but this parser's visitor consumes a stream of properly-nested open/close events — and it cannot re-parent nodes it already reported. The design: run the spec algorithm on a working copy of the stack, then reconcile — pop to the deepest shared ancestor and re-open the rest. The parser state stays spec-exact, so every token after the misnesting gets the same breadcrumbs a browser would report; only already-visited nodes stay where they were seen. This required hardening event provenance (token-identity for pushes, a one-shot latch for pops) so one source tag never produces two "real" events. The same reconcile helper fixed two pre-existing bugs: the '+A' mid-stack removal that silently corrupted breadcrumb names (e.g. <a><svg><foreignObject><a>), and the -FORM bail.

Verification

• html5lib suite: runtime skips fell from 420 to 340 with zero failures; 35 tests newly pass outright. The 44 tree-shape mismatches were each mechanically verified as the inherent single-pass limitation — post-parse breadcrumbs match a Dom\HTMLDocument sentinel's ancestor chain and normalization is idempotent — and are skip-listed with documented reason constants.
• Oracle sweep: 20,000 generated misnesting sequences, zero state divergences from PHP's native HTML5 parser.
• Performance: typical content parse time unchanged (365ms vs 379ms baseline on 2.7MB); previously-fatal misnested input parses in linear time; clone bookmarks release on GC (13 alive at EOF after 300 reconstruction rounds). Downstream interactivity-api and blocks groups pass untouched.

Tests: new wpHtmlProcessorActiveFormattingElements.php (12 tests: breadcrumbs, clone attributes, Noah's Ark, deep misnesting, event-stream nesting, seek stability); three stale bail-asserting tests updated to assert the now-working behavior; wpHtmlSupportRequiredActiveFormatReconstruction.php absorbed and removed. Class docblocks updated where they described these features as unsupported. PHPCS clean.

[sirreal]

Found one API behavior issue:

WP_HTML_Processor now routes virtual node attribute/class APIs through get_virtual_node_attribute_reader() whenever is_virtual() is true, but virtual POP events for reconstructed/adopted formatting elements are virtual too. That makes generated closing events expose the opener's attributes/classes, unlike real closing tags.

Example:

$p = WP_HTML_Processor::create_fragment( '<p><b class=bold>inside</p>outside' );
while ( $p->next_tag( array( 'tag_name' => 'B', 'class_name' => 'bold', 'tag_closers' => 'visit' ) ) ) {
    var_dump( $p->is_tag_closer(), $p->get_attribute( 'class' ) );
}

This matches the virtual </b> events and reports class => bold for them. A real </b> reports null, so class-filtered closer scans can now match generated closers unexpectedly. I think the virtual attribute/class methods should only delegate to the source-tag reader for virtual open events, e.g. guard with ! $this->is_tag_closer() before calling the reader.

[sirreal]

Review

Cross-checked run_adoption_agency_algorithm() and reconstruct_active_formatting_elements() step-by-step against the WHATWG spec text, and verified locally: --group html-api passes (1,506 tests), the html5lib suite gains 35 newly-passing cases vs the base commit (1,123 vs 1,088, runtime bails 340 vs 420), and PHPCS is clean on all changed files. The bookmark-position adjustments in the adoption agency inner loop, the working-stack index bookkeeping, and the reconcile-then-reopen event ordering all check out against the spec, and the pinned event-stream test (test_adoption_agency_event_stream_remains_properly_nested) is exactly the right regression guard for the reconciliation design. Rewriting stack-event provenance to token identity is strictly better than the old tag-name heuristic, and the added 'html' === $current_node->namespace check on the adoption-agency shortcut fixes a latent spec mismatch in the old code.

Findings

1. Virtual tag closers report their opener's attributes; real closers report none.

A real </b> returns null from get_attribute() (end tags have no attributes), but the virtual closer of a cloned or reconstructed formatting element returns the source opener's attributes, because the pop event carries the same token whose bookmark spans the opening tag:

$p = WP_HTML_Processor::create_fragment( '<p><b class="bold">inside</p>outside' );
while ( $p->next_token() ) {
	if ( 'B' === $p->get_token_name() && $p->is_tag_closer() ) {
		var_dump( $p->get_attribute( 'class' ) ); // string(4) "bold" — a real </b> gives NULL
	}
}

A consumer matching on attributes while walking tokens will now match virtual close events but never real ones. Suggest having get_virtual_node_attribute_reader() return null when the current element is a WP_HTML_Stack_Event::POP so closers behave uniformly.

2. Missing @since changelog notes. get_attribute(), get_attribute_names_with_prefix(), and class_list() all gained the same virtual-node behavior as has_class(), but only has_class() documents it with @since 7.1.0 Reports class names for reconstructed formatting elements…. Add matching notes to the other three.

3. The FORM divergence narrows a documented guarantee but isn't in the class-level docs. The run_adoption_agency_algorithm() docblock promises that "every token visited after this algorithm runs is reported with the ancestor chain a browser would report for it" — true for adoption, but the FORM change introduces the one case where following tokens get non-browser breadcrumbs (a browser keeps the closed FORM as ancestor of subsequent content). Today that's only documented in the html5lib skip constant (SKIP_HTML_PARSER_CANNOT_HOLD_FORM_OPEN), where get_breadcrumbs() consumers won't find it. Worth a sentence in the WP_HTML_Processor class docblock alongside the adoption/fostering discussion.

4. Noah's Ark signature reader skips change_parsing_namespace(). get_virtual_node_attribute_reader() sets the parsing namespace on its reader; get_attribute_comparison_signature() doesn't. Harmless today since the list of active formatting elements only ever holds HTML formatting elements, but either add the one-liner or a comment saying why it's unnecessary, so the asymmetry doesn't read as an oversight.

5. @ticket 58517 on the new tests. That ticket closed as fixed in 6.4; this work will presumably land under #65383 (the 7.1 HTML API tracking ticket). Consider annotating the new tests with the ticket the change actually lands under.

None of these block the approach — #1 is the only behavioral one and it's a one-line guard.