Bootstrap: fixed the issue where modify_qp failed or ah is Null due to a Bootstrap TCP ordering error

[gzshan]

Summary

This PR fixes an intermittent boot-stage failure in the UID boot plugin that causes downstream RDMA setup to fail, accompanied by one of the following errors:

• ibv_modify_qp failure (invalid remote QP attributes), or
• NULL Address Handle (ah == NULL) when creating QP/AH from bootstrap-exchanged data.

Background & Root Cause

Step 1 — Localizing the failure to bootstrap.

In large-scale runs we frequently hit ibv_modify_qp failures (errno 22 / EINVAL). Log analysis showed the QP attributes fed into modify_qp were invalid — specifically, the remote QPN and GID exchanged via bootstrap were zero. By dumping the payload before and after the bootstrap exchange, we confirmed the data was correct before the bootstrap alltoall, but became zero after it. This localized the bug to the bootstrap layer itself, not the RDMA transport.

Step 2 — Confirming cross-step message mis-matching.

The UID bootstrap mechanism establishes a short-lived TCP connection for each message and performs multiplexing and demultiplexing at the receiver based on the (peer, tag) combination. Since successive all-to-all and barrier steps legitimately reuse the same (peer, tag) identifiers, we suspected potential conflicts between these steps. To verify this, we modified the data associated with the barrier operation; in the traces where failures occurred, the bootstrap_recv call within the all-to-all step retrieved a connection carrying the barrier payload. This confirmed that the subsequently initiated barrier connection was received before the in-transit all-to-all connection and was erroneously processed as an all-to-all connection.

Step 3 — Root cause: kernel accept order diverging from wire order.

Tcpdump at the moment of failure showed the SYNs arriving in the expected wire order, but the receiver kernel handed them to accept() in a different order. Under high connection churn, the NIC's RSS hash distributes incoming TCP connections across multiple RX queues, each drained by a different CPU via softirq. When some of those CPUs are momentarily busy, their queues drain more slowly, and a connection that arrived later on the wire can complete the handshake and enter the accept queue earlier than one that arrived before it. Combined with the (peer, tag)-only matching in bootstrap_recv, this reordering silently swaps message payloads between steps — surfacing downstream as zeroed QPN/GID and ibv_modify_qp EINVAL.

Reproduction

We built a minimal standalone reproducer that issues many short-lived TCP connections in a tight burst, mimicking the bootstrap send/recv pattern. Under this workload the receiver's accept() order is easily observed to diverge from the sender's connect() order within seconds — reproducing the exact symptom described above and confirming that the reordering originates in the kernel accept path rather than in NVSHMEM.

Fix

Fix 1 — TX-drain barrier on the sender

Add a TX-drain barrier on the sender, right before close(), so that connection N is guaranteed to be enqueued on the receiver before the next connect()/SYN is issued for connection N+1.

New helper: bootstrap_send_drain_txq()
Reads the socket fd via nccl_fn_table(get_fd, ...).
Polls ioctl(fd, SIOCOUTQ, &unacked) — SIOCOUTQ reports unsent + unacknowledged bytes; it reaches 0 only after the peer has ACKed everything, which is only possible after the peer socket has reached ESTABLISHED, i.e. after this connection is already in the receiver's accept queue.

Sleeps 200µs between polls; bounded by a configurable timeout and by abort_flag.

Fix 2 — Disjoint tag spaces for alltoall and barrier

The reordering above is only observable because bootstrap_recv de-multiplexes purely by (peer, tag). In the original code, bootstrap_uid_alltoall and bootstrap_uid_barrier both started their tag counter from a small value and incremented it per step, so their tag ranges overlapped. If a barrier connection was accepted ahead of an in-flight alltoall connection, the (peer, tag) of the barrier message could collide with what the alltoall recv was waiting for, and the barrier payload would be consumed in place of the alltoall payload (which is exactly the "zeroed QPN/GID" symptom).

New environment variable

Added to env_defs.h:

NVSHMEM_BOOTSTRAP_UID_SEND_DRAIN_MS (int, default: 1000)
> 0 : enable barrier with the given timeout (ms).
<= 0 : disable the barrier entirely (restores the previous behavior).

Thanks