Skip to content

prov/efa: derive queued flag in efa_rdm_ope_process_queued_ope - #12434

Draft
a-szegel wants to merge 1 commit into
ofiwg:mainfrom
a-szegel:reform-queue-packet-loop
Draft

prov/efa: derive queued flag in efa_rdm_ope_process_queued_ope#12434
a-szegel wants to merge 1 commit into
ofiwg:mainfrom
a-szegel:reform-queue-packet-loop

Conversation

@a-szegel

Copy link
Copy Markdown
Contributor

efa_rdm_ope_process_queued_ope() took the EFA_RDM_OPE_QUEUED_* flag to process as a parameter, even though that flag is already recorded in ope->internal_flags. The four queued flags are mutually exclusive: an ope is linked onto ope_queued_list through a single queued_entry node, so at most one can be set at a time. Derive the flag from internal_flags instead of passing it, and assert the single-bit invariant.

With the flag self-derived, the progress loop in
efa_rdm_domain_progress_peers_and_queues() and the endpoint-close path collapse their per-flag probe sequences into a single call. The main loop previously issued up to four calls per queued ope -- three of which immediately returned after a redundant (internal_flags & flag) test -- each guarded by a caller-side "if (...) continue" branch. That becomes one call with one branch, removing three function calls and seven conditional branches per queued ope per progress cycle.

Add gtest coverage for the derivation: a no-op when no queued flag is set, and correct dispatch of the BEFORE_HANDSHAKE flag (which returns -FI_EAGAIN without a device round-trip when the peer has not handshaked).

efa_rdm_ope_process_queued_ope() took the EFA_RDM_OPE_QUEUED_* flag to
process as a parameter, even though that flag is already recorded in
ope->internal_flags. The four queued flags are mutually exclusive: an
ope is linked onto ope_queued_list through a single queued_entry node,
so at most one can be set at a time. Derive the flag from internal_flags
instead of passing it, and assert the single-bit invariant.

With the flag self-derived, the progress loop in
efa_rdm_domain_progress_peers_and_queues() and the endpoint-close path
collapse their per-flag probe sequences into a single call. The main
loop previously issued up to four calls per queued ope -- three of which
immediately returned after a redundant (internal_flags & flag) test --
each guarded by a caller-side "if (...) continue" branch. That becomes
one call with one branch, removing three function calls and seven
conditional branches per queued ope per progress cycle.

Add gtest coverage for the derivation: a no-op when no queued flag is
set, and correct dispatch of the BEFORE_HANDSHAKE flag (which returns
-FI_EAGAIN without a device round-trip when the peer has not handshaked).

Signed-off-by: Seth Zegelstein <szegel@amazon.com>
@a-szegel

Copy link
Copy Markdown
Contributor Author

I think UT's need some work, putting up draft PR to get coverage on CI

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant