Skip to content

Switch to/support one-sided MPI communication #71

@natj

Description

@natj

We expect to get performance boost from switching the current non-blocking MPI communication routines to one-sided communication. In addition, this would fix the host-based memory leak in MPICH (isend/irecv leak memory). I'll open this issue to explore this idea.

Changes required would be:

  1. mpi4cpp library does not have one-sided communication routine bindings, and so these need to be first added to the library.
  2. Runko send_data/recv_data functions in tiles have isend and irecv functions that need to be modified.
    • Easiest switch might be to use the Rput/Rget implementation of one-sided communciation that (similar to non-blocking) return requests. We can then check for message completion with the same mpi_wait functionality as before.
  3. MPI windows need to be opened in the beginning of the communications.
    • This could be easily done in the beginning (e.g., in a prelude step) for E and B arrays.
    • J array is more complicated because filtering uses implicit std::swap to change swap this array with temporary allocation.
    • Similarly, ParticleContainers can be re-allocated, and some functionality to dynamically open/close their windows are needed.
  4. MPI fences need to be set up for the communicators.
    • As an MVP, each rank could set these for comm_world but this might result in large overhead.
    • A better implementation would use corgi to sniff out the neighboring ranks and split local communicators for each rank with their unique fences.

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureNew feature or requestoptimizationCode optimization

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions