Skip to content

docs: Render C/F90/F08 bindings in man pages #13179

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

jsquyres
Copy link
Member

@jsquyres jsquyres commented Apr 7, 2025

Use the pympistandard Python module and associated MPI-4.1 MPI Standard JSON definition file to generate bindings -- regular and embiggened -- in the MPI man pages.

In the generation code, had to handle a variety of cases:

  • When there is no C binding
  • When there is no mpif.h / use mpi binding
  • When there is no use mpi_f08 binding
  • When there are embiggened C and/or F08 bindings
  • When a single man page includes documentation for multiple MPI APIs
  • When we do not have man pages for MPI APIs described in the official MPI Forum API JSON

Also, this commit adds a new submodule: the pympistandard Python module from the MPI Forum repository.

Some other cleanups were also necessary as part of this commit:

  • Add some missing man pages
  • Fix some random RST formatting errors in existing man pages

@jsquyres jsquyres requested review from hppritcha and bwbarrett April 7, 2025 01:34
@jsquyres jsquyres force-pushed the pr/generate-bindings-in-man-pages branch 3 times, most recently from e63f622 to 7791f2a Compare April 7, 2025 01:44
@jsquyres jsquyres force-pushed the pr/generate-bindings-in-man-pages branch 2 times, most recently from 3c48fba to 8b5045c Compare April 14, 2025 00:13
Use the pympistandard Python module and associated MPI-4.1 MPI
Standard JSON definition file to generate bindings -- regular and
embiggened -- in the MPI man pages.

In the generation code, had to handle a variety of cases:

* When there is no C binding
* When there is no mpif.h / use mpi binding
* When there is no use mpi_f08 binding
* When there are embiggened C and/or F08 bindings
* When a single man page includes documentation for multiple MPI APIs
* When we do not have man pages for MPI APIs described in the official
  MPI Forum API JSON

Also, this commit adds a new submodule: the pympistandard Python
module from the MPI Forum repository.

Some other cleanups were also necessary as part of this commit:

* Add some missing man pages
* Fix some random RST formatting errors in existing man pages

Signed-off-by: Jeff Squyres <[email protected]>
@jsquyres jsquyres force-pushed the pr/generate-bindings-in-man-pages branch from 8b5045c to b660590 Compare April 20, 2025 20:36
@jsquyres jsquyres marked this pull request as ready for review April 20, 2025 22:34
@jsquyres
Copy link
Member Author

jsquyres commented Apr 20, 2025

@bwbarrett @hppritcha I think this is now ready for review. In particular, please examine:

  • Added new 3rd-party/pympistandard git submodule
  • 3rd-party/Makefile.am: Added pympistandard to EXTRA_DIST
  • Using one generated file (man-openmpi/man3/bindings/mpi_init.rst) as a proxy for generating all the bindings RST files.
    • Remember: Sphinx doesn't grok VPATH. Even before this PR, we already copied everything to the builddir and do the Sphinx build there.
  • Although we generate a full set of HTML and man pages so that users do not have to have Sphinx installed to build/install Open MPI, this PR does not ship the generated RST source that contain the bindings in the tarball (specifically: docs/man-openmpi/man3/bindings/*.rst).
    • Instead, if a user chooses to build with Sphinx support (vs. using the pre-built HTML that is included in the tarball), a python script will be invoked during make in the docs directory to generate man-openmpi/man3/bindings/*.rst.
    • This turned out to be easier for various reasons than trying to include man-openmpi/man3/bindings/*.rst in the tarball (mainly because there are hundreds of generated *.rst files, and both listing them individually or via wildcard (.../*.rst) in docs/Makefile.am so that they would be both generated and picked up by EXTRA_DIST was creating problems).
    • I concluded that generating them at user build time was acceptable because we already require the end-user to have Python 3 (vs. having Sphinx, or flex, or other developer-level tools).

Copy link
Member

@hppritcha hppritcha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spot checked some of the more complex man pages and they look good.
double checked with make distcheck and that worked.
But a vpath build failed (with sphinx enabled) looking for the binding rst files:

/Users/hpp/ompi2/build_it/docs/man-openmpi/man3/MPI_Neighbor_alltoall.3.rst:16: CRITICAL: Problems with "include" directive path:
InputError: [Errno 2] No such file or directory: 'man-openmpi/man3/bindings/mpi_neighbor_alltoall.rst'.
man-openmpi/man3/MPI_Neighbor_alltoall.3.rst:10: CRITICAL: Problems with "include" directive path:
InputError: [Errno 2] No such file or directory: 'man-openmpi/man3/bindings/mpi_neighbor_alltoall.rst'.
/Users/hpp/ompi2/build_it/docs/man-openmpi/man3/MPI_Neighbor_alltoallv.3.rst:18: CRITICAL: Problems with "include" directive path:
InputError: [Errno 2] No such file or directory: 'man-openmpi/man3/bindings/mpi_neighbor_alltoallv.rst'.
man-openmpi/man3/MPI_Neighbor_alltoallv.3.rst:12: CRITICAL: Problems with "include" directive path:
InputError: [Errno 2] No such file or directory: 'man-openmpi/man3/bindings/mpi_neighbor_alltoallv.rst'.

is this to be expected? Namely we won't support vpath builds and sphinx doc generation?

@jsquyres
Copy link
Member Author

is this to be expected? Namely we won't support vpath builds and sphinx doc generation?

No, this is not expected. I tested VPATH builds and they were working (including make distcheck). Arrgh... I'll check again...

@hppritcha
Copy link
Member

Oh I should have added I was seeing this when building on my mac laptop.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants