docs: build process and faq (#529)

Adding some useful sections.

Signed-off-by: Henry Schreiner <[email protected]>

Author: henryiii

docs/build.md

@@ -0,0 +1,149 @@
+# Build procedure
+
+Modern Python build procedure is as follows:
+
+## SDist
+
+The SDist is a tarfile with all the code required to build the project, along
+with a little bit of metadata. To build an SDist, you use the `build` tool with
+the `--sdist` flag. For example, `pipx run build --sdist`. This:
+
+1. Reads `pyproject.toml` to get the `build-system` table.
+2. Set up a new isolated environment with the packages listed in
+   `build-system.requires`..
+3. Run `.get_requires_for_build_sdist(...)` inside the module listed in
+   `build-system.build-backend`, if it exists. If this returns a list, install
+   all the packages requested. This allows a backend to dynamically declare
+   dependencies.
+4. Run `.build_sdist(...)` inside the module listed in
+   `build-system.build-backend`. The backend produces an SDist file and returns
+   the filename.
+
+Details of the arguments are skipped above, but they allow arbitrary settings
+(called config-settings) to be passed to all the hook functions and handle
+directories. If you turn off isolated environment building (`--no-isolation` in
+`build`), then steps 2 and 3 are skipped. Note that pip cannot build SDists.
+
+Without build isolation, you can build an SDist manually with
+`python -c "from scikit_build_core.build import build_sdist; build_sdist('dist')"`.
+This will produce an SDist in the `dist` directory. For any other backend,
+substitute the backend above.
+
+## Wheel
+
+The wheel is a zip file (ending in `.whl`) with the built code of the project,
+along with required metadata. There is no code that executes on install; it is a
+simple unpack with a few rules about directories. Wheels do not contain
+`pyproject.toml` or other configuration files. To build an wheel, you use the
+`build` tool with the `--wheel` flag. For example, `pipx run build --wheel`.
+This:
+
+1. Reads `pyproject.toml` to get the `build-system` table.
+2. Set up a new isolated environment with the packages listed in
+   `build-system.requires`..
+3. Run `.get_requires_for_build_wheel(...)` inside the module listed in
+   `build-system.build-backend`, if it exists. If this returns a list, install
+   all the packages requested. This allows a backend to dynamically declare
+   dependencies.
+4. Run `.build_wheel(...)` inside the module listed in
+   `build-system.build-backend`. The backend produces an wheel file and returns
+   the filename.
+
+Details of the arguments are skipped above, but they allow arbitrary settings
+(called config-settings) to be passed to all the hook functions and handle
+directories. If you turn off isolated environment building
+(`--no-build-isolation` in `pip` or `--no-isolation` in `build`), then steps 2
+and 3 are skipped.
+
+:::{note}
+
+If you run build without arguments, it will build an SDist first, then will
+build a wheel from the SDist. This will error if you do not have a valid SDist.
+If you pass `--sdist --wheel`, it will build both directly from the source
+instead.
+
+:::
+
+There are a few other hooks as well; one to allow metadata to be produced
+without building a wheel, and editable versions of the wheel build. Editable
+"wheels" are temporary wheels that are only produced to immediately install and
+discard, and are expected to provide mechanisms to link back to the source code.
+
+## Installing
+
+Installing simply unpacks a wheel into the target filesystem. No code is run, no
+configuration files are present. If pip tries to install a repo or an SDist, it
+will first build a wheel[^1] as shown above, then install that. `installer` is a
+standalone tool that is designed entirely to install wheels.
+
+If you want to run code on install, you either have to use an SDist, or depend
+on a package that is SDist only. However, this is quite rarely required.
+
+There are several directories supported, at least. Besides unpacking to the
+site-packages directory, wheels can also have folders that get unpacked to the
+root of the environment and the Python header locations. But these are generally
+discouraged, with including files in the package's site-package directory and
+using `importlib.resources` to access them is preferred. If someone is not
+working in a virtual environment, having items installed to `/` or `/usr/local`
+for example might be surprising!
+
+## Binary wheels and distributing
+
+A wheel filename has several components:
+
+```
+scikit_build_core-0.1.2-py3-none-any.whl
+|_______________| |___| |_| |__| |_|
+         |          |    |    |    \
+       name      version |    |  platform
+                      python  |
+                             abi
+```
+
+The three new items here (compared to SDists) are the [compatibility tags][]:
+
+- `python tag`: The first version of Python the wheel is compatible with. Often
+  `py3` for pure Python wheels, or `py312` (etc) for compiled wheels.
+- `abi tag`: The interpreter ABI this was built for. `none` for pure Python
+  wheels or compiled wheels that don't use the Python API, `abi3` for stable ABI
+  / limited API wheels, and `cp312` (etc) for normal compiled wheels.
+- `platform tag`: This is the platform the wheel is valid on, such as `any`,
+  `linux_x86_64`, or `manylinux_2_17_x86_64`.
+
+The wheels produced by default are not designed to be redistributable. Making
+them redistributable depends on platform:
+
+- Linux: The `linux_*` tags cannot be uploaded to PyPI. You have to build the
+  wheels in a restricted environment (like the manylinux images) and run the
+  wheels through `auditwheel` to produce redistributable wheels. This will
+  verify you are only using the correct GLibC and restricted set of system
+  libraries, and will bundle external libraries into the wheel with mangled
+  symbols to avoid conflicts. These will have a `manylinux_*` or `musllinux_*`
+  tag, and can be uploaded to PyPI.
+- macOS: The wheels should be build with the official CPython releases, and
+  target a reasonable `MACOSX_DEPLOYMENT_TARGET` value (10.9 or newer). You
+  should run the wheels through `develwheel` to bundle external dependencies.
+  You'll also want to (carefully) cross compile for Apple Silicon.
+- Windows: this is the easiest, usually, as the wheels don't have special rules
+  on what Python or OS is being used. However, if you want to bundle
+  dependencies, you'll need `delvewheel`, which is a bit younger than the other
+  two packages, and has to do a few more intrusive workarounds, but otherwise
+  works like those packages.
+
+The easiest way to handle all the above for all Python versions, OSs,
+architectures, including testing, is to use [cibuildwheel][].
+
+<!-- prettier-ignore-start -->
+
+[^1]: This is the modern build mechanism. If no `pyproject.toml` is present,
+      pip/build will trigger a legacy build/install that either pretends a basic
+      `pyproject.toml` is present (build) or using legacy `setup.py ...` commands
+      (pip). If **both** `pyproject.toml` is not provide and `wheel` is not
+      present, `pip` will even fall back on using `setup.py install` instead of
+      `setup.py bdist_wheel`! You can avoid this whole mess with
+      scikit-build-core.
+
+[cibuildwheel]: https://cibuildwheel.readthedocs.io
+[compatibility tags]: https://packaging.python.org/en/latest/specifications/binary-distribution-format
+
+<!-- prettier-ignore-end -->

docs/examples/getting_started/ctypes/CMakeLists.txt

@@ -0,0 +1,6 @@
+cmake_minimum_required(VERSION 3.15...3.26)
+project(${SKBUILD_PROJECT_NAME} LANGUAGES C)
+
+add_library(example MODULE example.c)
+
+install(TARGETS example DESTINATION example)

docs/examples/getting_started/ctypes/example.c

@@ -0,0 +1 @@
+float square(float x) { return x * x; }

docs/examples/getting_started/ctypes/pyproject.toml

@@ -0,0 +1,7 @@
+[build-system]
+requires = ["scikit-build-core"]
+build-backend = "scikit_build_core.build"
+
+[project]
+name = "example"
+version = "0.0.1"

docs/examples/getting_started/ctypes/src/example/__init__.py

@@ -0,0 +1,7 @@
+# libexample = ctypes.CDLL(files("example") / "libexample.so")
+
+# square = libexample.square
+# square.argtypes = [ctypes.c_float]
+# square.restype = ctypes.c_float
+
+# __all__ = ["square"]

docs/faqs.md

@@ -0,0 +1,60 @@
+# FAQs
+
+This section covers common needs.
+
+## Starting a new project
+
+The easiest way to get started is to use the [Scientific Python cookie][], which
+makes a new project following the [Scientific Python Development Guidelines][].
+Scikit-build-core is one of the backends you can select. The project will have a
+lot of tooling prepared for you as well, including pre-commit checks and a
+noxfile; be sure to read the guidelines to see what is there and how it works.
+
+Another option is the [pybind11 example][].
+
+In the future, a CLI interface with a new project generator is planned.
+
+## Multithreaded builds
+
+For most generators, you can control the parallelization via a CMake define:
+
+```bash
+pip install -Ccmake.define.CMAKE_BUILD_PARALLEL_LEVEL=8 .
+```
+
+or an environment variable:
+
+```bash
+CMAKE_BUILD_PARALLEL_LEVEL=8 pip install .
+```
+
+The default generator on Unix-like platforms is Ninja, which automatically tries
+to run in parallel with the number of cores on your machine.
+
+## Dynamic setup.py options
+
+While we will eventually have some dynamic options, most common needs can be
+moved into your `CMakeLists.txt`. For example, if you had a custom `setup.py`
+option (which setuptools has deprecated as well), you can make it a CMake option
+and then pass it with `-Ccmake.define.<OPTION_NAME>=<value>`. If you need to
+customize configuration options, try `[[tool.scikit-build.overrides]]`. If that
+is missing some value you need, please open an issue and let us know.
+
+## Finding Python
+
+One common mistake when using FindPython is to forget to only request the
+`Development.Module` component. If you request `Development`, you will also
+require the `Development.Embed` component, which will require the Python
+libraries to be found for linking. When building a module on Unix, you do not
+link to Python - the Python symbols are already loaded in the interpreter.
+What's more, the manylinux image (which is used to make redistributable Linux
+wheels) does not have the Python libraries, both to avoid this mistake, and to
+reduce size.
+
+<!-- prettier-ignore-start -->
+
+[scientific python cookie]: https://github.com/scientific-python/cookie
+[scientific python development guide]: https://learn.scientific-python.org/development
+[pybind11 example]: https://github.com/pybind/scikit_build_example
+
+<!-- prettier-ignore-end -->

docs/index.md

@@ -23,6 +23,8 @@ configuration
 cmakelists
 crosscompile
 migration_guide
+build
+faqs
 changelog
 ```