Merge PR #9 | Update and create documentation using GitHub Pages

from kreshuklab/qy/create-docs

Author: qin-yu

.github/workflows/build-and-publish-docs.yml

@@ -0,0 +1,57 @@
+name: Build and Deploy Documentation
+
+on:
+  push:
+    branches:
+      - main
+      - qy/create-docs
+
+permissions:
+  contents: write
+
+jobs:
+  build-and-deploy:
+    runs-on: ubuntu-latest
+
+    steps:
+      # Checkout the repository
+      - name: Checkout Code
+        uses: actions/checkout@v4
+
+      # Configure Git credentials
+      - name: Configure Git Credentials
+        run: |
+          git config user.name "github-actions[bot]"
+          git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
+
+      # Set up Python
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.x"
+
+      # Generate cache ID
+      - name: Set Cache ID
+        run: echo "cache_id=$(date --utc '+%V')" >> $GITHUB_ENV
+
+      # Cache Python dependencies
+      - name: Cache Python Dependencies
+        uses: actions/cache@v4
+        with:
+          key: mkdocs-material-${{ env.cache_id }}
+          path: .cache
+          restore-keys: |
+            mkdocs-material-
+
+      # Install MkDocs and Plugins, Deploy Documentation
+      - name: Install Dependencies and Deploy Docs
+        run: |
+          pip install mkdocs-material \
+                      mkdocs-git-revision-date-localized-plugin \
+                      mkdocs-git-committers-plugin-2 \
+                      mkdocs-autorefs \
+                      mkdocstrings[python] \
+                      markdown-exec
+          mkdocs gh-deploy --force
+        env:
+          MKDOCS_GIT_COMMITTERS_APIKEY: ${{ secrets.MKDOCS_GIT_COMMITTERS_APIKEY }}

README.md

@@ -1,107 +1,73 @@
-# Plant Nuclei Segmentation Pipelines <!-- omit in toc -->
+# Nuclear Segmentation Pipelines <!-- omit in toc -->
 
-This repository hosts the code and guides for the pipelines used in the paper [_A deep learning-based toolkit for 3D nuclei segmentation and quantitative analysis in cellular and tissue context_](https://www.biorxiv.org/content/10.1101/2024.02.19.580954v1). It is structured in to four folders:
+![stardist_raw_and_segmentation](https://zenodo.org/records/8432366/files/stardist_raw_and_segmentation.jpg)
+
+The GoNuclear repository hosts the code and guides for the pipelines used in the paper [_A deep learning-based toolkit for 3D nuclei segmentation and quantitative analysis in cellular and tissue context_](https://doi.org/10.1242/dev.202800). It is structured in to four folders:
 
 - **stardist/** contains a 3D StarDist training and inference pipeline, `run-stardist`.
 - **plantseg/** contains configuration files for training and inference with PlantSeg.
 - **cellpose/** contains scripts for training and inference with Cellpose.
 - **evaluation/** contains modules for evaluating the segmentation results.
 
-## Table of Contents <!-- omit in toc -->
-
-- [Tools and Workflows](#tools-and-workflows)
-  - [StarDist](#stardist)
-  - [PlantSeg](#plantseg)
-  - [Cellpose](#cellpose)
-- [Data](#data)
-  - [Training Data](#training-data)
-  - [Preparing Data for Inference](#preparing-data-for-inference)
-- [Cite](#cite)
-
-
-## Tools and Workflows
-
-### StarDist
-
-*See [`run-stardist`'s README.md](stardist/README.md) for more details.*
-
-This is one of the most important contribution of this repository. If your nuclei are more or less uniform in shape, please consider using the `run-stardist` pipeline in this repository. It generate separate and round instance segmentation masks for your nuclei images.
-
-- The code and tutorial for running StarDist inference is in the `stardist/` folder
-- The pretrained model is automatically downloaded during inference (also available at [BioImage.IO: StarDist Plant Nuclei 3D ResNet](https://bioimage.io/#/?id=10.5281%2Fzenodo.8421755))
-- An example of segmentation results is shown below.
-
-![stardist_raw_and_segmentation](https://zenodo.org/records/8432366/files/stardist_raw_and_segmentation.jpg)
-
-### PlantSeg
-
-*See [PlantSeg's README.md](plantseg/README.md) for more details.*
-
-If your nuclei have irregular shapes, please consider using the PlantSeg pipeline. It generates instance masks for your nuclei images regardless of their nucleus size and shape.
-
-- The code and tutorial for running PlantSeg inference is in the `plantseg/` folder
-- The pretrained model is automatically downloaded during inference (also available at [BioImage.IO: PlantSeg Plant Nuclei 3D UNet](https://bioimage.io/#/?id=10.5281%2Fzenodo.8401064))
-- An example of segmentation results is shown below.
-
-![plantseg_raw_and_gasp_segmentation](https://zenodo.org/records/10070349/files/plantseg_raw_and_gasp_segmentation.jpg)
-
-### Cellpose
-
-*See [Cellpose's README.md](cellpose/README.md) for more details.*
-
-- The guide for running Cellpose inference and training is in the `cellpose/` folder
-
-## Data
-
-### Training Data
-
-The training data is publicly available on [BioImage Archive](https://www.ebi.ac.uk/biostudies/BioImages/studies/S-BIAD1026).
-
-An example of the raw image:
-
-![raw](https://zenodo.org/records/10070349/files/plantseg_raw.jpg)
-
-Some key information about the training data is listed below:
-
-```python
-original_voxel_size = {  # z, y, x
-    1135: [0.28371836501901143, 0.12678642066720086, 0.12678642066720086],  # validation
-    1136: [0.2837183895131086,  0.12756971653115998, 0.12756971653115998],  # training
-    1137: [0.2837183895131086,  0.1266211463645486,  0.1266211463645486 ],  # training
-    1139: [0.2799036917562724,  0.12674335484590543, 0.12674335484590543],  # training
-    1170: [0.27799632231404964, 0.12698523961670266, 0.12698522349145364],  # training
-}  # [0.2837, 0.1268, 0.1268] is taken as the median
-
-original_median_extents = {  # z, y, x
-    1135: [16, 32, 33],  # validation
-    1136: [16, 32, 32],  # training
-    1137: [16, 32, 32],  # training
-    1139: [16, 32, 33],  # training
-    1170: [16, 29, 30],  # training
-    'average':
-}  # [16, 32, 32] is taken as the median
+and are described in [**GoNuclear documentation** :book:](https://kreshuklab.github.io/go-nuclear/).
+
+## Data and Models
+
+Please go to [BioImage Archive S-BIAD1026](https://www.ebi.ac.uk/biostudies/BioImages/studies/S-BIAD1026) for the training data and models. I organised them in the following structure:
+
+```bash
+Training data
+├── 2d/
+│   ├── isotropic/
+│   │   ├── gold/
+│   │   └── initial/
+│   └── original/
+│       ├── gold/
+│       └── README.txt
+└── 3d_all_in_one/
+    ├── 1135.h5
+    ├── 1136.h5
+    ├── 1137.h5
+    ├── 1139.h5
+    └── 1170.h5
+
+Models
+├── cellpose/
+│   ├── cyto2_finetune/
+│   │   └── gold/
+│   ├── nuclei_finetune/
+│   │   ├── gold/
+│   │   └── initial/
+│   └── scratch_trained/
+│       └── gold/
+├── plantseg/
+│   └── 3dunet/
+│       ├── gold/
+│       ├── initial/
+│       ├── platinum/
+│       └── train_example.yml
+└── stardist/
+    ├── resnet/
+    │   ├── gold/
+    │   ├── initial/
+    │   └── platinum/
+    ├── train_example.yml
+    └── unet/
+        └── gold/
 ```
 
-**Note for training Cellpose:** The best image form for training StarDist and PlantSeg models are the original forms, i.e. the linked dataset is the one that provide the best results. However, to train Cellpose which only takes 2D training data, the images are prepared to be 2D slices of the rescaled isotropic 3D images. The 2D slices includes all XY, XZ and YZ slices ordered randomly by a random prefix in the file name. The 2D slices are saved as TIFF files.
-
-### Preparing Data for Inference
-
-Both HDF5 files and TIFF files can be directly used for both `run-stardist` and `plant-seg` inference. Go to the respective folders's README.md for more details.
-
-## Cite
+## Citation
 
 If you find this work useful, please cite our paper and the respective tools' papers:
 
 ```bibtex
-@article {Vijayan2024.02.19.580954,
-  author = {Athul Vijayan and Tejasvinee Atul Mody and Qin Yu and Adrian Wolny and Lorenzo Cerrone and Soeren Strauss and Miltos Tsiantis and Richard S. Smith and Fred Hamprecht and Anna Kreshuk and Kay Schneitz},
-  title = {A deep learning-based toolkit for 3D nuclei segmentation and quantitative analysis in cellular and tissue context},
-  elocation-id = {2024.02.19.580954},
-  year = {2024},
-  doi = {10.1101/2024.02.19.580954},
-  publisher = {Cold Spring Harbor Laboratory},
-  URL = {https://www.biorxiv.org/content/early/2024/02/21/2024.02.19.580954},
-  eprint = {https://www.biorxiv.org/content/early/2024/02/21/2024.02.19.580954.full.pdf},
-  journal = {bioRxiv}
+@article{vijayan2024deep,
+  title={A deep learning-based toolkit for 3D nuclei segmentation and quantitative analysis in cellular and tissue context},
+  author={Vijayan, Athul and Mody, Tejasvinee Atul and Yu, Qin and Wolny, Adrian and Cerrone, Lorenzo and Strauss, Soeren and Tsiantis, Miltos and Smith, Richard S and Hamprecht, Fred A and Kreshuk, Anna and others},
+  journal={Development},
+  volume={151},
+  number={14},
+  year={2024},
+  publisher={The Company of Biologists}
 }
 ```

cellpose/README.md

@@ -2,45 +2,52 @@
 
 This part of the repo concisely shows how to install, train and segment with Cellpose. In other word, it is a record of how Cellpose is used in this paper. Since my experiments show StarDist and PlantSeg have better 3D segmentation performance than Cellpose, this section is complete yet not extensive.
 
-- [Installation](#installation)
-  - [Install Miniconda](#install-miniconda)
-  - [Install `cellpose` using `pip`](#install-cellpose-using-pip)
-- [Segmentation](#segmentation)
-  - [Data Preparation](#data-preparation)
-  - [Segmentation Command](#segmentation-command)
-- [Training](#training)
-  - [Data Preparation](#data-preparation-1)
-  - [Training Command](#training-command)
-- [Cellpose Version and Code](#cellpose-version-and-code)
-
+* [Installation](#installation)
+  * [Install Miniconda](#install-miniconda)
+  * [Install `cellpose` using `pip`](#install-cellpose-using-pip)
+* [Segmentation](#segmentation)
+  * [Data Preparation](#data-preparation)
+  * [Segmentation Command](#segmentation-command)
+* [Training](#training)
+  * [Data Preparation](#data-preparation-1)
+  * [Training Command](#training-command)
+* [Cellpose Version and Code](#cellpose-version-and-code)
+* [Cite](#cite)
 
 ## Installation
 
 It is recommended to install this package in an environment managed by `conda`. We start the guide by installing Mini-`conda`.
 
 ### Install Miniconda
+
 First step required to use the pipeline is installing Miniconda. If you already have a working Anaconda setup you can go directly to the next step. Anaconda can be downloaded for all platforms from [here](https://www.anaconda.com/products/distribution). We suggest to use Miniconda, because it is lighter and install fewer unnecessary packages.
 
 To download Miniconda, open a terminal and type:
+
 ```bash
 wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
 ```
 
 Then install by typing:
+
 ```bash
 bash ./Miniconda3-latest-Linux-x86_64.sh
 ```
+
 and follow the installation instructions. The Miniconda3-latest-Linux-x86_64.sh file can be safely deleted.
 
 ### Install `cellpose` using `pip`
+
 To create and activate an `conda` environment for `cellpose`, then install `cellpose` in it, run the following commands in the terminal:
+
 ```bash
 conda create --name cellpose python=3.8
 conda activate cellpose
 pip install cellpose
 ```
 
 If you have a nvidia gpu, follow these steps to make use of it:
+
 ```bash
 pip uninstall torch
 conda install pytorch==1.12.0 cudatoolkit=11.3 -c pytorch
@@ -50,13 +57,18 @@ If you encounter error or need more explanation, go to [Cellpose's official inst
 
 ## Segmentation
 
+Although the PlantSeg and StarDist models from this study outperform the Cellpose models I trained. One may find the gold models in [BioImage Archive S-BIAD1026](https://www.ebi.ac.uk/biostudies/BioImages/studies/S-BIAD1026), or one of them [`philosophical-panda` at BioImage Model Zoo](https://bioimage.io/#/?tags=qin%20yu&id=philosophical-panda).
+
 ### Data Preparation
+
 Cellpose inference only segmenet TIFF images, not HDF5. However, it can take 3D volumes as input.
 
 ### Segmentation Command
 
 There are two ways of segmenting 3D images with Cellpose:
-- Segment 3D images slice by slice then stitch 2D segmentation results into 3D segmentation results. With this approach, the images doesn't have to be isotropic, as long as the XY planes have similar properties as the training data.
+
+* Segment 3D images slice by slice then stitch 2D segmentation results into 3D segmentation results. With this approach, the images doesn't have to be isotropic, as long as the XY planes have similar properties as the training data.
+
     ```bash
     cellpose \
         --pretrained_model PATH_TO_MODEL \
@@ -70,7 +82,9 @@ There are two ways of segmenting 3D images with Cellpose:
         --no_npy \
         --save_tif
     ```
-- Compute spatial flow of 3D images in all dimensions then segment the images in 3D directly. You may choose to rescale the images to be isotropic before segmentation, or specify the anisotropy to let Cellpose deal with the rescaling. Here I show the later.
+
+* Compute spatial flow of 3D images in all dimensions then segment the images in 3D directly. You may choose to rescale the images to be isotropic before segmentation, or specify the anisotropy to let Cellpose deal with the rescaling. Here I show the later.
+
     ```bash
     cellpose \
         --pretrained_model PATH_TO_MODEL \
@@ -86,13 +100,14 @@ There are two ways of segmenting 3D images with Cellpose:
         --save_tif
     ```
 
-
 ## Training
 
 ### Data Preparation
+
 Cellpose training only takes 2D images as input. To train on 3D images, we first need to split the 3D images into 2D images. Note that 3D images are better to be rescaled for isotropy in the resulting 2D training data.
 
 ### Training Command
+
 An example training command is shown below, which is used in the paper. The parameters `--learning_rate 0.1` and `--weight_decay 0.0001` are recommended by the [Cellpose official documentation](https://cellpose.readthedocs.io/en/latest/train.html).
 
 ```bash
@@ -106,4 +121,32 @@ cellpose --train --use_gpu \
 ```
 
 ## Cellpose Version and Code
+
 See [Cellpose's GitHub page](https://github.com/MouseLand/cellpose) for the code. Cellpose v2.0.5 was used for training and inference in this paper.
+
+## Cite
+
+If you find the code/models/datasets useful, please cite our paper and Cellpose:
+
+```bibtex
+@article{vijayan2024deep,
+  title={A deep learning-based toolkit for 3D nuclei segmentation and quantitative analysis in cellular and tissue context},
+  author={Vijayan, Athul and Mody, Tejasvinee Atul and Yu, Qin and Wolny, Adrian and Cerrone, Lorenzo and Strauss, Soeren and Tsiantis, Miltos and Smith, Richard S and Hamprecht, Fred A and Kreshuk, Anna and others},
+  journal={Development},
+  volume={151},
+  number={14},
+  year={2024},
+  publisher={The Company of Biologists}
+}
+
+@article{stringer2021cellpose,
+  title={Cellpose: a generalist algorithm for cellular segmentation},
+  author={Stringer, Carsen and Wang, Tim and Michaelos, Michalis and Pachitariu, Marius},
+  journal={Nature methods},
+  volume={18},
+  number={1},
+  pages={100--106},
+  year={2021},
+  publisher={Nature Publishing Group US New York}
+}
+```