Merge branch 'master' into low-rank-via-factor-relaxation

Author: cedricvincentcuaz

CONTRIBUTORS.md

@@ -2,12 +2,10 @@
 
 ## Creators and Maintainers 
 
-This toolbox has been created by
+This toolbox has been created by [Rémi Flamary](https://remi.flamary.com/)
+and [Nicolas Courty](http://people.irisa.fr/Nicolas.Courty/).
 
-* [Rémi Flamary](https://remi.flamary.com/)
-* [Nicolas Courty](http://people.irisa.fr/Nicolas.Courty/)
-
-It is currently maintained by 
+It is currently maintained by :
 
 * [Rémi Flamary](https://remi.flamary.com/)
 * [Cédric Vincent-Cuaz](https://cedricvincentcuaz.github.io/)

README.md

@@ -40,8 +40,7 @@ POT provides the following generic OT solvers (links to examples):
 * [Sampled solver of Gromov Wasserstein](https://pythonot.github.io/auto_examples/gromov/plot_gromov.html) for large-scale problem with any loss functions [33]
 * Non regularized [free support Wasserstein barycenters](https://pythonot.github.io/auto_examples/barycenters/plot_free_support_barycenter.html) [20].
 * [One dimensional Unbalanced OT](https://pythonot.github.io/auto_examples/unbalanced-partial/plot_UOT_1D.html) with KL relaxation and [barycenter](https://pythonot.github.io/auto_examples/unbalanced-partial/plot_UOT_barycenter_1D.html) [10, 25]. Also [exact unbalanced OT](https://pythonot.github.io/auto_examples/unbalanced-partial/plot_unbalanced_ot.html) with KL and quadratic regularization and the [regularization path of UOT](https://pythonot.github.io/auto_examples/unbalanced-partial/plot_regpath.html) [41]
-* [Partial Wasserstein and Gromov-Wasserstein](https://pythonot.github.io/auto_examples/unbalanced-partial/plot_partial_wass_and_gromov.html) (exact [29] and entropic [3]
-  formulations).
+* [Partial Wasserstein and Gromov-Wasserstein](https://pythonot.github.io/auto_examples/unbalanced-partial/plot_partial_wass_and_gromov.html) and [Partial Fused Gromov-Wasserstein](https://pythonot.github.io/auto_examples/gromov/plot_partial_fgw.html) (exact [29] and entropic [3] formulations).
 * [Sliced Wasserstein](https://pythonot.github.io/auto_examples/sliced-wasserstein/plot_variance.html) [31, 32] and Max-sliced Wasserstein [35] that can be used for gradient flows [36].
 * [Wasserstein distance on the circle](https://pythonot.github.io/auto_examples/plot_compute_wasserstein_circle.html) [44, 45]
 * [Spherical Sliced Wasserstein](https://pythonot.github.io/auto_examples/sliced-wasserstein/plot_variance_ssw.html) [46]
@@ -203,12 +202,9 @@ The examples folder contain several examples and use case for the library. The f
 
 ## Acknowledgements
 
-This toolbox has been created by
+This toolbox has been created by [Rémi Flamary](https://remi.flamary.com/) and [Nicolas Courty](http://people.irisa.fr/Nicolas.Courty/).
 
-* [Rémi Flamary](https://remi.flamary.com/)
-* [Nicolas Courty](http://people.irisa.fr/Nicolas.Courty/)
-
-It is currently maintained by
+It is currently maintained by :
 
 * [Rémi Flamary](https://remi.flamary.com/)
 * [Cédric Vincent-Cuaz](https://cedricvincentcuaz.github.io/)
@@ -219,8 +215,6 @@ POT has benefited from the financing or manpower from the following partners:
 
 <img src="https://pythonot.github.io/master/_static/images/logo_anr.jpg" alt="ANR" style="height:60px;"/><img src="https://pythonot.github.io/master/_static/images/logo_cnrs.jpg" alt="CNRS" style="height:60px;"/><img src="https://pythonot.github.io/master/_static/images/logo_3ia.jpg" alt="3IA" style="height:60px;"/><img src="https://pythonot.github.io/master/_static/images/logo_hiparis.png" alt="Hi!PARIS" style="height:60px;"/>
 
-
-
 ## Contributions and code of conduct
 
 Every contribution is welcome and should respect the [contribution guidelines](https://pythonot.github.io/master/contributing.html). Each member of the project is expected to follow the [code of conduct](https://pythonot.github.io/master/code_of_conduct.html).
@@ -391,3 +385,7 @@ Artificial Intelligence.
 [72] Thibault Séjourné, François-Xavier Vialard, and Gabriel Peyré (2021). [The Unbalanced Gromov Wasserstein Distance: Conic Formulation and Relaxation](https://proceedings.neurips.cc/paper/2021/file/4990974d150d0de5e6e15a1454fe6b0f-Paper.pdf). Neural Information Processing Systems (NeurIPS).
 
 [73] Séjourné, T., Vialard, F. X., & Peyré, G. (2022). [Faster Unbalanced Optimal Transport: Translation Invariant Sinkhorn and 1-D Frank-Wolfe](https://proceedings.mlr.press/v151/sejourne22a.html). In International Conference on Artificial Intelligence and Statistics (pp. 4995-5021). PMLR.
+
+[74] Chewi, S., Maunu, T., Rigollet, P., & Stromme, A. J. (2020). [Gradient descent algorithms for Bures-Wasserstein barycenters](https://proceedings.mlr.press/v125/chewi20a.html). In Conference on Learning Theory (pp. 1276-1304). PMLR.
+
+[75] Altschuler, J., Chewi, S., Gerber, P. R., & Stromme, A. (2021). [Averaging on the Bures-Wasserstein manifold: dimension-free convergence of gradient descent](https://papers.neurips.cc/paper_files/paper/2021/hash/b9acb4ae6121c941324b2b1d3fac5c30-Abstract.html). Advances in Neural Information Processing Systems, 34, 22132-22145.

RELEASES.md

@@ -7,7 +7,15 @@
 - Added feature `grad=last_step` for `ot.solvers.solve` (PR #693)
 - Automatic PR labeling and release file update check (PR #704)
 - Reorganize sub-module `ot/lp/__init__.py` into separate files (PR #714)
+- Fix warning raise when import the library (PR #716)
+- Implement projected gradient descent solvers for entropic partial FGW (PR #702)
 - Fix documentation in the module `ot.gaussian` (PR #718)
+- Refactored `ot.bregman._convolutional` to improve readability (PR #709)
+- Added `ot.gaussian.bures_barycenter_gradient_descent` (PR #680)
+- Added `ot.gaussian.bures_wasserstein_distance` (PR #680)
+- `ot.gaussian.bures_wasserstein_distance` can be batched (PR #680)
+- Backend implementation of `ot.dist` for (PR #701)
+- Updated documentation Quickstart guide and User guide with new API (PR #726)
 - Implement low rank through Factor Relaxation with Latent Coupling (PR #719)
 
 #### Closed issues
@@ -16,6 +24,7 @@
 - Add version number to the documentation (PR #696)
 - Update doc for default regularization in `ot.unbalanced` sinkhorn solvers (Issue #691, PR #700)
 - Clean documentation for `gromov`, `lp` and `unbalanced` folders (PR #710)
+- Clean references in documentation (PR #722)
 
 ## 0.9.5
 
@@ -44,7 +53,6 @@ This release also contains few bug fixes, concerning the support of any metric i
 - Notes before depreciating partial Gromov-Wasserstein function in `ot.partial` moved to ot.gromov  (PR #663)
 - Create `ot.gromov._partial` add new features `loss_fun = "kl_loss"` and `symmetry=False` to all solvers while increasing speed + updating adequatly `ot.solvers` (PR #663)
 - Added `ot.unbalanced.sinkhorn_unbalanced_translation_invariant` (PR #676)
-- Refactored `ot.bregman._convolutional` to improve readability (PR #709)
 
 #### Closed issues
 - Fixed `ot.gaussian` ignoring weights when computing means (PR #649, Issue #648)

docs/source/conf.py

@@ -347,7 +347,7 @@ def __getattr__(cls, name):
 }
 
 sphinx_gallery_conf = {
-    "examples_dirs": ["../../examples", "../../examples/da"],
+    "examples_dirs": ["../../examples"],
     "gallery_dirs": "auto_examples",
     "filename_pattern": "plot_",  # (?!barycenter_fgw)
     "nested_sections": False,

docs/source/index.rst

@@ -17,9 +17,10 @@ Contents
    :maxdepth: 1
 
    self
-   quickstart
-   all
+   auto_examples/plot_quickstart_guide
    auto_examples/index
+   user_guide
+   all
    releases
    contributors
    contributing

docs/source/quickstart.rst renamed to docs/source/user_guide.rst

@@ -1,6 +1,6 @@
 
-Quick start guide
-=================
+User guide
+==========
 
 In the following we provide some pointers about which functions and classes
 to use for different problems related to optimal transport (OT) and machine
@@ -136,12 +136,12 @@ instance the memory cost for an OT problem is always :math:`\mathcal{O}(n^2)` in
 memory because the cost matrix has to be computed. The exact solver in of time
 complexity :math:`\mathcal{O}(n^3\log(n))` and the Sinkhorn solver has been
 proven to be nearly :math:`\mathcal{O}(n^2)` which is still too complex for very
-large scale solvers.
+large scale solvers. For all the generic solvers we need to compute the cost
+matrix and the OT matrix of memory size :math:`\mathcal{O}(n^2)` which can be
+prohibitive for very large scale problems. 
 
-
-If you need to solve OT with large number of samples, we recommend to use
-entropic regularization and memory efficient implementation of Sinkhorn as
-proposed in `GeomLoss <https://www.kernel-operations.io/geomloss/>`_. This
+If you need to solve OT with large number of samples, we provide "lazy" memory efficient implementation of Sinkhorn in pure
+python and using `GeomLoss <https://www.kernel-operations.io/geomloss/>`_. This
 implementation is compatible with Pytorch and can handle large number of
 samples. Another approach to estimate the Wasserstein distance for very large
 number of sample is to use the trick from `Wasserstein GAN
@@ -193,15 +193,19 @@ that will return the optimal transport matrix :math:`\gamma^*`:
 
     # a and b are 1D histograms (sum to 1 and positive)
     # M is the ground cost matrix
+
+    # unified API
+    T = ot.solve(M, a, b).plan  # exact linear program
+
+    # classical API
     T = ot.emd(a, b, M)  # exact linear program
 
 The method implemented for solving the OT problem is the network simplex. It is
 implemented in C from [1]_. It has a complexity of :math:`O(n^3)` but the
 solver is quite efficient and uses sparsity of the solution.
 
 
-
-.. minigallery:: ot.emd
+.. minigallery:: ot.emd, ot.solve
     :add-heading: Examples of use for :any:`ot.emd`
     :heading-level: "
 
@@ -226,7 +230,12 @@ It can computed from an already estimated OT matrix with
 
     # a and b are 1D histograms (sum to 1 and positive)
     # M is the ground cost matrix
-    W = ot.emd2(a, b, M)  # Wasserstein distance / EMD value
+
+    # Wasserstein distance / EMD value with unified API
+    W = ot.solve(M, a, b, return_matrix=False).value  
+
+    # with classical API
+    W = ot.emd2(a, b, M)  
 
 Note that the well known  `Wasserstein distance
 <https://en.wikipedia.org/wiki/Wasserstein_metric>`_ between distributions a and
@@ -246,7 +255,7 @@ the :math:`W_1` Wasserstein distance can be done directly with :any:`ot.emd2`
 when providing :code:`M = ot.dist(xs, xt, metric='euclidean')` to use the Euclidean
 distance.
 
-.. minigallery:: ot.emd2
+.. minigallery:: ot.emd2, ot.solve
     :add-heading: Examples of use for :any:`ot.emd2`
     :heading-level: "
 
@@ -274,6 +283,10 @@ distributions. In the case when the finite sample dataset is supposed Gaussian,
 we provide :any:`ot.gaussian.bures_wasserstein_mapping` that returns the parameters for the
 Monge mapping.
 
+All those special cases are accessible with the unified API of POT through the
+function :any:`ot.solve_sample` with the parameter :code:`method` that allows to
+choose the method used to solve the problem (with :code:`method='1D'` or :code:`method='gaussian'`).
+
 
 Regularized Optimal Transport
 -----------------------------
@@ -330,13 +343,15 @@ The Sinkhorn-Knopp algorithm is implemented in :any:`ot.sinkhorn` and
 linear term. Note that the regularization parameter :math:`\lambda` in the
 equation above is given to those functions with the parameter :code:`reg`.
 
-    >>> import ot
-    >>> a = [.5, .5]
-    >>> b = [.5, .5]
-    >>> M = [[0., 1.], [1., 0.]]
-    >>> ot.sinkhorn(a, b, M, 1)
-    array([[ 0.36552929,  0.13447071],
-        [ 0.13447071,  0.36552929]])
+.. code:: python
+
+    # unified API
+    P = ot.solve(M, a, b, reg=1).plan  # OT Sinkhorn matrix
+    loss = ot.solve(M, a, b, reg=1).value # OT Sinkhorn value
+
+    # classical API
+    P = ot.sinkhorn(a, b, M, reg=1)  # OT Sinkhorn matrix
+    loss = ot.sinkhorn2(a, b, M, reg=1)  # OT Sinkhorn value
 
 More details about the algorithms used are given in the following note.
 
@@ -406,13 +421,10 @@ implementations are not optimized for speed but provide a robust implementation
 of algorithms in [18]_ [19]_.
 
 
-.. minigallery:: ot.sinkhorn
-    :add-heading: Examples of use for :any:`ot.sinkhorn`
+.. minigallery:: ot.sinkhorn ot.sinkhorn2
+    :add-heading: Examples of use for Sinkhorn algorithm
     :heading-level: "
 
-.. minigallery:: ot.sinkhorn2
-    :add-heading: Examples of use for :any:`ot.sinkhorn2`
-    :heading-level: "
 
 
 Other regularizations
@@ -969,18 +981,6 @@ For instance, to disable TensorFlow, set `export POT_BACKEND_DISABLE_TENSORFLOW=
 It's important to note that the `numpy` backend cannot be disabled.
 
 
-List of compatible modules
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-This list will get longer for new releases and will hopefully disappear when POT
-become fully implemented with the backend.
-
-- :any:`ot.bregman`
-- :any:`ot.gromov` (some functions use CPU only solvers with copy overhead)
-- :any:`ot.optim` (some functions use CPU only solvers with copy overhead)
-- :any:`ot.sliced`
-- :any:`ot.utils` (partial)
-
 
 FAQ
 ---

examples/gromov/plot_barycenter_fgw.py

@@ -91,7 +91,7 @@ def build_noisy_circular_graph(
     g = nx.Graph()
     g.add_nodes_from(list(range(N)))
     for i in range(N):
-        noise = float(np.random.normal(mu, sigma, 1))
+        noise = np.random.normal(mu, sigma, 1)[0]
         if with_noise:
             g.add_node(i, attr_name=math.sin((2 * i * math.pi / N)) + noise)
         else:
@@ -107,7 +107,7 @@ def build_noisy_circular_graph(
                 if i == N - 1:
                     g.add_edge(i, 1)
     g.add_edge(N, 0)
-    noise = float(np.random.normal(mu, sigma, 1))
+    noise = np.random.normal(mu, sigma, 1)[0]
     if with_noise:
         g.add_node(N, attr_name=math.sin((2 * N * math.pi / N)) + noise)
     else:
@@ -157,7 +157,7 @@ def graph_colors(nx_graph, vmin=0, vmax=7):
     plt.subplot(3, 3, i + 1)
     g = X0[i]
     pos = nx.kamada_kawai_layout(g)
-    nx.draw(
+    nx.draw_networkx(
         g,
         pos=pos,
         node_color=graph_colors(g, vmin=-1, vmax=1),
@@ -173,7 +173,7 @@ def graph_colors(nx_graph, vmin=0, vmax=7):
 
 # %% We compute the barycenter using FGW. Structure matrices are computed using the shortest_path distance in the graph
 # Features distances are the euclidean distances
-Cs = [shortest_path(nx.adjacency_matrix(x).todense()) for x in X0]
+Cs = [shortest_path(nx.adjacency_matrix(x).toarray()) for x in X0]
 ps = [np.ones(len(x.nodes())) / len(x.nodes()) for x in X0]
 Ys = [
     np.array([v for (k, v) in nx.get_node_attributes(x, "attr_name").items()]).reshape(
@@ -199,7 +199,7 @@ def graph_colors(nx_graph, vmin=0, vmax=7):
 
 # %%
 pos = nx.kamada_kawai_layout(bary)
-nx.draw(
+nx.draw_networkx(
     bary, pos=pos, node_color=graph_colors(bary, vmin=-1, vmax=1), with_labels=False
 )
 plt.suptitle("Barycenter", fontsize=20)