Update to the [email protected] interface

[Red-Portal]

This PR aims to update Turing's Variational module to match AdvancedVI's new interface starting from v0.3. I will try not to change the interface too much, but given the new features in AdvancedVI, I think breaking changes will be inevitable. Though the focus will be to provide a good default setting rather than to expose all the features.

Currently proposed interface:

using Turing
using AdvancedVI

d = randn(100)
Turing.@model function model()
   x ~ MvNormal(randn(100), 1)
   y ~ InverseGamma()
end
m = model()
q = Turing.Variational.q_fullrank_gaussian(Random.default_rng(), m)
n_iters = 1000
Turing.Variational.vi(m, q, n_iters)

Closes #2507
Closes #2508
Closes #2430

[yebai]

@Red-Portal, can you fix the tests before I take a look?

[Red-Portal]

@yebai I marked the PR as a draft so that we can first agree on an interface, and then I flesh out the implementation and the tests. Do we wish we proceed in another way?

[yebai]

Let's address the interface later or in a separate PR since that might require more discussions. For this PR, let's try to keep the VI interface non-breaking where possible.

[yebai]

@Red-Portal can you take a look at the following error:

ERROR: LoadError: UndefVarError: `turnprogress` not defined in `AdvancedVI`

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 78 lines in your changes missing coverage. Please review.

Project coverage is 47.32%. Comparing base (fc32e10) to head (43c19aa).

Files with missing lines	Patch %	Lines
src/variational/VariationalInference.jl	0.00%	46 Missing ⚠️
src/variational/bijectors.jl	0.00%	32 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (fc32e10) and HEAD (43c19aa). Click for more details.

HEAD has 6 uploads less than BASE

Flag	BASE (fc32e10)	HEAD (43c19aa)
	30	24

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #2506       +/-   ##
===========================================
- Coverage   84.05%   47.32%   -36.74%     
===========================================
  Files          21       21               
  Lines        1455     1456        +1     
===========================================
- Hits         1223      689      -534     
- Misses        232      767      +535     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[yebai]

Thanks @Red-Portal. I left some comments below.

One high-level comment: I suggest we unify vi_fullrank_gaussian and vi_meanfield_gaussian into a single function, q_distribution(...; gaussian=true|false, fullrank=true|false), to reduce code redundancy.

Note on CI errors:

• CI compains about missing ADVI() and TruncatedADAGrad().

[yebai]

Suggested change

	elseif n_trial == num_max_trials
	else

[Red-Portal]

If I commit this change then the initialization procedure will throw an error at the very first failure? The intention was to let the algorithm fail for at least num_max_trials times.

[yebai]

Can we get the dimentionality via num_params = length(varinfo_linked) instead of length(varinfo_linked[:])?

cc @mkarikom

[yebai]

Please move Bijectors.bijector(model::DynamicPPL.Model,...) to DynamicPPL.

cc @mhauru

[yebai]

Since this function is only used once, I suggest we inline it and add comments to explain its behaviour.

[Red-Portal]

Sorry for the delay! I've been traveling in the past weeks, but will start working on this now

[github-actions]

[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

	return q_init(rng, model; location, scale, meanfield=false, basedist=Normal(), kwargs...)
	return q_init(
	rng, model; location, scale, meanfield=false, basedist=Normal(), kwargs...
	)

[coveralls]

Pull Request Test Coverage Report for Build 14736179503

Details

• 0 of 78 (0.0%) changed or added relevant lines in 2 files are covered.
• 460 unchanged lines in 11 files lost coverage.
• Overall coverage decreased (-36.7%) to 47.321%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
src/variational/bijectors.jl	0	32	0.0%
src/variational/VariationalInference.jl	0	46	0.0%

Files with Coverage Reduction	New Missed Lines	%
src/essential/container.jl	1	87.1%
src/variational/VariationalInference.jl	1	0.0%
src/mcmc/is.jl	15	5.88%
ext/TuringDynamicHMCExt.jl	27	0.0%
src/mcmc/mh.jl	32	60.18%
src/mcmc/particle_mcmc.jl	41	58.33%
src/mcmc/emcee.jl	44	9.62%
src/mcmc/sghmc.jl	50	21.54%
src/stdlib/distributions.jl	50	0.0%
ext/TuringOptimExt.jl	53	0.0%

Totals
Change from base Build 14321422608:	-36.7%
Covered Lines:	689
Relevant Lines:	1456

---

💛 - Coveralls