pyAMG coarsening improperly applied in 3 dimensions

[bradyrx]

@pwolfram ,

Currently pyAMG coarsens the initial mesh in 3 dimensions, whereas it should probably only coarsen in 2D and then apply this filter to all vertical layers. I.e., we should only use pyAMG to coarsen the horizontal so that the vertical seeding option remains valid in the vertical.

For instance, here's the linear seeding case for an arbitrary grid cell in the 60to30km grid:

And after applying pyAMG:

The odd thing is that it still seeds the vertical column with 20 particles, but they don't at all follow the seeding strategy (e.g. linear or log).

[bradyrx]

As the code currently stands, it downsamples first (at the stage when it creates particles on cell centers), then adds passive floats. Perhaps these steps need to be flipped? However, I don't see why the seeding would get so messed up in the current format. It should be able to coarsen horizontal cell centers, and then apply the vertical weights when seeding.

[bradyrx]

@pwolfram , here's a simple first solution attempt: https://gist.github.com/bradyrx/ddb6c944e425ac5e63b8e8e7361809dd

This accepts a stride argument and simply strides through the lat and lon list removing columns. This doesn't produce a uniform coarsening, but does retain full vertical columns. Perhaps it could be improved by randomly selecting a subset of the lat and lon list, rather than striding through the ordered list. Current runtime is ~1 minute for 60to30.

[pwolfram]

@bradyrx, there is an obvious bug here if the coarsening is in 3D, but as I looked back through the code I don't think the coarsening is in 3D. It should have only been in 2D; but if it is in 3D this explains the extreme slowdown. I'm currently in progress on the horizontal interpolation and am going to focus on that now but agree this needs some revisiting. It isn't obvious why this is the case-- the downsampling looks like it should only be applied in 2D:

MPAS-Model/testing_and_setup/compass/utility_scripts/LIGHTparticles/make_particle_file.py

Line 315 in ea058b8

def cell_centers(f_init, downsample): #{{{

You are just doing the building step (no remapping), correct?

[bradyrx]

but if it is in 3D this explains the extreme slowdown

Yes, this would be encouraging in terms of the runtime (#19).

You're right on what it should be doing. I didn't notice that this was being called just from the cell_centers stage.

You are just doing the building step (no remapping), correct?

Correct. And there's no reason why the remapping should be called with the way I've been calling the script.

Example usage for our 30to10 case:

./make_particle_file.py -i init_file.nc -g graph_file -p 9600 -t 'surface','passive' --nvertlevels 15 --vertseedtype 'log' --spatialfilter SouthernOceanXYZ -d 1 -o particles.nc