Support anything that can save as PNG

[oxinabox]

Including plots.

Closes #62
Alternative to #76

cc @juliohm

[oxinabox]

For 1.0 the problem is that it can't resolve it's test-time dependencies.
Because 1.0 won't reresolve dependencies at test time.
See https://www.oxinabox.net/2021/02/13/Julia-1.6-what-has-changed-since-1.0.html#resolver-willing-to-downgrade-packages-to-install-new-ones-tiered-resolution

For other things, It seems to be because the plot is slightly different.
so I guess need to adjust the sensitivity or find a way to make the plot less sensitive to changes.

[oxinabox]

Failure on 1.0 Ubuntu. idk why. They are indeed slightly different

[juliohm]

Thank you @oxinabox for pushing this forward ❤️

[oxinabox]

@juliohm do you have any ability to dig into what is going on with the heatmap on ubuntu?

[juliohm]

I am on Manjaro Linux so maybe the issue would reproduce there as well? How can I help? I understand very little about ImageIO and about these possible discrepancies between OSes.

[oxinabox]

I am on Manjaro Linux so maybe the issue would reproduce there as well? How can I help? I understand very little about ImageIO and about these possible discrepancies between OSes.

Can you check out this branch and see if tests pass for you locally?

[juliohm]

I get a couple of warnings but all the tests pass.

[oxinabox]

hmmm, Not sure what to do here.
Maybe mark it conditionally broken and open an issue?
Maybe we can up the tolerance? idk how to up tolerence

[juliohm]

I had the same question in the packages where I am using ReferenceTests.jl How to increase tolerance? The current threshold with PSNR is not very intuitive, at least for me, and guidelines are appreciated. Perhaps increase the default tolerance in ReferenceTests.jl would also be a good idea.

Any thoughts on that @johnnychen94 ? It would be nice to fix this OS-specific issue, perhaps just increasing tolerance for now.

[johnnychen94]

A nice and clear solution!

[oxinabox]

Warning: test fails because PSNR -22.433345794677734 < 15 for the heat map

[oxinabox]

OK, I have spun up a linux VM and replicated the failure, not that i can replace it, fixing it should be easier

[oxinabox]

Confirmed it is Plots/GR actually making a clearly different output on linux.

Linux

Mac / Windows

[oxinabox]

This is JuliaPlots/Plots.jl#2127
Now that i know it is a know bug, I feel ok just disabling this test til it is fixed.
It's not our problem to fix.

If there are no objections I will merge this tomorrow