Description
Would be good to update the version bound for FiniteDifferences to v0.10.0, which fixed some issues with handling complex numbers: JuliaDiff/FiniteDifferences.jl#76.
To really test rrules with complex numbers with FD, we'll need to make sure the inputs/output/sensitivity types are correct. I think the rough logic we discussed is:
If the output is complex, then the "sensitivity" of the output must be complex, and the sensitivities of all inputs will be complex, so all inputs must be complex
This is just for FD; the rrule should be tested with the user-provided inputs.
I used this heuristic in this gist testing Zygote. However, cases like these when testing Zygote rely on other rrules, so this won't work with ChainRules. The only way I can think to handle this is for the rrule_test or _make_fdm_call to pass a modified version of the function to FiniteDifferences with modified inputs.
I haven't thought at all about what to do when the output is something besides a number or array. I assume these aren't supported by rrule_test. This proposal also doesn't handle cases where one wants to test rrules whose inputs are e.g. structured arrays, where FD will force the pulled back sensitivity to have the same structure as the inputs. See FluxML/Zygote.jl#608 (comment) for an example.
Would be good also to test holomorphicness when complex numbers are involved, but I don't know much about this.