Related to #457 I was doing some profiling on our custom fixit rules, and determined that the number of files being linted was the primary determinate in fixit runtime.
To that end, I've setup a repository that demonstrates the performance issues. It contains a large number of empty __init__.py files in a tree of modules, about 41,000 files in total. It has a single fixit rule with a visit_Module that does nothing except return. Running fixit against this tree pegs all of my cores to 100% and takes about 40 seconds to complete.
https://github.com/jarshwah/experiments-fixit is the repository which I'm using to document my findings. So far I've only looked at the file-finding, but plan to dig in further where time permits.
Related to #457 I was doing some profiling on our custom fixit rules, and determined that the number of files being linted was the primary determinate in fixit runtime.
To that end, I've setup a repository that demonstrates the performance issues. It contains a large number of empty
__init__.pyfiles in a tree of modules, about 41,000 files in total. It has a single fixit rule with avisit_Modulethat does nothing except return. Running fixit against this tree pegs all of my cores to 100% and takes about 40 seconds to complete.https://github.com/jarshwah/experiments-fixit is the repository which I'm using to document my findings. So far I've only looked at the file-finding, but plan to dig in further where time permits.