Optimization and performance improvements

[cristianlivella]

Currently, processing reports with several thousand transactions takes many seconds.

There are definitely functions that are executed several times unnecessarily, and values that could be cached.

It is necessary to review which functions are the most computationally heavy and improve and/or cache them.

[cristianlivella]

This is a screenshot of a performance report made using Blackfire, with a set of ~100000 transactions (60000 purchases, 25000 sales and 15000 expenses, with 100 different cryptocurrencies).

86% of the time is taken by the initial parsing and computation made on the transactions file.
In particular, the 25% is taken by the parsing with PhpSpreadsheet, and the 53% by the research of purchase transactions, for the calculation of capital gains.

The findRelativePurchases and getPurchaseAmountRemaining definitely have a huge margin for improvement, as at the moment particularly inefficient loops are performed.

Concerning the loading time required by PhpSpreadsheet, we need to check if we can improve something, or we could at least, during the first request, save the array in a faster readable format (JSON? serialize?) for the following requests.

[cristianlivella]

Using the same input file as before, just parsing from CSV to array takes about 6.5 seconds on my laptop.

Saving the array to a json file doesn't take any significant additional time.

Successive reads of the array from the json file only take about 0.1 seconds, so this is definitely a change to make.

Encryption and decryption with AES does not seem to affect the loading time significantly, so the json file can be encrypted with the same key as the source file, before being saved.

[cristianlivella]

With the last improvements, the processing time for the sample file with ~100000 transactions decreased from 32 to 6 seconds on my laptop (-80%).

There's definitely still something that can be improved, but for now I'm satisfied.

This is the updated call graph from Blackfire:

[cristianlivella]

With the last improvements, the processing time for the sample file with ~100000 transactions decreased from 6 to 3 seconds on my laptop (-50%).

(click for high res)