Write tests for caching functionality

[mdedetrich]

@romainreuillon If possible can you at some point write tests for the #100 feature. There might be a more natural mechanism (i.e. https://www.scala-sbt.org/1.x/docs/Caching.html) to do caching but its best to have a test first so we don't create regression.