hdbscan, distance matrix

[kmzapp]

Currently the complete distance matrix is computed in the hdbscan function. Is it possible that parts of it are computed and used sequentially for the mutual reachability distance such that it could be stored in smaller objects? I currently get an error message about too large vector size when using the function on a large dataset.

[mhahsler]

I think this would be a nice feature to have. I will refer this to Matt.

[peekxc]

I would love to have this as well. One could probably precompute the core distances only, and then change the MST code to compute the mutual reachability distances on demand. I can't remember if there was a reason for not doing that in the first place.

But I'm open to suggestions, there's probably a better way. @kmzapp did you have any other ideas on how to actually achieve this algorithmically?

[kmzapp]

Thank you for the quick reply. I was thinking it might be either possible to compute it on demand or to store it somehow differently that it does not create one too large object. But I do not have a precise idea how to achieve that.