Hello! A question about Kitnet

[postyear]

Hello. i want to cite this paper, but I don’t understand something:

The largest RMSE of the output layer is set as the φ(cutoff-threshold) in kitnet, then:
a) Execute: s = RMSE(~ x,h θ (~ x))
b) Verdict: if(s ≥ φβ) then Alert

i dont know How to determine β，‘In the paper uses the raw RMSE score’, Does that mean β=1?

[ymirsky]

For anomaly detectors, you can use any value > 0 as a threshold, the trick is finding a level which results in an acceptable TPR, FPR, and FNR. To find this, I recommend you do the following:

1. get the RMSE values for your training set (all benign samples)
2. fit a distribution to it (I found that my data usually follows a log-normal distribution, alternatively use a histogram instead)
3. select your threshold by using the RMSE value where the probability of getting a higher value lower than your requirement (eg, p=0.001)

The above is if you only have normal (clean) data, but if you have examples of anomalous data you can use AUC and other metrics to help you find the best threshold. Just note that in this case you are tuning the model to your set of anomalies which may not capture all anomalies that may occur in the future.

[postyear]

OK, thank you for the advice. It seems to be a posteriori; if conduct comparative experiments, is a simple and direct method to take the maximum RMSE as the threshold?