Drift in AR experience ( Filament and ARCore )

[gomtesh-amazon]

I am working on ARCore + Filament application. I need to render 3D objects at specific location in the space given by list of (X,Y,Z) co-ordinates considering initial location of phone as (0,0,0). I am rendering 3D assets using Filament and creating camera background using frames captured by ARCore. Since, the assets are located at specific 3D coordinates, they should not move with camera. But, in my case, when I move my camera, sometimes, for short period of time, 3D objects move along with camera and then go back to original place. What can be the possible reason for this ? Will using Anchor help here ? If yes, How I can add Filament renderables to ARCore anchor ?