Speed of monai.deploy.core.Application

[Leengit]

Is it usually this slow?

Mimicking the 02_mednist_app example, I have created a monai.deploy.core.Application subclass to do some inference. Its compose function instantiates fivemonai.deploy.core.Operator subclasses of mine and connects the flow. Where the inference on an already loaded .pth file took well less than a second before I started using monai-deploy, my new Application subclass is taking 3 or 4 seconds to start up, before it starts showing me its progress through the five Operator subclasses. I am launching with the equivalent of

os.system(f"monai-deploy exec {deploy_app_path} -m {zip_path} -i {input_dir_path} -o {output_dir_path}")

I am using Ubuntu 20.04 and Python 3.9.10.

[MMelQin]

Thanks @Leengit for the question and sorry for the belated reply.

The App SDK app initialization does take some time, and in my crude measurements, about 2 second for the Spleen and UNETR example app. On the application initialization, the execution context needs to be created, by first figuring out the graph, mapping the I/O in memory as well FS, and more importantly and costly, the loading of the TorchScript.

If the cost of initialization is a big concern, and you have a batch of discreet inputs to process, there are ways to reuse the already instantiated App object, by creating a new main function and call the run method multiple times. I wrote up an example and put it on the discussion board.