[After #607][api/single] Support executorch-llama #574
Conversation
anyj0527
requested review from
again4you,
gichan-jang,
jaeyun-jung,
jijoongmoon,
myungjoo and
songgot
as code owners
|
📝 TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #574. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/. |
anyj0527
force-pushed
the
single-llama
branch
2 times, most recently
from
83c9879 to
ed9a143
Compare
| ml_tensors_info_set_tensor_type (in_info, 0, ML_TENSOR_TYPE_UINT8); | ||
| ml_tensors_info_set_tensor_dimension (in_info, 0, dim); | ||
|
|
||
| ml_tensors_data_create (in_info, &in_data); |
There was a problem hiding this comment.
How about checking if in_data is created properly?
status = ml_tensors_data_create (in_info, &in_data);
EXPECT_EQ (status, ML_ERROR_NONE);
| ml_tensors_info_set_tensor_dimension (in_info, 0, dim); | ||
|
|
||
| ml_tensors_data_create (in_info, &in_data); | ||
| ml_tensors_data_set_tensor_data (in_data, 0, prompt.c_str (), prompt.size () + 1); |
There was a problem hiding this comment.
Let's check the result, how about you?
EXPECT_EQ (status, ML_ERROR_NONE);
| goto error; | ||
| } | ||
| /* handle flexible single */ | ||
| if (info->nnfw == ML_NNFW_TYPE_EXECUTORCH_LLAMA) { |
There was a problem hiding this comment.
I think if there are more flexible filter, we may revise this part later.
The llama2c filter I created doesn't seem to be scalable due to API limitations, so I won't add it now.
| ML_NNFW_TYPE_NCNN = 18, /**< Tencent ncnn (Since 9.0) */ | ||
| ML_NNFW_TYPE_TENSORRT = 19, /**< NVidia Tensor-RT (Since 9.0) */ | ||
| ML_NNFW_TYPE_QNN = 20, /**< Qualcomm QNN (Qualcomm® AI Engine Direct) (Since 9.0) */ | ||
| ML_NNFW_TYPE_EXECUTORCH_LLAMA = 21, /**< ExecuTorch Llama runner */ |
|
Please do not merge this until tizen gbm release is done! |
- Let single API handle flexible filter. Signed-off-by: Yongjoo Ahn <[email protected]>
- Add a enum for new nnfw `EXECUTORCH_LLAMA` Signed-off-by: Yongjoo Ahn <[email protected]>
anyj0527
force-pushed
the
single-llama
branch
from
ed9a143 to
8f8b2e4
Compare
anyj0527
changed the title
- Let single API handle executorch-llama as flexible filter thus do invoke-dynamic. Signed-off-by: Yongjoo Ahn <[email protected]>
- Add a simple test to show how to use executorch-llama with single api Signed-off-by: Yongjoo Ahn <[email protected]>
anyj0527
force-pushed
the
single-llama
branch
from
8f8b2e4 to
cd1847b
Compare
6 participants
EXECUTORCH_LLAMA