unable to reproduce results on llava-llama-2

[payphone131]

I tried using liuhaotian/llava-llama-2-13b-chat-lightning-preview on the 40 manual dataset. I found this llava model is very hard to jailbreak even with the adversarial image, which is different from the reported figures.

[juealcs]

I have faced same problem. @Unispac Please provide guidance to run on liuhaotian/llava-llama-2-13b-chat-lightning-preview model.
Thanks

[payphone131]

I have faced same problem. @Unispac Please provide guidance to run on liuhaotian/llava-llama-2-13b-chat-lightning-preview model. Thanks

Hi, I found the adversarial attack on llava did not improve the ASR (under both white box and black box settings). Are you facing the same problem?