[Bug] Attention_mask is all True due to the mismatched pad_token_id

[StevenZzz07]

Checklist

• 1. I have searched related issues but cannot get the expected help.
• 2. The bug has not been fixed in the latest version.
• 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

Describe the bug

I am working on finetuning InternVL2.5-2B model. Now i am faced with a problem in def concat_pad_data_collator in InternVL/internvl_chat/internvl/patch/pad_data_collator.py.

The input feature["input_ids"] has been padded with default pad_token_id=2 (from InternVL2.5_2B's config.json) during preprocess_internvl2_5(with group_by_length set False). However, the input_ids will be padded again in concat_pad_data_collator with pad_token_id=0. In this way, the attention_mask turns out to be all True since the function default pad_token_id(0) is not equal to InternVL_2.5's default pad_token_id(2). To solve this, we can use partial function to set the pad_token_id to tokenizer.pad_token_id when defining the collator in InternVL/internvl_chat/internvl/train/internvl_chat_finetune.py

However, I would like to confirm whether it is indeed a problem.
Look forward to ur reply, thks :)

Reproduction

train

Environment

transformers==4.47.1

Error traceback