Cannot load weights to AdaMatcher

[Coronal-Halo]

When I try to load the weights you provided to AdaMatcher by doing:
model.load_state_dict(torch.load(checkpoint_path)['state_dict'])

It gives me the following error, do you know what is causing it? Looks like the weight file is not matching

raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for AdaMatcher:
Missing key(s) in state_dict: "backbone.bn1.weight", "backbone.bn1.bias", "backbone.bn1.running_mean", "backbone.bn1.running_var", "backbone.layer1.0.conv1.weight", "backbone.layer1.0.conv2.weight", "backbone.layer1.0.bn1.weight", "backbone.layer1.0.bn1.bias", "backbone.layer1.0.bn1.running_mean", "backbone.layer1.0.bn1.running_var", "backbone.layer1.0.bn2.weight", "backbone.layer1.0.bn2.bias", "backbone.layer1.0.bn2.running_mean", "backbone.layer1.0.bn2.running_var", "backbone.layer1.1.conv1.weight", "backbone.layer1.1.conv2.weight", "backbone.layer1.1.bn1.weight", "backbone.layer1.1.bn1.bias", "backbone.layer1.1.bn1.running_mean", "backbone.layer1.1.bn1.running_var", "backbone.layer1.1.bn2.weight", "backbone.layer1.1.bn2.bias", "backbone.layer1.1.bn2.running_mean", "backbone.layer1.1.bn2.running_var", "backbone.layer2.0.conv1.weight", "backbone.layer2.0.conv2.weight", "backbone.layer2.0.bn1.weight", "backbone.layer2.0.bn1.bias", "backbone.layer2.0.bn1.running_mean", "backbone.layer2.0.bn1.running_var", "backbone.layer2.0.bn2.weight", "backbone.layer2.0.bn2.bias", "backbone.layer2.0.bn2.running_mean", "backbone.layer2.0.bn2.running_var", "backbone.layer2.0.downsample.0.weight", "backbone.layer2.0.downsample.1.weight", "backbone.layer2.0.downsample.1.bias", "backbone.layer2.0.downsample.1.running_mean", "backbone.layer2.0.downsample.1.running_var", "backbone.layer2.1.conv1.weight", "backbone.layer2.1.conv2.weight", "backbone.layer2.1.bn1.weight", "backbone.layer2.1.bn1.bias", "backbone.layer2.1.bn1.running_mean", "backbone.layer2.1.bn1.running_var", "backbone.layer2.1.bn2.weight", "backbone.layer2.1.bn2.bias", "backbone.layer2.1.bn2.running_mean", "backbone.layer2.1.bn2.running_var", "backbone.layer3.0.conv1.weight", "backbone.layer3.0.conv2.weight", "backbone.layer3.0.bn1.weight", "backbone.layer3.0.bn1.bias", "backbone.layer3.0.bn1.running_mean", "backbone.layer3.0.bn1.running_var", "backbone.layer3.0.bn2.weight", "backbone.layer3.0.bn2.bias", "backbone.layer3.0.bn2.running_mean", "backbone.layer3.0.bn2.running_var", "backbone.layer3.0.downsample.0.weight", "backbone.layer3.0.downsample.1.weight", "backbone.layer3.0.downsample.1.bias", "backbone.layer3.0.downsample.1.running_mean", "backbone.layer3.0.downsample.1.running_var", "backbone.layer3.1.conv1.weight", "backbone.layer3.1.conv2.weight", "backbone.layer3.1.bn1.weight", "backbone.layer3.1.bn1.bias", "backbone.layer3.1.bn1.running_mean", "backbone.layer3.1.bn1.running_var", "backbone.layer3.1.bn2.weight", "backbone.layer3.1.bn2.bias", "backbone.layer3.1.bn2.running_mean", "backbone.layer3.1.bn2.running_var", "backbone.conv1.weight", "backbone.layer3_outconv.weight", "backbone.layer2_outconv.weight", "backbone.layer2_outconv2.0.weight", "backbone.layer2_outconv2.1.weight", "backbone.layer2_outconv2.1.bias", "backbone.layer2_outconv2.1.running_mean", "backbone.layer2_outconv2.1.running_var", "backbone.layer2_outconv2.3.weight", "backbone.layer1_outconv.weight", "backbone.layer1_outconv2.0.weight", "backbone.layer1_outconv2.1.weight", "backbone.layer1_outconv2.1.bias", "backbone.layer1_outconv2.1.running_mean", "backbone.layer1_outconv2.1.running_var", "backbone.layer1_outconv2.3.weight", "feature_interaction.cas_module.block.0.weight", "feature_interaction.cas_module.block.0.bias", "feature_interaction.cas_module.block.2.weight", "feature_interaction.cas_module.block.2.bias", "feature_interaction.layers1.0.q_proj.weight", "feature_interaction.layers1.0.k_proj.weight", "feature_interaction.layers1.0.v_proj.weight", "feature_interaction.layers1.0.merge.weight", "feature_interaction.layers1.0.mlp.0.weight", "feature_interaction.layers1.0.mlp.2.weight", "feature_interaction.layers1.0.pre_norm_q.weight", "feature_interaction.layers1.0.pre_norm_q.bias", "feature_interaction.layers1.0.pre_norm_kv.weight", "feature_interaction.layers1.0.pre_norm_kv.bias", "feature_interaction.layers1.0.norm2.weight", "feature_interaction.layers1.0.norm2.bias", "feature_interaction.layers1.1.q_proj.weight", "feature_interaction.layers1.1.k_proj.weight", "feature_interaction.layers1.1.v_proj.weight", "feature_interaction.layers1.1.merge.weight", "feature_interaction.layers1.1.mlp.0.weight", "feature_interaction.layers1.1.mlp.2.weight", "feature_interaction.layers1.1.pre_norm_q.weight", "feature_interaction.layers1.1.pre_norm_q.bias", "feature_interaction.layers1.1.pre_norm_kv.weight", "feature_interaction.layers1.1.pre_norm_kv.bias", "feature_interaction.layers1.1.norm2.weight", "feature_interaction.layers1.1.norm2.bias", "feature_interaction.feature_embed.weight", "feature_interaction.decoder.layers.0.self_attn.q_proj.weight", "feature_interaction.decoder.layers.0.self_attn.q_proj.bias", "feature_interaction.decoder.layers.0.self_attn.k_proj.weight", "feature_interaction.decoder.layers.0.self_attn.k_proj.bias", "feature_interaction.decoder.layers.0.self_attn.v_proj.weight", "feature_interaction.decoder.layers.0.self_attn.v_proj.bias", "feature_interaction.decoder.layers.0.self_attn.merge.weight", "feature_interaction.decoder.layers.0.multihead_attn.q_proj.weight", "feature_interaction.decoder.layers.0.multihead_attn.q_proj.bias", "feature_interaction.decoder.layers.0.multihead_attn.k_proj.wet", "fine_module.attention.layers.0.pre_norm_kv.bias", "fine_module.attention.layers.0.norm2.weight", "fine_module.attention.layers.0.norm2.bias", "fine_module.attention.layers.1.q_proj.weight", "fine_module.attention.layers.1.k_proj.weight", "fine_module.attention.layers.1.v_proj.weight", "fine_module.attention.layers.1.merge.weight", "fine_module.attention.layers.1.mlp.0.weight", "fine_module.attention.layers.1.mlp.2.weight", "fine_module.attention.layers.1.pre_norm_q.weight", "fine_module.attention.layers.1.pre_norm_q.bias", "fine_module.attention.layers.1.pre_norm_kv.weight", "fine_module.attention.layers.1.pre_norm_kv.bias", "fine_module.attention.layers.1.norm2.weight", "fine_module.attention.layers.1.norm2.bias", "fine_module.down_proj.weight", "fine_module.down_proj.bias", "fine_module.merge_feat.weight", "fine_module.merge_feat.bias", "fine_module.heatmap_conv.0.weight", "fine_module.heatmap_conv.0.bias", "fine_module.heatmap_conv.1.weight", "fine_module.heatmap_conv.1.bias", "fine_module.heatmap_conv.3.weight", "fine_module.heatmap_conv.3.bias".

Unexpected key(s) in state_dict: "matcher.backbone.conv1.weight", "matcher.backbone.bn1.weight", "matcher.backbone.bn1.bias", "matcher.backbone.bn1.running_mean", "matcher.backbone.bn1.running_var", "matcher.backbone.bn1.num_batches_tracked", "matcher.backbone.layer1.0.conv1.weight", "matcher.backbone.layer1.0.conv2.weight", "matcher.backbone.layer1.0.bn1.weight", "matcher.backbone.layer1.0.bn1.bias", "matcher.backbone.layer1.0.bn1.running_mean", "matcher.backbone.layer1.0.bn1.running_var", "matcher.backbone.layer1.0.bn1.num_batches_tracked", "matcher.backbone.layer1.0.bn2.weight", "matcher.backbone.layer1.0.bn2.bias", "matcher.backbone.layer1.0.bn2.running_mean", "matcher.backbone.layer1.0.bn2.running_var", "matcher.backbone.layer1.0.bn2.num_batches_tracked", "matcher.backbone.layer1.1.conv1.weight", "matcher.backbone.layer1.1.conv2.weight", "matcher.backbone.layer1.1.bn1.weight", "matcher.backbone.layer1.1.bn1.bias", "matcher.backbone.layer1.1.bn1.running_mean", "matcher.backbone.layer1.1.bn1.running_var", "matcher.backbone.layer1.1.bn1.num_batches_tracked", "matcher.backbone.layer1.1.bn2.weight", "matcher.backbone.layer1.1.bn2.bias", "matcher.backbone.layer1.1.bn2.running_mean", "matcher.backbone.layer1.1.bn2.running_var", "matcher.backbone.layer1.1.bn2.num_batches_tracked", "matcher.backbone.layer2.0.conv1.weight", "matcher.backbone.layer2.0

[AbyssGaze]

Once you load the checkpoint directly, maybe you need reference the code:
model.load_state_dict({k.replace('matcher.', ''): v for k, v in weights.items()})

[Coronal-Halo]

I see, thanks.

[Coronal-Halo]

What should be the input image pair's dimensions? I am trying to test a pair of my custom images, I resize and transpose them to the dimensions as used in the paper 480 x 640 such that the "image0" and "image1" tensors both have dimension [1, 3, 480, 640], but when I feed this pair to the model it outputs None.

[Inverse-function]

你好，请问问题解决了吗？我也想实现一对图像的匹配结果可视化，您可不可以跟我分享一下您的demo，或者告诉我您最终是怎么加载权重的？谢谢

[Coronal-Halo]

I forgot if I solved this specific problem, but I did not end up using this model since I encountered too many errors overall and eventually resorted to other methods.