Using a pretrained model as backbone

[mrinath123]

HI!
I created a binary classification model using "tf_efficientnet_b4_ns", the model class is below

class Model1(nn.Module):
    def __init__(self,pretrained):
        super().__init__()
        self.backbone = timm.create_model("tf_efficientnet_b4_ns", pretrained=pretrained, num_classes=0, drop_rate=0., drop_path_rate=0.,global_pool='')
        self.pool = nn.AdaptiveAvgPool2d(1)
        self.fc3_B = nn.Linear(1792,1)
    
    def forward(self,image):
        image = self.backbone(image)
        
        if(len(image.shape) == 4):#for efficientnet models
            image = self.pool(image)
            image = image.view(image.shape[0], -1)

        dec2 = self.fc3_B(image)
        return dec2

I trained this model, giving some good accuracy/AUC, Now I am trying to use this model with its learned weights as the backbone of an effdet model, how to do that?

[mrinath123]

My idea is that I want to detect objects similar to the data I used the model to classify, so I thought of reusing the weights of my classification models for the detection model,rather than training an OD model from scratch.

[rwightman]

@mrinath123 there's no builtin support to load a classifier model into the backbone of the effdet model from a checkpoint -- it only supports loading pretrained weights from the timm pretrained weights if the pretrained_backbone flag is set...

However, it'd be fairly straightforward after the effdet model is created, to load state dict from your pretrained model torch.save(your_model.backbone.state_dict()) into the model.backbone of the effdet model...

[mrinath123]

@rwightman THanks, but es I was not specifically telling about classification models, we could use any type of backbone trained on any type of task, classification was just an example tried by me.
Also like can we directly upload the weights to the backbone from the model I have shown? or I have to do some changes