Loss is 0 after training

[s2005lg]

Hello all, me again.
I have the following code:

import torch
import torch.nn as nn
from sklearn.datasets import make_moons
from sklearn.model_selection import train_test_split
import torchmetrics
import matplotlib.pyplot as plt

n_samples = 1000

X, y = make_moons(n_samples, noise=0.03, random_state=42)
print("X shape is: ", X.shape)
print("y shape is: ", y.shape)

X = torch.from_numpy(X).type(torch.float)
y = torch.from_numpy(y).type(torch.float)

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, shuffle=True)

print("X_train is: ", X_train[:10])
print("y_train is: ", y_train[:10])


# define the model
# model_03 = nn.Sequential(
#     nn.Linear(in_features=2, out_features=10),
#     nn.ReLU(),
#     nn.Linear(in_features=10, out_features=10),
#     nn.Relu(),
#     nn.Linear(in_features=10, out_features=1),
# )

class MoonsClassificationModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear_layer_stack = nn.Sequential(
                nn.Linear(in_features=2, out_features=10),
                nn.ReLU(),
                nn.Linear(in_features=10, out_features=10),
                nn.ReLU(),
                nn.Linear(in_features=10, out_features=1),
        )

    def forward(self, x):
        return self.linear_layer_stack(x)
model_03 = MoonsClassificationModel()

# Loss function
loss_fn = nn.BCEWithLogitsLoss()

# Optimizer
optimizer = torch.optim.Adam(params=model_03.parameters(), lr=0.1)

epochs = 1000

for epoch in range(epochs):
    model_03.train()

    # Forward pass
    y_logits = model_03(X_train).squeeze()
    
    # Calculate loss
    loss = loss_fn(y_logits, y_train)

    # Optimizer zero grad
    optimizer.zero_grad()

    # Backward pass
    loss.backward()

    # Optimizer step
    optimizer.step()

    # Testing
    model_03.eval()
    with torch.inference_mode():
        test_logits = model_03(X_test).squeeze()
        test_loss = loss_fn(test_logits, y_test)
    if epoch % 100 == 0:
        print(f"Epoch: {epoch} | Loss: {loss:.4f} | Test Loss: {test_loss:.4f}")

# Evaluation
model_03.eval()
with torch.inference_mode():
    y_logits = model_03(X_test).squeeze()
    y_pred = torch.round(torch.sigmoid(y_logits))

    accuracy_fn = torchmetrics.Accuracy(task="binary")
    accuracy_fn.update(y_pred, y_test.long())
    accuracy_value = accuracy_fn.compute()
    print(f"Accuracy: {accuracy_value:.2f}")

However, after running the code. The loss is ZERO

Can anyone help.
Thanks a lot

[37Lime]

Greetings @s2005lg,

You are giving the model a relatively simple problem. This means that the loss value will be low.

Hence, try changing the rounding of your loss value in the print statement:
print(f"Epoch: {epoch} | Loss: {loss:.10f} | Test Loss: {test_loss:.10f}

Allowing for more decimals in the print statement should make the loss visible to you.

I hope this helps.

Best wishes.

[s2005lg]

Hello @37Lime ,
Greetings.
Indeed, I realized that it might be because the problem was too simple, and then I reduced the linear and that solved the problem. But it's true that, as you say, I gave the model a relatively too simple problem