Fix Python – What does model.eval() do in pytorch?

Question

Asked By – Gulzar

When should I use .eval()? I understand it is supposed to allow me to “evaluate my model”. How do I turn it back off for training?

Example training code using .eval().

Now we will see solution for issue: What does model.eval() do in pytorch?


Answer

model.eval() is a kind of switch for some specific layers/parts of the model that behave differently during training and inference (evaluating) time. For example, Dropouts Layers, BatchNorm Layers etc. You need to turn off them during model evaluation, and .eval() will do it for you. In addition, the common practice for evaluating/validation is using torch.no_grad() in pair with model.eval() to turn off gradients computation:

# evaluate model:
model.eval()

with torch.no_grad():
    ...
    out_data = model(data)
    ...

BUT, don’t forget to turn back to training mode after eval step:

# training step
...
model.train()
...

This question is answered By – trsvchn

This answer is collected from stackoverflow and reviewed by FixPython community admins, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0