How to get the loss of each sample in a batch separately?

[cuthalionn]

Hi,
I am using the pretrained T5ForConditionalGeneration model.
During training, the forward pass returns a single loss averaged over each batch (The default behavior implemented by CrossEntropyLoss), however, I want to access the loss value coming from each sample separately. This would normally be done as follows:

CrossEntropyLoss(..., reduction = "none")

So what is the best practice to get individual losses from T5 in a similar manner? Do I need to change the source code?
Can this be added to the model arguments where the user chooses how they want the loss values to be returned?

Thank you!
Taha