Toro

Loss

Loss functions. Each takes (input, target) and returns a scalar loss tensor.

FunctionDescription
binaryCrossEntropyWithLogit inp targetBCE=max⁡(x,0)−x⋅y+ln⁡(1+e−∣x∣)\text{BCE} = \max(x,0) - x \cdot y + \ln(1 + e^{-\lvert x \rvert})
cosent scale cosineScores labelsCoSENT pairwise ranking loss.
log⁡∑exp⁡(λ(cos⁡i−cos⁡j))\log\sum\exp(\lambda(\cos_i - \cos_j)) over pairs with y_i < y_j.
cosineScores and labels both have shape [batch].
cosineEmbedding margin x1 x2 targetCosine embedding loss for similarity learning.
crossEntropy inp targetH(p,q)=−(1/n)∑log⁡softmax(x)yiH(p,q) = -(1/n)\sum \log\text{softmax}(x)_{y_i}
ctc logProbs targets inputLengths targetLengthsCTC loss for sequence-to-sequence alignment.
huber delta inp targetHuber loss: smooth combination of L1 and L2.
klDiv inp targetKL(p∥q)=(1/n)∑pi(log⁡pi−qi)\text{KL}(p \| q) = (1/n)\sum p_i (\log p_i - q_i). Expects log-probabilities as input and probabilities as target.
l1 inp targetL1=(1/n)∑∥xi−yi∥\text{L1} = (1/n)\sum\|x_i - y_i\|
mse inp targetMSE=(1/n)∑(xi−yi)2\text{MSE} = (1/n)\sum(x_i - y_i)^2
nll inp targetNLL=−(1/n)∑xi,yi\text{NLL} = -(1/n)\sum x_{i,y_i}
smoothL1 beta inp targetSmoothL1=(1/n)∑zi\text{SmoothL1} = (1/n)\sum z_i where zi=0.5xi2/βz_i = 0.5 x_i^2/\beta if \|x_i\| < \beta, else ∥xi∥−0.5β\|x_i\| - 0.5\beta
tripletMargin margin anchor positive negativeTriplet margin loss for metric learning.

binaryCrossEntropyWithLogit

binaryCrossEntropyWithLogit inp target

BCE=max⁡(x,0)−x⋅y+ln⁡(1+e−∣x∣)\text{BCE} = \max(x,0) - x \cdot y + \ln(1 + e^{-\lvert x \rvert})

Parameters

  • inp : Tensor
  • target : Tensor

Returns Tensor

cosent

cosent scale cosineScores labels

CoSENT pairwise ranking loss. log⁡∑exp⁡(λ(cos⁡i−cos⁡j))\log\sum\exp(\lambda(\cos_i - \cos_j)) over pairs with y_i < y_j. cosineScores and labels both have shape [batch].

Parameters

  • scale : float
  • cosineScores : Tensor
  • labels : Tensor

Returns Tensor

cosineEmbedding

cosineEmbedding margin x1 x2 target

Cosine embedding loss for similarity learning.

Parameters

  • margin : float
  • x1 : Tensor
  • x2 : Tensor
  • target : Tensor

Returns Tensor

crossEntropy

crossEntropy inp target

H(p,q)=−(1/n)∑log⁡softmax(x)yiH(p,q) = -(1/n)\sum \log\text{softmax}(x)_{y_i}

Parameters

  • inp : Tensor
  • target : Tensor

Returns Tensor

ctc

ctc logProbs targets inputLengths targetLengths

CTC loss for sequence-to-sequence alignment.

Parameters

  • logProbs : Tensor
  • targets : Tensor
  • inputLengths : Tensor
  • targetLengths : Tensor

Returns Tensor

huber

huber delta inp target

Huber loss: smooth combination of L1 and L2.

Parameters

  • delta : float
  • inp : Tensor
  • target : Tensor

Returns Tensor

klDiv

klDiv inp target

KL(p∥q)=(1/n)∑pi(log⁡pi−qi)\text{KL}(p \| q) = (1/n)\sum p_i (\log p_i - q_i). Expects log-probabilities as input and probabilities as target.

Parameters

  • inp : Tensor
  • target : Tensor

Returns Tensor

l1

l1 inp target

L1=(1/n)∑∣xi−yi∣\text{L1} = (1/n)\sum|x_i - y_i|

Parameters

  • inp : Tensor
  • target : Tensor

Returns Tensor

mse

mse inp target

MSE=(1/n)∑(xi−yi)2\text{MSE} = (1/n)\sum(x_i - y_i)^2

Parameters

  • inp : Tensor
  • target : Tensor

Returns Tensor

nll

nll inp target

NLL=−(1/n)∑xi,yi\text{NLL} = -(1/n)\sum x_{i,y_i}

Parameters

  • inp : Tensor
  • target : Tensor

Returns Tensor

smoothL1

smoothL1 beta inp target

SmoothL1=(1/n)∑zi\text{SmoothL1} = (1/n)\sum z_i where zi=0.5xi2/βz_i = 0.5 x_i^2/\beta if |x_i| < \beta, else ∣xi∣−0.5β|x_i| - 0.5\beta

Parameters

  • beta : float
  • inp : Tensor
  • target : Tensor

Returns Tensor

tripletMargin

tripletMargin margin anchor positive negative

Triplet margin loss for metric learning.

Parameters

  • margin : float
  • anchor : Tensor
  • positive : Tensor
  • negative : Tensor

Returns Tensor