Tensorflow Glossary Part 3 - Loss functions

Loss functions are the key to optimizing any machine learning algorithm in Tensorflow. It is important to select the right loss function for any machine learning problem which is then fed into the different optimizer functions.

tf.losses.cosine_distance

Adds a cosine-distance loss to the training procedure. (deprecated arguments)

Weighted loss float Tensor. If reduction is NONE, this has the same shape as labels; otherwise, it is scalar.

tf.losses.get_regularization_loss

``````tf.losses.get_regularization_loss(
scope=None,
name='total_regularization_loss'
)
``````

Gets the total regularization loss.

tf.losses.hinge_loss

``````tf.losses.hinge_loss(
labels,
logits,
weights=1.0,
scope=None,
loss_collection=tf.GraphKeys.LOSSES,
reduction=Reduction.SUM_BY_NONZERO_WEIGHTS
)
``````

Adds a hinge loss to the training procedure.

tf.losses.huber_loss

``````tf.losses.huber_loss(
labels,
predictions,
weights=1.0,
delta=1.0,
scope=None,
loss_collection=tf.GraphKeys.LOSSES,
reduction=Reduction.SUM_BY_NONZERO_WEIGHTS
)
``````

Adds a Huber Loss to the training procedure

tf.losses.log_loss

``````tf.losses.log_loss(
labels,
predictions,
weights=1.0,
epsilon=1e-07,
scope=None,
loss_collection=tf.GraphKeys.LOSSES,
reduction=Reduction.SUM_BY_NONZERO_WEIGHTS
)
``````

Adds a Log Loss term to the training procedure.

weights acts as a coefficient for the loss. If a scalar is provided, then the loss is simply scaled by the given value. If weights is a tensor of size [batch_size], then the total loss for each sample of the batch is rescaled by the corresponding element in the weights vector. If the shape of weights matches the shape of predictions, then the loss of each measurable element of predictions is scaled by the corresponding value of weights.

tf.losses.mean_squared_error

``````tf.losses.mean_squared_error(
labels,
predictions,
weights=1.0,
scope=None,
loss_collection=tf.GraphKeys.LOSSES,
reduction=Reduction.SUM_BY_NONZERO_WEIGHTS
)
``````

Adds a Sum-of-Squares loss to the training procedure.

weights acts as a coefficient for the loss. If a scalar is provided, then the loss is simply scaled by the given value. If weights is a tensor of size [batch_size], then the total loss for each sample of the batch is rescaled by the corresponding element in the weights vector. If the shape of weights matches the shape of predictions, then the loss of each measurable element of predictions is scaled by the corresponding value of weights.

tf.losses.sigmoid_cross_entropy

``````tf.losses.sigmoid_cross_entropy(
multi_class_labels,
logits,
weights=1.0,
label_smoothing=0,
scope=None,
loss_collection=tf.GraphKeys.LOSSES,
reduction=Reduction.SUM_BY_NONZERO_WEIGHTS
)
``````

Creates a cross-entropy loss using tf.nn.sigmoid_cross_entropy_with_logits.

weights acts as a coefficient for the loss. If a scalar is provided, then the loss is simply scaled by the given value. If weights is a tensor of shape [batch_size], then the loss weights apply to each corresponding sample.

tf.losses.softmax_cross_entropy

``````tf.losses.softmax_cross_entropy(
onehot_labels,
logits,
weights=1.0,
label_smoothing=0,
scope=None,
loss_collection=tf.GraphKeys.LOSSES,
reduction=Reduction.SUM_BY_NONZERO_WEIGHTS
)

``````

Creates a cross-entropy loss using tf.nn.softmax_cross_entropy_with_logits.

weights acts as a coefficient for the loss. If a scalar is provided, then the loss is simply scaled by the given value. If weights is a tensor of shape [batch_size], then the loss weights apply to each corresponding sample.

If label_smoothing is nonzero, smooth the labels towards 1/num_classes: new_onehot_labels = onehot_labels * (1 - label_smoothing) + label_smoothing / num_classes

Written on April 26, 2018
[ ]