Tensorflow Glossary Part 3 - Loss functions

Loss functions are the key to optimizing any machine learning algorithm in Tensorflow. It is important to select the right loss function for any machine learning problem which is then fed into the different optimizer functions.

tf.losses.cosine_distance

Adds a cosine-distance loss to the training procedure. (deprecated arguments)

Weighted loss float Tensor. If reduction is NONE, this has the same shape as labels; otherwise, it is scalar.

tf.losses.get_regularization_loss

tf.losses.get_regularization_loss(
    scope=None,
    name='total_regularization_loss'
)

Gets the total regularization loss.

tf.losses.hinge_loss

tf.losses.hinge_loss(
    labels,
    logits,
    weights=1.0,
    scope=None,
    loss_collection=tf.GraphKeys.LOSSES,
    reduction=Reduction.SUM_BY_NONZERO_WEIGHTS
)

Adds a hinge loss to the training procedure.

tf.losses.huber_loss

tf.losses.huber_loss(
    labels,
    predictions,
    weights=1.0,
    delta=1.0,
    scope=None,
    loss_collection=tf.GraphKeys.LOSSES,
    reduction=Reduction.SUM_BY_NONZERO_WEIGHTS
)

Adds a Huber Loss to the training procedure

tf.losses.log_loss

tf.losses.log_loss(
    labels,
    predictions,
    weights=1.0,
    epsilon=1e-07,
    scope=None,
    loss_collection=tf.GraphKeys.LOSSES,
    reduction=Reduction.SUM_BY_NONZERO_WEIGHTS
)

Adds a Log Loss term to the training procedure.

weights acts as a coefficient for the loss. If a scalar is provided, then the loss is simply scaled by the given value. If weights is a tensor of size [batch_size], then the total loss for each sample of the batch is rescaled by the corresponding element in the weights vector. If the shape of weights matches the shape of predictions, then the loss of each measurable element of predictions is scaled by the corresponding value of weights.

tf.losses.mean_squared_error

tf.losses.mean_squared_error(
    labels,
    predictions,
    weights=1.0,
    scope=None,
    loss_collection=tf.GraphKeys.LOSSES,
    reduction=Reduction.SUM_BY_NONZERO_WEIGHTS
)

Adds a Sum-of-Squares loss to the training procedure.

weights acts as a coefficient for the loss. If a scalar is provided, then the loss is simply scaled by the given value. If weights is a tensor of size [batch_size], then the total loss for each sample of the batch is rescaled by the corresponding element in the weights vector. If the shape of weights matches the shape of predictions, then the loss of each measurable element of predictions is scaled by the corresponding value of weights.

tf.losses.sigmoid_cross_entropy

tf.losses.sigmoid_cross_entropy(
    multi_class_labels,
    logits,
    weights=1.0,
    label_smoothing=0,
    scope=None,
    loss_collection=tf.GraphKeys.LOSSES,
    reduction=Reduction.SUM_BY_NONZERO_WEIGHTS
)

Creates a cross-entropy loss using tf.nn.sigmoid_cross_entropy_with_logits.

weights acts as a coefficient for the loss. If a scalar is provided, then the loss is simply scaled by the given value. If weights is a tensor of shape [batch_size], then the loss weights apply to each corresponding sample.

tf.losses.softmax_cross_entropy

tf.losses.softmax_cross_entropy(
    onehot_labels,
    logits,
    weights=1.0,
    label_smoothing=0,
    scope=None,
    loss_collection=tf.GraphKeys.LOSSES,
    reduction=Reduction.SUM_BY_NONZERO_WEIGHTS
)

Creates a cross-entropy loss using tf.nn.softmax_cross_entropy_with_logits.

weights acts as a coefficient for the loss. If a scalar is provided, then the loss is simply scaled by the given value. If weights is a tensor of shape [batch_size], then the loss weights apply to each corresponding sample.

If label_smoothing is nonzero, smooth the labels towards 1/num_classes: new_onehot_labels = onehot_labels * (1 - label_smoothing) + label_smoothing / num_classes

Written on April 26, 2018
[ ]