假設我想同時訓練三個模型（模型 1、模型 2 和模型 3），并且在訓練時讓模型一和模型二與主網絡（模型 1）共同共享損失。因此主模型可以從層間的其他兩個模型中學習表示。損失總計 = (權重1)損失m1 + (權重2)(損失m1 - 損失m2) + (權重3)(損失m1 - 損失m3)到目前為止我有以下內容：def threemodel(num_nodes, num_class, w1, w2, w3): #w1; w2; w3 are loss weights in1 = Input((6373,)) enc1 = Dense(num_nodes)(in1) enc1 = Dropout(0.3)(enc1) enc1 = Dense(num_nodes, activation='relu')(enc1) enc1 = Dropout(0.3)(enc1) enc1 = Dense(num_nodes, activation='relu')(enc1) out1 = Dense(units=num_class, activation='softmax')(enc1) in2 = Input((512,)) enc2 = Dense(num_nodes, activation='relu')(in2) enc2 = Dense(num_nodes, activation='relu')(enc2) out2 = Dense(units=num_class, activation='softmax')(enc2) in3 = Input((768,)) enc3 = Dense(num_nodes, activation='relu')(in3) enc3 = Dense(num_nodes, activation='relu')(enc3) out3 = Dense(units=num_class, activation='softmax')(enc3) adam = Adam(lr=0.0001) model = Model(inputs=[in1, in2, in3], outputs=[out1, out2, out3]) model.compile(loss='categorical_crossentropy', #continu together optimizer='adam', metrics=['accuracy'] not sure know what changes need to be made here)## I am confused on how to formulate the shared losses equation here to share the losses of out2 and out3 with out1.經過一番搜索后，似乎可以執行以下操作：loss_1 = tf.keras.losses.categorical_crossentropy(y_true_1, out1) loss_2 = tf.keras.losses.categorical_crossentropy(y_true_2, out2) loss_3 = tf.keras.losses.categorical_crossentropy(y_true_3, out3) model.add_loss((w1)*loss_1 + (w2)*(loss_1 - loss_2) + (w3)*(loss_1 - loss_3))這可以嗎？我覺得通過執行上面建議的操作并沒有真正執行我想要的操作，即讓主模型（mod1）從各層之間的其他兩個模型（mod2 和 mod3）學習表示。有什么建議么？
1 回答
尚方寶劍之說
TA貢獻1788條經驗 獲得超4個贊
由于您對使用可訓練權重不感興趣（我將它們標記為系數以將它們與可訓練權重區分開），您可以連接輸出并將它們作為單個輸出傳遞給自定義損失函數。這意味著這些系數將在訓練開始時可用。
您應該提供如上所述的自定義損失函數。損失函數預計只接受 2 個參數，因此您應該使用這樣一個函數categorical_crossentropy，它也應該熟悉您感興趣的參數，例如coeffs和num_class。因此，我使用所需的參數實例化一個包裝函數，然后將內部實際損失函數作為主損失函數傳遞。
from tensorflow.keras.layers import Dense, Dropout, Input, Concatenate
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.models import Model
from tensorflow.python.framework import ops
from tensorflow.python.framework import smart_cond
from tensorflow.python.ops import math_ops
from tensorflow.python.ops import array_ops
from tensorflow.python.keras import backend as K
def categorical_crossentropy_base(coeffs, num_class):
def categorical_crossentropy(y_true, y_pred, from_logits=False, label_smoothing=0):
"""Computes the categorical crossentropy loss.
Args:
y_true: tensor of true targets.
y_pred: tensor of predicted targets.
from_logits: Whether `y_pred` is expected to be a logits tensor. By default,
we assume that `y_pred` encodes a probability distribution.
label_smoothing: Float in [0, 1]. If > `0` then smooth the labels.
Returns:
Categorical crossentropy loss value.
https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/keras/losses.py#L938-L966
"""
y_pred1 = y_pred[:, :num_class] # the 1st prediction
y_pred2 = y_pred[:, num_class:2*num_class] # the 2nd prediction
y_pred3 = y_pred[:, 2*num_class:] # the 3rd prediction
# you should adapt the ground truth to contain all 3 ground truth of course
y_true1 = y_true[:, :num_class] # the 1st gt
y_true2 = y_true[:, num_class:2*num_class] # the 2nd gt
y_true3 = y_true[:, 2*num_class:] # the 3rd gt
loss1 = K.categorical_crossentropy(y_true1, y_pred1, from_logits=from_logits)
loss2 = K.categorical_crossentropy(y_true2, y_pred2, from_logits=from_logits)
loss3 = K.categorical_crossentropy(y_true3, y_pred3, from_logits=from_logits)
# combine the losses the way you like it
total_loss = coeffs[0]*loss1 + coeffs[1]*(loss1 - loss2) + coeffs[2]*(loss2 - loss3)
return total_loss
return categorical_crossentropy
in1 = Input((6373,))
enc1 = Dense(num_nodes)(in1)
enc1 = Dropout(0.3)(enc1)
enc1 = Dense(num_nodes, activation='relu')(enc1)
enc1 = Dropout(0.3)(enc1)
enc1 = Dense(num_nodes, activation='relu')(enc1)
out1 = Dense(units=num_class, activation='softmax')(enc1)
in2 = Input((512,))
enc2 = Dense(num_nodes, activation='relu')(in2)
enc2 = Dense(num_nodes, activation='relu')(enc2)
out2 = Dense(units=num_class, activation='softmax')(enc2)
in3 = Input((768,))
enc3 = Dense(num_nodes, activation='relu')(in3)
enc3 = Dense(num_nodes, activation='relu')(enc3)
out3 = Dense(units=num_class, activation='softmax')(enc3)
adam = Adam(lr=0.0001)
total_out = Concatenate(axis=1)([out1, out2, out3])
model = Model(inputs=[in1, in2, in3], outputs=[total_out])
coeffs = [1, 1, 1]
model.compile(loss=categorical_crossentropy_base(coeffs=coeffs, num_class=num_class), optimizer='adam', metrics=['accuracy'])
不過，我不確定有關準確性的指標。但我認為無需其他更改即可發揮作用。我也在使用K.categorical_crossentropy，但是您當然也可以自由地使用其他實現來更改它。
添加回答
舉報
0/150
提交
取消