亚洲在线久爱草,狠狠天天香蕉网,天天搞日日干久草,伊人亚洲日本欧美

為了賬號安全,請及時綁定郵箱和手機立即綁定
已解決430363個問題,去搜搜看,總會有你想問的

在 keras 中同時訓練神經網絡并讓它們在訓練時共同分擔損失?

在 keras 中同時訓練神經網絡并讓它們在訓練時共同分擔損失?

慕慕森 2024-01-24 15:37:17
假設我想同時訓練三個模型(模型 1、模型 2 和模型 3),并且在訓練時讓模型一和模型二與主網絡(模型 1)共同共享損失。因此主模型可以從層間的其他兩個模型中學習表示。損失總計 = (權重1)損失m1 + (權重2)(損失m1 - 損失m2) + (權重3)(損失m1 - 損失m3)到目前為止我有以下內容:def threemodel(num_nodes, num_class, w1, w2, w3):    #w1; w2; w3 are loss weights        in1 = Input((6373,))    enc1 = Dense(num_nodes)(in1)    enc1 = Dropout(0.3)(enc1)    enc1 = Dense(num_nodes, activation='relu')(enc1)    enc1 = Dropout(0.3)(enc1)    enc1 = Dense(num_nodes, activation='relu')(enc1)    out1 = Dense(units=num_class, activation='softmax')(enc1)        in2 = Input((512,))    enc2 = Dense(num_nodes, activation='relu')(in2)    enc2 = Dense(num_nodes, activation='relu')(enc2)        out2 = Dense(units=num_class, activation='softmax')(enc2)        in3 = Input((768,))    enc3 = Dense(num_nodes, activation='relu')(in3)    enc3 = Dense(num_nodes, activation='relu')(enc3)        out3 = Dense(units=num_class, activation='softmax')(enc3)        adam = Adam(lr=0.0001)        model = Model(inputs=[in1, in2, in3], outputs=[out1, out2, out3])        model.compile(loss='categorical_crossentropy', #continu together          optimizer='adam',          metrics=['accuracy'] not sure know what changes need to be made here)## I am confused on how to formulate the shared losses equation here to share the losses of out2 and out3 with out1.經過一番搜索后,似乎可以執行以下操作:loss_1 = tf.keras.losses.categorical_crossentropy(y_true_1, out1)  loss_2 = tf.keras.losses.categorical_crossentropy(y_true_2, out2)  loss_3 = tf.keras.losses.categorical_crossentropy(y_true_3, out3)  model.add_loss((w1)*loss_1 + (w2)*(loss_1 - loss_2) + (w3)*(loss_1 - loss_3))這可以嗎?我覺得通過執行上面建議的操作并沒有真正執行我想要的操作,即讓主模型(mod1)從各層之間的其他兩個模型(mod2 和 mod3)學習表示。有什么建議么?
查看完整描述

1 回答

?
尚方寶劍之說

TA貢獻1788條經驗 獲得超4個贊

由于您對使用可訓練權重不感興趣(我將它們標記為系數以將它們與可訓練權重區分開),您可以連接輸出并將它們作為單個輸出傳遞給自定義損失函數。這意味著這些系數將在訓練開始時可用。


您應該提供如上所述的自定義損失函數。損失函數預計只接受 2 個參數,因此您應該使用這樣一個函數categorical_crossentropy,它也應該熟悉您感興趣的參數,例如coeffs和num_class。因此,我使用所需的參數實例化一個包裝函數,然后將內部實際損失函數作為主損失函數傳遞。


from tensorflow.keras.layers import Dense, Dropout, Input, Concatenate

from tensorflow.keras.optimizers import Adam

from tensorflow.keras.models import Model


from tensorflow.python.framework import ops

from tensorflow.python.framework import smart_cond

from tensorflow.python.ops import math_ops

from tensorflow.python.ops import array_ops

from tensorflow.python.keras import backend as K



def categorical_crossentropy_base(coeffs, num_class):


    def categorical_crossentropy(y_true, y_pred, from_logits=False, label_smoothing=0):

        """Computes the categorical crossentropy loss.

      Args:

        y_true: tensor of true targets.

        y_pred: tensor of predicted targets.

        from_logits: Whether `y_pred` is expected to be a logits tensor. By default,

          we assume that `y_pred` encodes a probability distribution.

        label_smoothing: Float in [0, 1]. If > `0` then smooth the labels.

      Returns:

        Categorical crossentropy loss value.

        https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/keras/losses.py#L938-L966

      """

        y_pred1 = y_pred[:, :num_class]  # the 1st prediction

        y_pred2 = y_pred[:, num_class:2*num_class]  # the 2nd prediction

        y_pred3 = y_pred[:, 2*num_class:]  # the 3rd prediction


        # you should adapt the ground truth to contain all 3 ground truth of course

        y_true1 = y_true[:, :num_class]  # the 1st gt

        y_true2 = y_true[:, num_class:2*num_class]  # the 2nd gt

        y_true3 = y_true[:, 2*num_class:]  # the 3rd gt


        loss1 = K.categorical_crossentropy(y_true1, y_pred1, from_logits=from_logits)

        loss2 = K.categorical_crossentropy(y_true2, y_pred2, from_logits=from_logits)

        loss3 = K.categorical_crossentropy(y_true3, y_pred3, from_logits=from_logits)


        # combine the losses the way you like it

        total_loss = coeffs[0]*loss1 + coeffs[1]*(loss1 - loss2) + coeffs[2]*(loss2 - loss3)

        return total_loss


    return categorical_crossentropy


in1 = Input((6373,))

enc1 = Dense(num_nodes)(in1)

enc1 = Dropout(0.3)(enc1)

enc1 = Dense(num_nodes, activation='relu')(enc1)

enc1 = Dropout(0.3)(enc1)

enc1 = Dense(num_nodes, activation='relu')(enc1)

out1 = Dense(units=num_class, activation='softmax')(enc1)


in2 = Input((512,))

enc2 = Dense(num_nodes, activation='relu')(in2)

enc2 = Dense(num_nodes, activation='relu')(enc2)

out2 = Dense(units=num_class, activation='softmax')(enc2)


in3 = Input((768,))

enc3 = Dense(num_nodes, activation='relu')(in3)

enc3 = Dense(num_nodes, activation='relu')(enc3)

out3 = Dense(units=num_class, activation='softmax')(enc3)


adam = Adam(lr=0.0001)


total_out = Concatenate(axis=1)([out1, out2, out3])

model = Model(inputs=[in1, in2, in3], outputs=[total_out])


coeffs = [1, 1, 1]

model.compile(loss=categorical_crossentropy_base(coeffs=coeffs, num_class=num_class),  optimizer='adam', metrics=['accuracy'])

不過,我不確定有關準確性的指標。但我認為無需其他更改即可發揮作用。我也在使用K.categorical_crossentropy,但是您當然也可以自由地使用其他實現來更改它。


查看完整回答
反對 回復 2024-01-24
  • 1 回答
  • 0 關注
  • 143 瀏覽
慕課專欄
更多

添加回答

舉報

0/150
提交
取消
微信客服

購課補貼
聯系客服咨詢優惠詳情

幫助反饋 APP下載

慕課網APP
您的移動學習伙伴

公眾號

掃描二維碼
關注慕課網微信公眾號