我正在嘗試制作一個 RNN 模型（在 Pytorch 中），它需要幾個句子，然后將其分類為Class 0或Class 1。為了這個問題，我們假設句子的 max_len 為 4，時間步長的 max_amount 為 5。因此，每個數據點都在表單上（0 是用于填充填充值的值）： x[1] = [ # Input features at timestep 1 [1, 48, 91, 0], # Input features at timestep 2 [20, 5, 17, 32], # Input features at timestep 3 [12, 18, 0, 0], # Input features at timestep 4 [0, 0, 0, 0], # Input features at timestep 5 [0, 0, 0, 0] ] y[1] = [1]當我每個目標只有一個句子時：我只是將每個單詞傳遞給嵌入層，然后傳遞給 LSTM 或 GRU，但是當每個目標有一系列句子時，我有點卡住了怎么辦？如何構建可以處理句子的嵌入？
1 回答
幕布斯6054654
TA貢獻1876條經驗 獲得超7個贊
最簡單的方法是使用 2 種 LSTM。
準備玩具數據集
xi = [
# Input features at timestep 1
[1, 48, 91, 0],
# Input features at timestep 2
[20, 5, 17, 32],
# Input features at timestep 3
[12, 18, 0, 0],
# Input features at timestep 4
[0, 0, 0, 0],
# Input features at timestep 5
[0, 0, 0, 0]
]
yi = 1
x = torch.tensor([xi, xi])
y = torch.tensor([yi, yi])
print(x.shape)
# torch.Size([2, 5, 4])
print(y.shape)
# torch.Size([2])
然后，x是輸入的批次。這里batch_size= 2。
嵌入輸入
vocab_size = 1000
embed_size = 100
hidden_size = 200
embed = nn.Embedding(vocab_size, embed_size)
# shape [2, 5, 4, 100]
x = embed(x)
第一個詞-LSTM是將每個序列編碼成一個向量
# convert x into a batch of sequences
# Reshape into [2, 20, 100]
x = x.view(bs * 5, 4, 100)
wlstm = nn.LSTM(embed_size, hidden_size, batch_first=True)
# get the only final hidden state of each sequence
_, (hn, _) = wlstm(x)
# hn shape [1, 10, 200]
# get the output of final layer
hn = hn[0] # [10, 200]
第二個seq-LSTM是將序列編碼成單個向量
# Reshape hn into [bs, num_seq, hidden_size]
hn = hn.view(2, 5, 200)
# Pass to another LSTM and get the final state hn
slstm = nn.LSTM(hidden_size, hidden_size, batch_first=True)
_, (hn, _) = slstm(hn) # [1, 2, 200]
# Similarly, get the hidden state of the last layer
hn = hn[0] # [2, 200]
添加一些分類層
pred_linear = nn.Linear(hidden_size, 1)
# [2, 1]
output = torch.sigmoid(pred_linear(hn))
添加回答
舉報
0/150
提交
取消