我的最終目標是使用 NER 模型來識別自定義實體。在此之前，我將文本數據標記為單詞和句子。我有一個文本文件（.txt）文件夾，我使用操作系統庫打開并讀入 Jupyter。讀取文本文件后，每當我嘗試標記文本文件時，都會收到類型錯誤。請告訴我我做錯了什么？我的代碼如下，謝謝。import osoutfile = open('result.txt', 'w')path = "C:/Users/okeke/Documents/Work flow/IT Text analytics Project/Extract/Dubuque_text-nlp"files = os.listdir(path)for file in files: outfile.write(str(os.stat(path + "/" + file).st_size) + '\n')outfile.close()這段代碼運行良好，每當我運行輸出文件時，我都會在下面得到這個outfile<_io.TextIOWrapper name='result.txt' mode='w' encoding='cp1252'>接下來，標記化。from nltk.tokenize import sent_tokenize, word_tokenize sent_tokens = sent_tokenize(outfile)print(outfile)word_tokens = word_tokenize(outfile)print(outfile但運行上面的代碼后出現錯誤。檢查下面是否有錯誤---------------------------------------------------------------------------TypeError Traceback (most recent call last)<ipython-input-22-62f66183895a> in <module> 1 from nltk.tokenize import sent_tokenize, word_tokenize----> 2 sent_tokens = sent_tokenize(outfile) 3 print(outfile) 4 5 #word_tokens = word_tokenize(text)~\AppData\Local\Continuum\anaconda3\envs\nlp_course\lib\site-packages\nltk\tokenize\__init__.py in sent_tokenize(text, language) 93 """ 94 tokenizer = load('tokenizers/punkt/{0}.pickle'.format(language))---> 95 return tokenizer.tokenize(text) 96 97 # Standard word tokenizer.TypeError: expected string or bytes-like object
1 回答
不負相思意
TA貢獻1777條經驗 獲得超10個贊
（移動評論來回答）
您正在嘗試處理文件對象而不是文件中的文本。創建文本文件后，重新打開它并在標記化之前讀取整個文件。
試試這個代碼：
import os
outfile = open('result.txt', 'w')
path = "C:/Users/okeke/Documents/Work flow/IT Text analytics Project/Extract/Dubuque_text-nlp"
files = os.listdir(path)
for file in files:
with open(path + "/" + file) as f:
outfile.write(f.read() + '\n')
#outfile.write(str(os.stat(path + "/" + file).st_size) + '\n')
outfile.close() # done writing
from nltk.tokenize import sent_tokenize, word_tokenize
with open('result.txt') as outfile: # open for read
alltext = outfile.read() # read entire file
print(alltext)
sent_tokens = sent_tokenize(alltext) # process file text. tokenize sentences
word_tokens = word_tokenize(alltext) # process file text. tokenize words
添加回答
舉報
0/150
提交
取消