我正在嘗試制作一個具有多線程的網絡爬蟲以使其更快。我想讓每次執行都增加價值。但有時該值會跳過或重復自身。import threadingnum = 0def scan(): while True: global num num += 1 print(num) open('logs.txt','a').write(str(f'{num}\n'))for x in range(500): threading.Thread(target=scan).start()結果：22557810101213131316171919222324252628292931323334預期結果：12345678910
3 回答
犯罪嫌疑人X
TA貢獻2080條經驗 獲得超4個贊
因此，由于該變量num是共享資源，因此您需要對其進行鎖定。這是按如下方式完成的：
num_lock = threading.Lock()
每次要更新共享變量時，都需要線程先獲取鎖。一旦獲得鎖，只有該線程可以更新 num 的值，而在當前線程獲得鎖時，其他線程將無法這樣做。
確保在執行此操作時使用waitortry-finally塊，以保證即使當前線程無法更新共享變量也將釋放鎖。
是這樣的：
num_lock.acquire()
try:
num+=1
finally:
num_lock.release()
使用with：
with num_lock:
num+=1
當年話下
TA貢獻1890條經驗 獲得超9個贊
一個重要的標注，除了threading.Lock：
用于
join使父線程等待分叉線程完成。沒有這個，線程仍然會競爭。
假設我正在使用num線程完成后：
import threading
lock, num = threading.Lock(), 0
def operation():
global num
print("Operation has started")
with lock:
num += 1
threads = [threading.Thread(target=operation) for x in range(10)]
for t in threads:
t.start()
for t in threads:
t.join()
print(num)
沒有連接，不一致（9 被打印一次，否則 10）：
Operation has started
Operation has started
Operation has started
Operation has started
Operation has startedOperation has started
Operation has started
Operation has started
Operation has started
Operation has started9
使用 join，它是一致的：
Operation has started
Operation has started
Operation has started
Operation has started
Operation has started
Operation has started
Operation has started
Operation has started
Operation has started
Operation has started
10
梵蒂岡之花
TA貢獻1900條經驗 獲得超5個贊
似乎是一種競爭條件。您可以使用鎖，以便只有一個線程可以獲得特定的數字。使用 lock 寫入輸出文件也是有意義的。
這是一個帶鎖的例子。當然，您不能保證輸出的寫入順序，但每個項目都應該恰好寫入一次。在此示例中，我添加了 10000 的限制，以便您可以更輕松地檢查最終是否將所有內容寫入測試代碼，因為否則無論您在什么時候中斷它，都很難驗證數字是被跳過還是只是等待鎖定以寫入輸出。
不是my_num共享的，因此您已經在該with num_lock部分內聲明了它之后，您可以自由釋放該鎖（保護共享num）然后繼續在my_num外部使用with，而其他線程可以訪問該鎖以聲明自己的價值. 這最大限度地減少了持有鎖的持續時間。
import threading
num = 0
num_lock = threading.Lock()
file_lock = threading.Lock()
def scan():
global num_lock, file_lock, num
while num < 10000:
with num_lock:
num += 1
my_num = num
# do whatever you want here using my_num
# but do not touch num
with file_lock:
open('logs.txt','a').write(str(f'{my_num}\n'))
threads = [threading.Thread(target=scan) for _ in range(500)]
for thread in threads:
thread.start()
for thread in threads:
thread.join()
添加回答
舉報
0/150
提交
取消