3 回答

TA貢獻1824條經驗 獲得超5個贊
在你的情況下tokens = {"Token": <last_token>}
像這樣修改您的代碼,以便可以為每個令牌發送一個請求。
tokens = set()
'''
<- You can use list also but in this case set is better as it will ensure only
one request for one token even if your tokens file contains duplicate line.
'''
url_list = [
"https://www.google.com/api/"
]
tokens = set()
with open("tokens.txt", "r") as f:
file_lines = f.readlines()
for line in file_lines:
tokens.add(line.strip())
token_data = {"Token": None}
def makerequest(url):
for token in tokens:
token_data["Token"] = token
html = requests.post(url,stream=True, data=token_data)
print(token)
# do something with html here
# don't return or break

TA貢獻2019條經驗 獲得超9個贊
你在做
數據 = 令牌
那一點tokens
是最后一行的分配。如果你想要所有的令牌,你需要做一些像j這樣的事情:
tokens = set() for line file_lines: tokens.add(......)

TA貢獻1844條經驗 獲得超8個贊
您的代碼的問題是令牌字典的創建 - 您循環提供令牌,但您總是覆蓋映射到“令牌”鍵的值。
此外,您的代碼中還有一些不良做法。
請像您一樣小心內聯打開文件
file_lines = open("tokens.txt", "r").readlines()
而是將其用作上下文管理器
with open("tokens.txt", "r") as file: file_lines = file.readlines()
這可以確保文件在您閱讀后再次關閉 - 在您的情況下,您需要確保文件被關閉(即使在崩潰等情況下)
其次,避免在函數中使用全局變量。根據你的代碼,我假設你想用每個標記查詢不同的 url - 所以函數應該接受這兩個作為參數。然后我會分別創建一個組合列表,比如
url_token_combs = [(url, token.strip()) for url in url_list for token in file_lines]
最后,更改您的功能以使用傳遞給它的參數而不是全局參數,例如:
def makerequest(url_token ): url , token = url_token html = requests.post(url,stream=True, data=token) return html.content
這使您現在可以使用以下線程遍歷代碼:
import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
from time import time
def makerequest(url_token):
url , token = url_token
html = requests.post(url,stream=True, data=tokens)
print(tokens)
return html.content
if __name__ == "__main__":
start = time()
url_list = [
"https://www.google.com/api/"
]
with open("tokens.txt", "r") as file:
file_lines = file.readlines()
tokens = [{'Token':line.replace('/n','') }for line in file_lines ]
url_tokens = [(url, token.strip()) for url in url_list for token in tokens]
processes = []
with ThreadPoolExecutor(max_workers=200) as executor:
for url_token in url_tokens:
processes.append(executor.submit(makerequest, url_token))
for task in as_completed(processes):
print(task.result())
print(f'Time taken: {time() - start}')
添加回答
舉報