雖然能夠成功地抓取第一頁，但它不允許我執行第二頁。請注意，我不想對 Selinum 執行此操作。import requestsfrom bs4 import BeautifulSoupurl = 'https://google.com/search?q=In+order+to&hl=en'headers = {'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:77.0) Gecko/20100101 Firefox/77.0'}page = 1while True: print() print('Page {}...'.format(page)) print('-' * 80) soup = BeautifulSoup(requests.get(url, headers=headers).content, 'html.parser') for h in soup.select('h3'): print(h.get_text(strip=True)) next_link = soup.select_one('a:contains("Next")') if not next_link: break url = 'https://google.com' + next_link['href'] page += 1結果：Page 1...--------------------------------------------------------------------------------In order to Synonyms, In order to Antonyms | Thesaurus.comIn order to - English Grammar Today - Cambridge Dictionaryin order to - WiktionaryWhat is another word for "in order to"? - WordHippoIn Order For (someone or something) To | Definition of In ...In Order For | Definition of In Order For by Merriam-WebsterIn order to definition and meaning | Collins English DictionaryUsing "in order to" in English - English Study PageIN ORDER (FOR SOMEONE / SOMETHING ) TO DO ...262 In Order To synonyms - Other Words for In Order ToSearches related to In order toOnly the following pseudo-classes are implemented: nth-of-type.錯誤就出在這里：next_link = soup.select_one('a:contains("Next")')
1 回答
MMMHUHU
TA貢獻1834條經驗 獲得超8個贊
您可以用作lxml解析器而不是html.parser
安裝它pip install lxml
import requests
from bs4 import BeautifulSoup
url = 'https://google.com/search?q=In+order+to&hl=en'
headers = {'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:77.0) Gecko/20100101 Firefox/77.0'}
page = 1
while True:
print()
print('Page {}...'.format(page))
print('-' * 80)
soup = BeautifulSoup(requests.get(url, headers=headers).content, 'lxml')
for h in soup.select('h3'):
print(h.get_text(strip=True))
next_link = soup.select_one('a:contains("Next")')
if not next_link:
break
url = 'https://google.com' + next_link['href']
page += 1
添加回答
舉報
0/150
提交
取消