soup 對象為none的問題
craw 350 : https://baike.baidu.com/item/%E6%A2%85%E7%88%B1%E8%8A%B3/983014
craw 351 : https://baike.baidu.com/item/%E5%93%88%E5%A8%81%C2%B7%E5%87%AF%E7%89%B9%E5%B0%94/4528376
Traceback (most recent call last):
? File "c:/Users/VULCAN/Desktop/文件/爬蟲學習.py", line 127, in <module>
? ? obj_spider.craw(root_url)
? File "c:/Users/VULCAN/Desktop/文件/爬蟲學習.py", line 114, in craw
? ? new_urls, new_data = self.parser.parse(new_url, html_cont)
? File "c:/Users/VULCAN/Desktop/文件/爬蟲學習.py", line 69, in parse
? ? new_data = self._get_new_data(page_url, soup)
? File "c:/Users/VULCAN/Desktop/文件/爬蟲學習.py", line 57, in _get_new_data
? ? title_node = soup.find('dd', class_="lemmaWgt-lemmaTitle-title").find("h1")
AttributeError: 'NoneType' object has no attribute 'find'
爬取到一半后報錯,相關代碼如下
def?parse(self,?page_url,?html_cont): ????if?page_url?is?None?or?html_cont?is?None: ????????return ????soup?=?BeautifulSoup(html_cont,?'html.parser',?from_encoding='utf-8') ????new_urls?=?self._get_new_urls(page_url,?soup) ????new_data?=?self._get_new_data(page_url,?soup) ????return?new_urls,?new_data
求大神解答??!
2020-03-29
已解決,自己看錯了??