通過查看網絡選項卡(在開發工具面板中)并重現刷新圖表時創建的請求,我找到了一種從華爾街日報的圖表頁面下載數據的方法。它的工作原理如下:import requestsimport jsonimport timefrom datetime import datetime as dtfrom urllib.parse import urlencode data = { "Step":"PT5M", "TimeFrame":"D1", "StartDate":int(dt(2019, 5, 1).timestamp()*1000), "EndDate":int(dt(2019, 5, 5).timestamp()*1000), "EntitlementToken":"57494d5ed7ad44af85bc59a51dd87c90", "IncludeMockTick":True, "FilterNullSlots":False, "FilterClosedPoints":True, "IncludeClosedSlots":False, "IncludeOfficialClose":True, "InjectOpen":False, "ShowPreMarket":False, "ShowAfterHours":False, "UseExtendedTimeFrame":True, "WantPriorClose":False, "IncludeCurrentQuotes":False, "ResetTodaysAfterHoursPercentChange":False, "Series":[{"Key":"STOCK/US/XNYS/ABB","Dialect":"Charting","Kind":"Ticker","SeriesId":"s1","DataTypes":["Last"],"Indicators":[{"Parameters":[{"Name":"Period","Value":"50"}],"Kind":"SimpleMovingAverage","SeriesId":"i2"},{"Parameters":[],"Kind":"Volume","SeriesId":"i3"}]}] }data = { 'json' : json.dumps(data)}data = urlencode(data)headers = { 'Accept': 'application/json, text/javascript, */*; q=0.01', 'Dylan2010.EntitlementToken': '57494d5ed7ad44af85bc59a51dd87c90', 'Origin': 'https://quotes.wsj.com', 'Referer': 'https://quotes.wsj.com/ABB/advanced-chart', 'Sec-Fetch-Mode': 'cors', 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 Safari/537.36'}url = 'https://api.wsj.net/api/michelangelo/timeseries/history?' + data + '&ckey=57494d5ed7'print(url)r = requests.get(url, headers = headers)r.text這很棒,因為它非常簡單且有效,但是我只能檢索最多 25 天左右的分鐘數據。另一方面,晨星圖表似乎有更多的分鐘數據可用,我想用它做同樣的事情:只需通過查找更新時在后臺進行的 javascript 調用來從網站獲取數據圖表。但是,當我查看網絡選項卡時,在更改日期范圍時看不到任何呼叫。我對 javascript 不太了解,想知道他們使用什么替代機制來實現這一目標。(也許是異步/獲???)有誰知道我怎么可能看到這些電話?
Scraping:從圖表中提取數據
慕哥9229398
2021-10-14 17:45:30