(爬虫入门)爬取东方财富网上的股票信息
·
爬取东方财富网上面的股票的数据,将不同页的股票信息爬取输出
随着时间推移,网站可能出现变更,并不能保证代码的有效性
代码如下
import re
import requests
import json
import pandas as pd
pd.set_option('display.max_columns', None)
pd.set_option('display.max_rows', None)
pd.set_option('display.unicode.ambiguous_as_wide', True)
pd.set_option('display.unicode.east_asian_width', True)
pd.set_option('display.width', 5000)
def HTML(url):
gupiao_list = []
headers = {"user-agent" : "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36"}
try:
r = requests.get(url, headers = headers, timeout = 30)
r.raise_for_status()
r.encoding = r.apparent_encoding
html = r.text
except Exception as e:
print("wrong:" + e)
pat = re.compile("\[\{.*?\}\]")
data = pat.findall(html)
js = json.loads(data[0])
gupiao_list.append(("代码","名称","最新价","涨跌幅","涨跌额","成交量","成交额","振幅","最高","最低","今开","昨收","量比"))
for i in range(len(js)):
diezhangfu = str(js[i]["f3"]) + "%"
zhenfu = str(js[i]["f7"]) + "%"
gupiao_list.append((js[i]["f12"],js[i]["f14"],js[i]["f2"],zhenfu,js[i]["f4"],js[i]["f5"],js[i]["f6"],zhenfu,js[i]["f15"],js[i]["f16"],js[i]["f17"],js[i]["f18"],js[i]["f10"]))
df = pd.DataFrame(gupiao_list)
return df
def main():
for i in range(1,21):
url = "https://30.push2.eastmoney.com/api/qt/clist/get?cb=jQuery1124004229850317009731_1585637567592&pn="+str(i)+"&pz=20&po=1&np=1&ut=bd1d9ddb04089700cf9c27f6f7426281&fltt=2&invt=2&fid=f3&fs=m:0+t:5&fields=f1,f2,f3,f4,f5,f6,f7,f8,f9,f10,f12,f13,f14,f15,f16,f17,f18,f20,f21,f23,f24,f25,f26,f22,f33,f11,f62,f128,f136,f115,f152&_=1585637567593"
print(HTML(url))
main()

魔乐社区(Modelers.cn) 是一个中立、公益的人工智能社区,提供人工智能工具、模型、数据的托管、展示与应用协同服务,为人工智能开发及爱好者搭建开放的学习交流平台。社区通过理事会方式运作,由全产业链共同建设、共同运营、共同享有,推动国产AI生态繁荣发展。
更多推荐


所有评论(0)