爬取东方财富网上面的股票的数据,将不同页的股票信息爬取输出

随着时间推移,网站可能出现变更,并不能保证代码的有效性

代码如下
import re
import requests
import json
import pandas as pd


pd.set_option('display.max_columns', None)
pd.set_option('display.max_rows', None)
pd.set_option('display.unicode.ambiguous_as_wide', True)
pd.set_option('display.unicode.east_asian_width', True)
pd.set_option('display.width', 5000)

def HTML(url):
    gupiao_list = []
    headers = {"user-agent" : "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36"}
    try:
        r = requests.get(url, headers = headers, timeout = 30)
        r.raise_for_status()
        r.encoding = r.apparent_encoding
        html = r.text
    except Exception as e:
        print("wrong:" + e)
    pat = re.compile("\[\{.*?\}\]")
    data = pat.findall(html)
    js = json.loads(data[0])
    gupiao_list.append(("代码","名称","最新价","涨跌幅","涨跌额","成交量","成交额","振幅","最高","最低","今开","昨收","量比"))
    for i in range(len(js)):
        diezhangfu = str(js[i]["f3"]) + "%"
        zhenfu = str(js[i]["f7"]) + "%"
        gupiao_list.append((js[i]["f12"],js[i]["f14"],js[i]["f2"],zhenfu,js[i]["f4"],js[i]["f5"],js[i]["f6"],zhenfu,js[i]["f15"],js[i]["f16"],js[i]["f17"],js[i]["f18"],js[i]["f10"]))
    df = pd.DataFrame(gupiao_list)
    return df
    
def main():
    for i in range(1,21):
        url = "https://30.push2.eastmoney.com/api/qt/clist/get?cb=jQuery1124004229850317009731_1585637567592&pn="+str(i)+"&pz=20&po=1&np=1&ut=bd1d9ddb04089700cf9c27f6f7426281&fltt=2&invt=2&fid=f3&fs=m:0+t:5&fields=f1,f2,f3,f4,f5,f6,f7,f8,f9,f10,f12,f13,f14,f15,f16,f17,f18,f20,f21,f23,f24,f25,f26,f22,f33,f11,f62,f128,f136,f115,f152&_=1585637567593"
        print(HTML(url))

    
main()

结果显示

Logo

魔乐社区(Modelers.cn) 是一个中立、公益的人工智能社区,提供人工智能工具、模型、数据的托管、展示与应用协同服务,为人工智能开发及爱好者搭建开放的学习交流平台。社区通过理事会方式运作,由全产业链共同建设、共同运营、共同享有,推动国产AI生态繁荣发展。

更多推荐