记录在爬取微博博主微博数据时使用Python3中Json数据遍历取指定值
json地址
请求的json地址为：https://m.weibo.cn/api/container/getIndex?type=uid&value=6414205745&containerid=1076036414205745
遍历数据
遍历json数据中“data”–“cards”–“mblog”–“page_info”–“media_info”下的视频链接地址，即“stream_url”中的http://f.video.weibocdn.com/7zCpXkfllx07CUfbom0001041201UmjF0E010.mp4?label=mp4_ld&template=640×360.25.0&trans_finger=40a32e8439c5409a63ccf853562a60ef&ori=0&ps=1CwnkDw1GXwCQx&Expires=1602643858&ssig=U2pxNFDptM&KID=unistore,video
Python代码
- import requests
- headers = {
- ‘User-Agent’: ‘Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/604.1.38 (KHTML, like Gecko) Version/11.0 Mobile/15A372 Safari/604.1’,
- ‘Cookie‘: ‘_T_WM=78921707454; XSRF-TOKEN=fe3a88; WEIBOCN_FROM=1110003030; MLOGIN=0; M_WEIBOCN_PARAMS=luicode%3D10000011%26lfid%3D1076036414205745%26fid%3D1005056414205745%26uicode%3D10000011’,
- }
- params ={
- ‘type‘: ‘uid’,
- ‘value’: 6414205745,
- ‘containerid’: 1076036414205745
- }
- res = requests.get(‘https://m.weibo.cn/api/container/getIndex’, headers=headers, params=params).json()
- for item in res[‘data’][‘cards’]:
- try:
- if ‘retweeted_status’ in item[‘mblog’].keys():
- print(item[‘mblog’][‘retweeted_status’][‘page_info’][‘media_info’][‘stream_url’])
- else:
- print(item[‘mblog’][‘page_info’][‘media_info’][‘stream_url’])
- except Exception as e:
- pass
代码说明
1.import requests 即引入requests请求库
- import requests
2.headers即构造爬虫请求头
- headers = {
- ‘User-Agent’: ‘Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/604.1.38 (KHTML, like Gecko) Version/11.0 Mobile/15A372 Safari/604.1’,
- ‘Cookie‘: ‘_T_WM=78921707454; XSRF-TOKEN=fe3a88; WEIBOCN_FROM=1110003030; MLOGIN=0; M_WEIBOCN_PARAMS=luicode%3D10000011%26lfid%3D1076036414205745%26fid%3D1005056414205745%26uicode%3D10000011’,
- }
3.params即url参数传递
- params ={
- ‘type‘: ‘uid’,
- ‘value’: 6414205745,
- ‘containerid’: 1076036414205745
- }
4.res即请求后返回的json数据
- res = requests.get(‘https://m.weibo.cn/api/container/getIndex’, headers=headers, params=params).json()
5.for item in res[‘data’][‘cards’]:即在函数为res的json数据[‘data’][‘cards’]目录下循环
6.判断[‘mblog’]里是否包含retweeted_status，如果包含返回[‘mblog’][‘retweeted_status’][‘page_info’][‘media_info’]下stream_url对应值
- if ‘retweeted_status’ in item[‘mblog’].keys():
- print(item[‘mblog’][‘retweeted_status’][‘page_info’][‘media_info’][‘stream_url’])
7.如果不包含retweeted_status，则返回[‘mblog’][‘page_info’][‘media_info’]下的stream_url对应值
- else:
- print(item[‘mblog’][‘page_info’][‘media_info’][‘stream_url’])
运行结果
- http://f.video.weibocdn.com/7zCpXkfllx07CUfbom0001041201UmjF0E010.mp4?label=mp4_ld&template=640×360.25.0&trans_finger=40a32e8439c5409a63ccf853562a60ef&ori=0&ps=1CwnkDw1GXwCQx&Expires=1602645297&ssig=d1ZuPIEVDE&KID=unistore,video
- http://f.video.weibocdn.com/6gaRTrhplx07Hb7p05fO01041200KXvn0E010.mp4?label=mp4_ld&template=640×360.25.0&trans_finger=6006a648d0db83b7d9951b3cee381a9c&ori=0&ps=1CwnkDw1GXwCQx&Expires=1602645297&ssig=bkDerNKeQF&KID=unistore,video
- http://f.video.weibocdn.com/6gaRTrhplx07Hb7p05fO01041200KXvn0E010.mp4?label=mp4_ld&template=640×360.25.0&trans_finger=6006a648d0db83b7d9951b3cee381a9c&ori=0&ps=1CwnkDw1GXwCQx&Expires=1602645297&ssig=bkDerNKeQF&KID=unistore,video
- http://f.video.weibocdn.com/6gaRTrhplx07Hb7p05fO01041200KXvn0E010.mp4?label=mp4_ld&template=640×360.25.0&trans_finger=6006a648d0db83b7d9951b3cee381a9c&ori=0&ps=1CwnkDw1GXwCQx&Expires=1602645297&ssig=bkDerNKeQF&KID=unistore,video
- http://f.video.weibocdn.com/ClGEq7yclx07H8NkMC2Q0104120053fx0E010.mp4?label=mp4_ld&template=640×360.25.0&trans_finger=40a32e8439c5409a63ccf853562a60ef&ori=0&ps=1CwnkDw1GXwCQx&Expires=1602645297&ssig=lj%2FqbLLZ8r&KID=unistore,video
- http://f.video.weibocdn.com/5M4bafcxlx07H6kUixWE01041201jIvC0E010.mp4?label=mp4_ld&template=640×360.25.0&trans_finger=6006a648d0db83b7d9951b3cee381a9c&ori=0&ps=1CwnkDw1GXwCQx&Expires=1602645297&ssig=S73r3lf5p3&KID=unistore,video
- http://f.video.weibocdn.com/5M4bafcxlx07H6kUixWE01041201jIvC0E010.mp4?label=mp4_ld&template=640×360.25.0&trans_finger=6006a648d0db83b7d9951b3cee381a9c&ori=0&ps=1CwnkDw1GXwCQx&Expires=1602645297&ssig=S73r3lf5p3&KID=unistore,video
- http://f.video.weibocdn.com/5M4bafcxlx07H6kUixWE01041201jIvC0E010.mp4?label=mp4_ld&template=640×360.25.0&trans_finger=6006a648d0db83b7d9951b3cee381a9c&ori=0&ps=1CwnkDw1GXwCQx&Expires=1602645297&ssig=S73r3lf5p3&KID=unistore,video