多线程爬取元气壁纸中的图片

引言

前几天我手一哆嗦安装了附加软件,结果安装了个元气壁纸,感觉里面的质量还不戳,那就爬下来吧。

过程

因为是客户端,也就是C/S架构的,那就考虑一下监听数据包吧,上wireshark就太大材小用了,fiddler就足够用了,加上proxifier全局代理,很容易就找到了真实的url,结果很搞笑,居然有web版的,看样子是B/S架构的程序,只不过通过应用进行了一次封装。

注意

我发现最近提交的博文中,存在下载的代码,选用的都是z盘,使用过程中需要自己改掉奥。z盘是怎么出来的建议看一下之前写的博文--Ramdisk 内存盘提高系统性能。

技术点

  1. python
  2. request
  3. json
  4. threading

代码

import requests
import json
import threading
import random


def get_requestHeaders():
    headers = {
        'accept': 'application/json, text/plain, */*',
        'accept-encoding': 'gzip, deflate, br',
        'accept-language': 'zh-CN,zh;q=0.9',
        'content-length': '172',
        'content-type': 'application/json;charset=UTF-8',
        'dnt': '1',
        'origin': 'https://wallpaper.zhhainiao.com',
        'referer': 'https://wallpaper.zhhainiao.com/',
        'sec-ch-ua': '"Google Chrome";v="89", "Chromium";v="89", ";Not A Brand";v="99"',
        'sec-ch-ua-mobile': '?0',
        'sec-fetch-dest': 'empty',
        'sec-fetch-mode': 'cors',
        'sec-fetch-site': 'same-site',
        'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.82 Safari/537.36',
        'x-cf-device-id': 'xxxx-xxx-xxx',
        'x-cf-platform': 'webview',

    }
    return headers


def thd(dt,page):
    for i in dicts:
        if i["Jpg4kUrl"] != "":
            url = i["Jpg4kUrl"]
        elif i["Jpg1920Url"] != "":
            url = i["Jpg1920Url"]
        name = i["wname"]
        if name == '':
            name = str(random.randint(1, 9)) + str(random.randint(1, 99)) + str(random.randint(1, 999999)) + ".jpg"

        print(url, name,page)
        pic = requests.get(url).content
        with open("z:/spider/" + name + ".jpg", 'wb') as f:
            f.write(pic)


if __name__ == '__main__':

    for page in range(1, 100):
        r = requests.post("https://pcwallpaper.zhhainiao.com/wallpaper/static/list",
                          headers=get_requestHeaders(),
                          data='{"login_info":{},"cate_id":2,"tag_id":null,"sort_type":2,"page":' + str(page) +
                               ',"page_size":24,'
                               '"common":{"open_id":null,"token":null,"device_id":null,"player_version":0,"platform":"pc"}}')
        json_data = r.text
        dumps = json.loads(json_data)
        dicts = dumps["data"]["list"]
        url = ""
        name = ""
        threading.Thread(target=thd, args=(dicts,page,)).start()

评论 0

还没有评论,来说点什么吧。

提交后需要审核才会公开。