python

导航

python3爬虫如何带上cookie?

来源 :中华考试网 2020-10-11

  Cookie的英文原意是“点心”,它是在客户端访问Web服务器时,服务器在客户端硬盘上存放的信息,好像是服务器发送给客户的“点心”。服务器可以根据Cookie来跟踪客户状态,这对于需要区别客户的场合(如电子商务)特别有用。

  当客户端首次请求访问服务器时,服务器先在客户端存放包含该客户的相关信息的Cookie,以后客户端每次请求访问服务器时,都会在HTTP请求数据中包含Cookie,服务器解析HTTP请求中的Cookie,就能由此获得关于客户的相关信息。

  下面我们就来看一下python3爬虫带上cookie的方法:

  1、直接将Cookie写在header头部

  # coding:utf-8

  import requests

  from bs4 import BeautifulSoup

  cookie = '''cisession=19dfd70a27ec0eecf1fe3fc2e48b7f91c7c83c60;CNZZDATA1000201968=181584

  6425-1478580135-https%3A%2F%2Fwww.baidu.com%2F|1483922031;Hm_lvt_f805f7762a9a2

  37a0deac37015e9f6d9=1482722012,1483926313;Hm_lpvt_f805f7762a9a237a0deac37015e9f6d9=14839

  26368'''

  header = {

  'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Geck

  o) Chrome/53.0.2785.143 Safari/537.36',

  'Connection': 'keep-alive',

  'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',

  'Cookie': cookie}

  url = 'https://kankandou.com/book/view/22353.html'

  wbdata = requests.get(url,headers=header).text

  soup = BeautifulSoup(wbdata,'lxml')

  print(soup)

  2、使用requests插入Cookie

  # coding:utf-8

  import requests

  from bs4 import BeautifulSoup

  cookie = {

  "cisession":"19dfd70a27ec0eecf1fe3fc2e48b7f91c7c83c60",

  "CNZZDATA100020196":"1815846425-1478580135-https%3A%2F%2Fwww.baidu.com%2F|1483

  922031",

  "Hm_lvt_f805f7762a9a237a0deac37015e9f6d9":"1482722012,1483926313",

  "Hm_lpvt_f805f7762a9a237a0deac37015e9f6d9":"1483926368"

  }

  url = 'https://kankandou.com/book/view/22353.html'

  wbdata = requests.get(url,cookies=cookie).text

  soup = BeautifulSoup(wbdata,'lxml')

  print(soup)

分享到

相关资讯