SoFunction
Updated on 2024-12-20

Python3 crawler of urllib carrying cookies to crawl the web page method

Python3 crawler of urllib carrying cookies to crawl the web page method

Updated December 28, 2018 11:49:52 by haeasringnar
Today I will share with you a Python3 crawler of urllib carrying cookies to crawl the web, with good reference value, I hope to help you. Together follow the editor over to see it

As shown below:

import 
import 
 
url = '/5273088553/info'
# Access in the normal way
# headers = {
#  'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36'
# }
# Carrying cookies for access
headers = {
'GET https':'///5273088553/info HTTP/1.1',
'Host':' ',
'Connection':' keep-alive',
'Upgrade-Insecure-Requests':' 1',
'User-Agent':' Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36',
'Accept':' text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
# 'Referer: https':'///',
'Accept-Language':' zh-CN,zh;q=0.9',
'Cookie':' _T_WM=c1913301844388de10cba9d0bb7bbf1e; SUB=_2A253Wy_dDeRhGeNM7FER-CbJzj-IHXVUp7GVrDV6PUJbkdANLXPdkW1NSesPJZ6v1GA5MyW2HEUb9ytQW3NYy19U; SUHB=0bt8SpepeGz439; SCF=Aua-HpSw5-z78-02NmUv8CTwXZCMN4XJ91qYSHkDXH4W9W0fCBpEI6Hy5E6vObeDqTXtfqobcD2D32r0O_5jSRk.; SSOLoginState=1516199821',
}
request = (url=url,headers=headers)
response = (request)
# Output all
# print(().decode('gbk'))
# Write the contents to a file
with open('','wb') as fp:
 (())

Above this Python3 crawler of urllib carrying cookies to crawl the web is all I have to share with you, I hope to be able to give you a reference, and I hope you will support me more.

  • Python3
  • urllib
  • cookie

Related articles

  • Python3 implementation of the code to print arbitrary width of the rhombus

    This article introduces the Python3 implementation of printing any width of the rhombus code, with good reference value, I hope to help you. Together, follow the editor over to see it
    2020-04-04

  • Example of a python text progress bar

    This article introduces the python text progress bar, with some reference value, interested partners can refer to, I hope to be able to bring you help!
    2021-12-12

  • Example of basic methods for loc and iloc fetching data in python DataFrame

    This article mainly gives you an introduction to the python DataFrame loc and iloc to take the data of the basic method, the text through the example code is very detailed, for everyone to learn or work with certain reference learning value, need friends can refer to the next!
    2022-02-02

  • Details of python reading .mat files and saving variables as .mat files

    This article introduces you to the python read .mat file and will be stored as a variable .mat file details, mat file is the standard format of matlab data storage, the text of the code through the very detailed, you can refer to the next!
    2024-06-06

  • Detailed python crawl pop-ups and data analysis

    This article introduces the detailed python crawl pop-ups and data analysis, this article gives you a very detailed introduction to everyone's learning or work has a certain reference value, the need for friends can refer to the following
    2020-11-11

  • Python implementation of a simple speech recognition system

    This article introduces the Python implementation of a simple speech recognition system, with some reference value, the need for friends can refer to.
    2017-12-12

  • 3 lines of Python code to implement clip music

    You think it takes a long time to edit music? The rest of the 3 lines of statements Python can be done instantly. In this article, we will talk about the realization of the steps in detail, the text of the sample code to explain the details, interested in trying their hands!
    2022-06-06

  • pycharm double click unresponsive (can not open the problem solution)

    This article introduces the pycharm double-click unresponsive (can not open the problem solution), the text through the sample code is very detailed, for everyone's learning or work has a certain reference and learning value, the need for friends below with the editorial to learn together!
    2020-01-01

  • python using yaml format files

    This article introduces the use of python yaml format file method, the text of the sample code through the introduction of the very detailed, for everyone's learning or work has a certain reference value of learning, the need for friends below with the editorial to learn together!
    2023-07-07

  • Solve the problem of Failed to install package when pycharm downloads libraries

    Many small partners encounter pycharm download library appears Failed to install package do not know how to solve, the following small make up to bring the solution, the need for friends refer to the next bar
    2021-09-09

Latest Comments