Python3 crawler to crawl the people's network list and save as json function example [based on request, lxml and json module].

In this article, the example of Python3 crawler to crawl the list of people's network and save it as a json function. Shared for your reference, as follows:

python3 crawler of crawling the people's network list and save as a json file. These days have been learning to use python3 crawl data, today record, the code is very simple and easy to get started.

First of all, you need to install python3. If you don't have it yet, you can refer to the python3 installation and configuration article on this site.

First you need to install therequestscap (a poem)lxmlcap (a poem)jsonThree modules

Manual file creation is required

coding

import requests
from lxml import etree
import json
# Construct headers to simulate browser access
url="/meirongfuwu/"
headers = {'User-Agent':'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36','referer':url}
response=(url,headers=headers)
body= #Fetching web content
html=(body,())
gethtml=('//div[contains(@class,"media-body-title")]')
# Stored as an array list
jsondata = []
for item in gethtml:
  jsonone={}
  jsonone['title']=('.//a[contains(@class,"ad-title")]/text()')[0]
  jsonone['url']=('.//a[contains(@class,"ad-title")]/attribute::href')[0]
  jsonone['phone']=('.//button[contains(@class,"contact-button")]/attribute::data-contact')[0]
  (jsonone)
# Save as json
with open("./",'w',encoding='utf-8') as json_file:
  (jsondata,json_file,ensure_ascii=False)

in the end

PS: Here we recommend a few more practical json online tools for your reference:

on-lineJSONCode checking, inspection, beautification and formatting tools:
http://tools./code/json

JSONOnline formatting tool:
http://tools./code/jsonformat

Online XML/JSONInterconversion tool:
http://tools./code/xmljson

jsonCode online formatting/beautification/compression/editing/conversion tool:
http://tools./code/jsoncodeformat

on-linejsonCompression/Endorsing Tool:
http://tools./code/json_yasuo_trans

More about Python related content can be viewed on this site's topic: thePython Socket Programming Tips Summary》、《Python Regular Expression Usage Summary》、《Python Data Structures and Algorithms Tutorial》、《Summary of Python function usage tips》、《Summary of Python string manipulation techniques》、《Python introductory and advanced classic tutorialsand theSummary of Python file and directory manipulation techniques》

I hope the description of this article will help you in Python programming.