This article describes the implementation method of python concurrent downloader. Share it for your reference, as follows:
Concurrent downloader
Concurrent download principle
from gevent import monkey
import gevent
import
# It is required when there is time-consuming operationmonkey.patch_all()
def my_downLoad(url):
print('GET: %s' % url)
resp = (url)
data = ()
print('%d bytes received from %s.' % (len(data), url))
([
(my_downLoad, '/'),
(my_downLoad, '/'),
(my_downLoad, '/'),
])
Running results
GET: /
GET: /
GET: /
111327 bytes received from /.
172054 bytes received from /.
215035 bytes received from /.
From the above, we can see that the relevant information of baidu was sent first, and then itcast and itheima were successively. However, the order of data received is not necessarily the same as the order of sending, which reflects asynchronousness, that is, it is not sure when the data will be received, and the order is not certain.
Implement multiple video downloads
from gevent import monkey
import gevent
import
#This sentence is needed when doing it only with IOmonkey.patch_all()
def my_downLoad(file_name, url):
print('GET: %s' % url)
resp = (url)
data = ()
with open(file_name, "wb") as f:
(data)
print('%d bytes received from %s.' % (len(data), url))
([
(my_downLoad, "1.mp4", '/05day-08-%E3%80%90%E7%90%86%E8%A7%A3%E3%80%91%E5%87%BD%E6%95%B0%E4%BD%BF%E7%94%A8%E6%80%BB%E7%BB%93%EF%BC%88%E4%B8%80%EF%BC%89.mp4'),
(my_downLoad, "2.mp4", '/05day-03-%E3%80%90%E6%8E%8C%E6%8F%A1%E3%80%91%E6%97%A0%E5%8F%82%E6%95%B0%E6%97%A0%E8%BF%94%E5%9B%9E%E5%80%BC%E5%87%BD%E6%95%B0%E7%9A%84%E5%AE%9A%E4%B9%89%E3%80%81%E8%B0%83%E7%94%A8%28%E4%B8%8B%29.mp4'),
])
The URL above can be replaced with URLs that you need to download videos, music, pictures, etc.
For more information about Python, please view the special topic of this site: "Summary of Python Socket Programming Tips》、《Python data structure and algorithm tutorial》、《Summary of Python function usage tips》、《Summary of Python string operation skills》、《Python introduction and advanced classic tutorials"and"Summary of Python file and directory operation skills》
I hope this article will be helpful to everyone's Python programming.