How to obtain and download images using Selenium batch automation

Selenium batch automation to obtain and download pictures

In modern web development, automated testing and data crawling have become an indispensable part. As a powerful automated testing tool, Selenium can be used not only to test web applications, but also to obtain pictures on web pages in batches. This article will introduce how to use Selenium batch automation to obtain and download images.

1. Preparation

Install Selenium: First, you need to install Selenium in your system. You can use pip to install:

pip install selenium

Download WebDriver: Selenium needs to be used in conjunction with WebDriver, a browser driver used to perform browser operations. You need to download the corresponding WebDriver according to your browser type. For example, if you are using Chrome, you need to download ChromeDriver.

Import the necessary libraries: In Python scripts, you need to import Selenium's libraries and libraries used to process images (such as PIL).

from selenium import webdriver  
from PIL import Image  
import io

2. Write code

Start the browser driver: Create a browser driver instance and open a web page. Here is a Chrome example:

driver = ('path/to/chromedriver')  
('')

Find image elements: Use Selenium's locators to find image elements on the page. You can use CSS selector, XPath, etc. to locate image elements. For example, use the CSS selector to locate all img tags:

images = driver.find_elements_by_css_selector('img')

Download the image: For each found image element, extract its src attribute and then use the PIL library to download the image. Here is a simple example:

for image in images:  
    image_url = image.get_attribute('src')  
    response = driver.execute_script("return fetch('" + image_url + "');")  
    image_data =   
    image = ((image_data))  
    (image_url.split('/')[-1])  # Save the image locally and keep the file name unchanged

Close the browser driver: After completing the image download, close the browser driver.

()

3. Things to note

Cross-domain problem: If the image comes from a different domain, you may encounter cross-domain problem. In this case, you need to deal with the browser's homologous policy restrictions. One workaround is to use a browser plugin or service to allow cross-domain requests.
Image size and resolution: When downloading images, you may need to adjust the size and resolution of the images to suit your needs. You can use the PIL library's resize method to resize the image when saving the image. For example: ((width, height)).

Complete example:

from selenium import webdriver
from  import By
import time
import requests
from io import BytesIO
from PIL import Image

def spider(url):
    driver = ()
    (url)
    driver.maximize_window()  # Window Maximization    (30)
    a=1
    for j in range(2,25):
        try:
            try:
                for i in range(1,100):
                    a+=1
                    image_url=driver.find_element(,'//*[@]/p/img[{}]'.format(i)).get_attribute("src")
                    print(image_url)
                    #//*[@]/div/div/div/div[1]/div/div[2]/div/div[3]/div/div/div/div[4]
                    #//*[@]/div/div/div/div[1]/div/div[2]/div/div[3]/div/div/div/div[23]
                    response = (image_url)
                    image_data = BytesIO()
                    image = (image_data)
                    ("image/{}.jpg".format(a))
                    print("{}Download successfully".format(a))
            except:
                print("Downloaded")
            driver.find_element(,'//*[@]/div/div/div/div[1]/div/div[2]/div/div[3]/div/div/div/div[{}]'.format(j+1)).click()
            (0.5)
        except:
            print("No download.")
            pass
    ()
    quit()

if __name__ == '__main__':
    spider('/web/#/Previewtextbook?contentId_=e8e86045f7a441e5828da5c3b21bcacc&uip=1')

This is the article about how to obtain and download pictures using Selenium batch automation. For more related Python Selenium to obtain and download pictures, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!