Several ways to solve the problem of pip install numpy too slow

introduction

When doing Python scientific computing, data analysis or machine learning,numpyIt is one of the most basic and most commonly used libraries. However, many users are installingnumpyWhen you are experiencing extremely slow download speed or even failure. This article will introduce in detail how to speed up from multiple perspectives such as problem analysis, solutions, and optimization suggestions.numpyand compare the advantages and disadvantages of different methods. In addition, we will also explore how to call Python-generated CSV data in Java to achieve cross-language collaboration.

1. Why is pip install numpy very slow?

1.1 The official PyPI server abroad

The default server of Python Package Index (PyPI) is located abroad. Domestic users may be affected by network latency, firewall restrictions or international bandwidth when accessing, resulting in slow download speed.

1.2 There are many numpy dependencies

numpyIt is a scientific computing library with underlying dependenciesBLAS、LAPACKHigh-performance math libraries such as high-performance math libraries need to be compiled or downloaded during installation, resulting in a longer installation time.

1.3 Network caching issues

If the previous installation fails,pipYou may try to use cached files, resulting in duplicate downloads or stuttering.

2. 7 ways to speed up pip install numpy

2.1 Using domestic mirror source (recommended)

Domestic universities and enterprises maintain PyPI mirror source, which can greatly improve download speed. Commonly used mirror sources include:

• Tsinghua mirror:/simple
• Alibaba Cloud Mirror:/pypi/simple/
• Douban mirror:/simple/

Temporary use of mirrors

pip install numpy -i /simple

Permanently modify the pip image

exist~/.pip/(Linux/Mac) orC:\Users\<username>\pip\(Windows) added:

[global]
index-url = /simple
trusted-host = 

2.2 Install using conda (suitable for Anaconda users)

If Anaconda or Miniconda is installed, you can usecondaInstallnumpy, its default source is usually faster than PyPI:

conda install numpy

Configure conda domestic mirror

conda config --add channels /anaconda/pkgs/free/
conda config --set show_channel_urls yes

2.3 Use --no-cache-dir to avoid caching problems

If the previous installation fails, you can disable cache and re-download:

pip install numpy --no-cache-dir -i /simple

2.4 Upgrade pip

Old versionpipThe download may be slow, so it is recommended to upgrade first:

python -m pip install --upgrade pip

2.5 Offline installation (suitable for network-free environments)

• Download on other machinesnumpyof.whldocument:
  • Official download:PyPI numpy
  • Tsinghua mirror:numpy whl
• Manual installation:

pip install numpy-1.24.4-cp39-cp39-win_amd64.whl

2.6 Use --trusted-host to solve SSL problems

Some mirror sources may not have HTTPS certificates, you can add--trusted-host：

pip install numpy -i /pypi/simple/ --trusted-host 

2.7 Using Docker (Advanced Solution)

If the environment is complex, you can use preinstallation directlynumpyDocker image:

docker pull python:3.9-slim
docker run -it python:3.9-slim bash
pip install numpy -i /simple

3. Java calls Python-generated CSV data

Suppose we generate battery capacity data using Python (as in the introductionfull_capacity_data.csv), how to read and process this data in Java?

3.1 Java reads CSV files

useopencsvLibrary (need to be introduced by Maven):

<dependency>
    <groupId></groupId>
    <artifactId>opencsv</artifactId>
    <version>5.7.1</version>
</dependency>

Java code examples

import ;
import ;
import ;

public class CSVDataReader {
    public static void main(String[] args) {
        String csvFile = "full_capacity_data.csv";
        
        try (CSVReader reader = new CSVReader(new FileReader(csvFile))) {
            List<String[]> data = ();
            
            // Skip the header            for (int i = 1; i < (); i++) {
                String[] row = (i);
                int cycle = (row[0]);
                double capacity = (row[1]);
                ("Cycle: %d, Capacity: %.3f Ah%n", cycle, capacity);
            }
        } catch (Exception e) {
            ();
        }
    }
}

3.2 Hybrid programming using Python-Java

If you need to call Python scripts directly in Java, you can useProcessBuilder：

import ;
import ;

public class PythonExecutor {
    public static void main(String[] args) {
        try {
            ProcessBuilder pb = new ProcessBuilder("python", "generate_capacity_data.py");
            Process process = ();
            
            BufferedReader reader = new BufferedReader(
                new InputStreamReader(())
            );
            
            String line;
            while ((line = ()) != null) {
                (line);
            }
            
            int exitCode = ();
            ("Python script execution is completed, exit code: " + exitCode);
        } catch (Exception e) {
            ();
        }
    }
}

4. Summary

Best Practice Recommendations

• Priority is given to the use of domestic mirrors (such as Tsinghua and Alibaba Cloud).
• Recommended by Anacondaconda install numpy。
• When Java calls Python data, CSV is the most common format, and JSON or database storage can also be considered.

5. Expand thinking

• Is it possible to usepipAccelerate other libraries?
  Yes, all PyPI libraries are mirror-accelerated.
• How to optimize the performance of Java calling Python?
  Can considerJythonorGraalVMImplement more efficient Python-Java interaction.

Conclusion

Through the methods introduced in this article, you can greatly improvenumpyInstallation speed and enable efficient data interaction between Python and Java.

The above is the detailed content of several methods for Python to solve the problem of pip install numpy too slow. For more information about Python pip install numpy too slow, please follow my other related articles!