1. Use built-in functions and libraries
Python comes with a lot of efficient built-in functions and libraries, which are usually much faster than the code you write yourself. Try to use these ready-made tools instead of reinventing the wheel yourself.
Example:
# Not recommended: Manually calculate the sum of list elementsdef sum_list_manual(lst):
total = 0
for num in lst:
total += num
return total
# Recommended: Use the built-in sum functiondef sum_list_builtin(lst):
return sum(lst)
# testnumbers = [1, 2, 3, 4, 5]
print(sum_list_manual(numbers)) # Output: 15print(sum_list_builtin(numbers)) # Output: 152. Use list comprehension instead of loop
List comprehensions are not only more concise, but also faster execution.
Example:
# Not recommended: Use for loop to create a squared listsquares_loop = []
for i in range(10):
squares_loop.append(i ** 2)
# Recommended: Use list comprehensionsquares_comprehension = [i ** 2 for i in range(10)]
print(squares_loop) # Output: [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]print(squares_comprehension) # Output: [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]3. Save memory using generator expressions
When you work with large data sets, generator expressions can save a lot of memory because it only generates data when needed.
Example:
# Not recommended: Use list to store all square numberssquares_list = [i ** 2 for i in range(1000000)] # Recommended: Use generator expressionssquares_generator = (i ** 2 for i in range(1000000)) # Calculate the sumprint(sum(squares_list)) # Requires a lot of memoryprint(sum(squares_generator)) # Less memory usage
4. Use set for member checking
gather(set) The time complexity of the data structure of searching elements is O(1), and the list (list) is O(n). Therefore, if frequent member checks are required, usesetIt will be faster.
Example:
# Not recommended: Use list for member checkingmy_list = list(range(1000000))
if 999999 in my_list:
print("Found!")
# Recommended: Use collections for member checkingmy_set = set(range(1000000))
if 999999 in my_set:
print("Found!")
5. Use dictionary instead of multiple conditional judgments
When there are multiple conditional judgments, you can consider using dictionaries to simplify the code and improve efficiency.
Example:
# Not recommended: Use multiple if-elsedef get_grade(score):
if score >= 90:
return 'A'
elif score >= 80:
return 'B'
elif score >= 70:
return 'C'
else:
return 'D'
# Recommended: Use dictionary mappinggrade_mapping = {
(90, 100): 'A',
(80, 89): 'B',
(70, 79): 'C',
(0, 69): 'D'
}
def get_grade_dict(score):
for key, value in grade_mapping.items():
if key[0] <= score <= key[1]:
return value
print(get_grade(85)) # Output: Bprint(get_grade_dict(85)) # Output: B6. Use local variables to improve performance
Local variables are accessed faster than global variables. Therefore, try to use local variables in the loop.
Example:
# Not recommended: Use global variablesglobal_var = 10
def increment_global():
global global_var
for _ in range(1000000):
global_var += 1
# Recommended: Use local variablesdef increment_local():
local_var = 10
for _ in range(1000000):
local_var += 1
increment_global() # Slowerincrement_local() # Faster7. Use functools.lru_cache to cache results
For time-consuming function calls, you can usefunctools.lru_cacheto cache the results and avoid repeated calculations.
Example:
import functools
# Not recommended: Recalculate every calldef fibonacci(n):
if n <= 1:
return n
return fibonacci(n-1) + fibonacci(n-2)
# Recommended: Use [email protected]_cache(maxsize=None)
def fibonacci_cached(n):
if n <= 1:
return n
return fibonacci_cached(n-1) + fibonacci_cached(n-2)
print(fibonacci(30)) # Slowerprint(fibonacci_cached(30)) # Faster8. Use numpy to process numerical calculations
numpyis a library dedicated to scientific computing, which provides efficient array operation capabilities.
Example:
import numpy as np
# Not recommended: Matrix multiplication using pure Python listsmatrix_a = [[1, 2], [3, 4]]
matrix_b = [[5, 6], [7, 8]]
result = [[0, 0], [0, 0]]
for i in range(len(matrix_a)):
for j in range(len(matrix_b[0])):
for k in range(len(matrix_b)):
result[i][j] += matrix_a[i][k] * matrix_b[k][j]
# Recommended: Use numpy for matrix multiplicationmatrix_a_np = (matrix_a)
matrix_b_np = (matrix_b)
result_np = (matrix_a_np, matrix_b_np)
print(result) # Output: [[19, 22], [43, 50]]print(result_np) # Output: [[19 22] [43 50]]9. Use pandas to process data
pandasIt is a powerful data analysis library, especially suitable for processing tabular data.
Example:
import pandas as pd
# Not recommended: Use pure Python dictionary and list processing datadata = {'Name': ['Alice', 'Bob', 'Charlie'], 'Age': [25, 30, 35]}
ages = [age for age in data['Age'] if age > 30]
# Recommended: Use pandas to process datadf = (data)
filtered_df = df[df['Age'] > 30]
print(ages) # Output: [35]print(filtered_df) # Output: Name Age # 2 Charlie 35
10. Use multiprocessing for parallel processing
If your tasks are parallelized, usemultiprocessingModules can significantly improve performance.
Example:
import multiprocessing
def square(x):
return x ** 2
# Not recommended: Serial processingresults_serial = [square(i) for i in range(10)]
# Recommended: Parallel processingpool = ()
results_parallel = (square, range(10))
()
()
print(results_serial) # Output: [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]print(results_parallel) # Output: [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]11. Asynchronous programming using asyncio
For I/O intensive tasks, useasyncioIt can improve the response speed of the program.
Example:
import asyncio
async def fetch_data():
await (1)
return "Data fetched"
# Not recommended: Synchronous waitingdef sync_fetch():
import time
(1)
return "Data fetched"
# Recommended: Asynchronous Waitingasync def main():
result = await fetch_data()
print(result)
(main()) # Output: Data fetched12. Compile Python code with cython
cython can compile Python code into C code, thereby improving execution speed.
Example:
# Not recommended: Pure Python codedef slow_function(n):
return sum(i * i for i in range(n))
# Recommended: Use Cython to compile# Define in file slow_function.pyxdef fast_function(int n):
cdef int i, result = 0
for i in range(n):
result += i * i
return result
# Compile and importfrom import setup
from import cythonize
setup(ext_modules=cythonize("slow_function.pyx"))
from slow_function import fast_function
print(slow_function(1000000)) # Slowerprint(fast_function(1000000)) # Faster13. Accelerate numerical calculation using numba
numbaNumerical calculations can be accelerated through instant compilation technology.
Example:
import numba
@
def fast_sum(a, b):
return a + b
# Not recommended: Pure Python additiondef slow_sum(a, b):
return a + b
print(slow_sum(10, 20)) # Output: 30print(fast_sum(10, 20)) # Output: 3014. Use cProfile for performance analysis
Before optimizing your code, use itcProfileFind out where the bottleneck lies.
Example:
import cProfile
def profile_me():
total = 0
for i in range(1000000):
total += i
return total
('profile_me()')
15. Use line_profiler for line-by-line analysis
line_profilerIt can help you find which line of code is the most time-consuming.
Example:
# Install line_profiler# pip install line_profiler
@profile
def function_to_profile():
a = 2
b = 3
c = a + b
return c
function_to_profile()
Practical case: Optimizing image processing algorithm
Suppose you need to write an image processing program that converts an image to a grayscale image. Let's take a look at how to apply the above techniques to optimize this process.
Original code:
from PIL import Image
def to_grayscale(image_path):
image = (image_path)
width, height =
grayscale_image = ('L', (width, height))
for x in range(width):
for y in range(height):
r, g, b = ((x, y))
gray = int(0.299 * r + 0.587 * g + 0.114 * b)
grayscale_image.putpixel((x, y), gray)
grayscale_image.save('')
to_grayscale('')
Optimized code:
import numpy as np
from PIL import Image
def to_grayscale_optimized(image_path):
image = (image_path).convert('RGB')
image_array = (image)
grayscale_array = (image_array[..., :3], [0.299, 0.587, 0.114]).astype(np.uint8)
grayscale_image = (grayscale_array, 'L')
grayscale_image.save('')
to_grayscale_optimized('')
In this practical case, we usednumpyto perform efficient array operations, avoid explicit double loops, thereby greatly improving the running speed of the program.
Summarize
This article introduces 15 techniques to make Python programs run quickly, including using built-in functions and libraries, list comprehensions, generator expressions, collection member checks, dictionary mapping, local variables, cache results,numpyandpandaslibrary, parallel and asynchronous processing,cythonandnumbaCompilation, performance analysis tools, etc. With these tips, you can write more efficient and elegant Python code.
This is the end of this article about 15 Python running speed optimization techniques. For more related Python optimization techniques, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!