SoFunction
Updated on 2024-10-28

Python Advanced Application Comparison: Efficiently Calculating the Length of the Longest Line in a Large File

The first 2 methods mainly use list parsing, which has a slightly worse performance, while the last one uses generator expressions, which are more memory-efficient than list parsing

List parsing is very similar to generator expressions:

list resolution

[expr for iter_var in iterable if cond_expr]

generator expression (math.)

(expr for iter_var in iterable if cond_expr)

Method 1: Most primitive

Copy Code The code is as follows.

longest = 0
f = open(FILE_PATH,"r")
allLines = [() for line in ()]
()
for line in allLines:
    linelen = len(line)
    if linelen>longest:
        longest = linelen

Method 2: Simplicity

Copy Code The code is as follows.

f = open(FILE_PATH,"r")
allLineLens = [len(()) for line in f]
longest = max(allLineLens)
()

Cons: When iterating f line by line, list parsing requires reading all lines of the file into memory and generating the list

Method 3: Simplest and most memory efficient

Copy Code The code is as follows.

f = open(FILE_PATH,"r")
longest = max(len(line) for line in f)
()

or

Copy Code The code is as follows.

print max(len(()) for line in open(FILE_PATH))