Python methods to read and write txt, Excel files and JSON files

Python provides a variety of ways to read and write files of different types, including text files (txt), Excel files, and JSON files. Here are some commonly used methods and example codes:

Read/write txt file

Basic reading txt

Read txt file

Use built-inopenFunction

# Read the entire file contentwith open('', 'r', encoding='utf-8') as file:
    content = ()
    print(content)

# Read file content line by linewith open('', 'r', encoding='utf-8') as file:
    for line in file:
        print(())

Write to txt file

Use built-inopenFunction

# Write text to file (overwrite mode)with open('', 'w', encoding='utf-8') as file:
    ('Hello, World!\n')

# Append text to filewith open('', 'a', encoding='utf-8') as file:
    ('Appending a new line.\n')

Read complex data by row

Read and process each row of data row by row

# Suppose our file content is as follows:# Name, Age, City
# Alice, 30, New York
# Bob, 25, Los Angeles

with open('', 'r', encoding='utf-8') as file:
    header = ().strip().split(', ')
    data = []
    for line in file:
        values = ().split(', ')
        record = dict(zip(header, values))
        (record)
    print(data)

Process large txt text files (read line by line to save memory)

# Read large files line by linewith open('large_file.txt', 'r', encoding='utf-8') as file:
    for line in file:
        process_line(line)  # Customized processing functions

Read/write Excel files

Basic Reading

Read Excel files

usepandasLibrary

import pandas as pd

# Read Excel filesdf = pd.read_excel('', sheet_name='Sheet1')
print(df)

useopenpyxlLibrary (for .xlsx files)

from openpyxl import load_workbook

# Read Excel fileswb = load_workbook('')
sheet = wb['Sheet1']
for row in sheet.iter_rows(values_only=True):
    print(row)

Write to Excel file

usepandasLibrary

import pandas as pd

# Create a DataFramedata = {'Name': ['Alice', 'Bob'], 'Age': [25, 30]}
df = (data)

# Write to Excel filedf.to_excel('', sheet_name='Sheet1', index=False)

useopenpyxlLibrary

from openpyxl import Workbook

# Create a new Excel filewb = Workbook()
sheet = 
 = 'Sheet1'

# Write data(['Name', 'Age'])
(['Alice', 25])
(['Bob', 30])

# Save the file('')

Handle complex Excel files (multiple worksheets)

usepandasRead multiple worksheets

import pandas as pd

# Read all sheets of Excel filesxls = ('')
sheets = {}
for sheet_name in xls.sheet_names:
    sheets[sheet_name] = pd.read_excel(xls, sheet_name=sheet_name)
    print(f"Sheet: {sheet_name}")
    print(sheets[sheet_name])

useopenpyxlRead line by line

from openpyxl import load_workbook

# Read Excel fileswb = load_workbook('')
for sheet_name in :
    sheet = wb[sheet_name]
    print(f"Sheet: {sheet_name}")
    for row in sheet.iter_rows(values_only=True):
        print(row)

Process large Excel files (using pandas’ chunksize parameter)

import pandas as pd

# Read large Excel files block by blockchunk_size = 1000
for chunk in pd.read_excel('large_file.xlsx', sheet_name='Sheet1', chunksize=chunk_size):
    process_chunk(chunk)  # Customized processing functions

Read/write JSON files

Basic Reading

Basic reading of JSON files

Use built-injsonModule

import json

# Read JSON fileswith open('', 'r', encoding='utf-8') as file:
    data = (file)
    print(data)

Write to JSON file

Use built-injsonModule

import json

# datadata = {'name': 'Alice', 'age': 25}

# Write to JSON filewith open('', 'w', encoding='utf-8') as file:
    (data, file, ensure_ascii=False, indent=4)

Read nested data:

Read nested json files:

Read nested JSON data

import json

# Suppose our JSON file content is as follows:# {
#     "name": "Alice",
#     "age": 30,
#     "address": {
#         "city": "New York",
#         "zipcode": "10001"
#     },
#     "phones": ["123-456-7890", "987-654-3210"]
# }

with open('', 'r', encoding='utf-8') as file:
    data = (file)
    print(data)

# Access nested dataprint(data['address']['city'])  # Output: New Yorkprint(data['phones'][0])  # Output: 123-456-7890

Write nested JSON data

import json

#Nested datadata = {
    "name": "Alice",
    "age": 30,
    "address": {
        "city": "New York",
        "zipcode": "10001"
    },
    "phones": ["123-456-7890", "987-654-3210"]
}

# Write to JSON filewith open('', 'w', encoding='utf-8') as file:
    (data, file, ensure_ascii=False, indent=4)

Complex reading of json files

Reading JSON files by line is useful when working with large files or streaming data. Here are some ways to read JSON files by line:

Method 1: Read and parse each line JSON line by line

If each line of a JSON file is an independent JSON object, you can read and parse each line line by line:

import json

# Suppose our JSON file content is as follows, each line is an independent JSON object:# {"name": "Alice", "age": 30}
# {"name": "Bob", "age": 25}

with open('', 'r', encoding='utf-8') as file:
    for line in file:
        data = (())
        print(data)

Method 2: Read and parse nested JSON line by line

If the JSON file is an array containing multiple objects, you can useijsonLibrary parses nested JSON data line by line:

import ijson

# Suppose our JSON file content is as follows:# [
#     {"name": "Alice", "age": 30},
#     {"name": "Bob", "age": 25}
# ]

with open('', 'r', encoding='utf-8') as file:
    parser = (file)
    for prefix, event, value in parser:
        if ('.name') and event == 'string':
            print(f"Name: {value}")
        elif ('.age') and event == 'number':
            print(f"Age: {value}")

Method 3: Process large JSON files (blocked reading)

For very large JSON files, chunked reading and processing can be considered:

import json

def process_chunk(chunk):
    for line in chunk:
        data = (())
        print(data)

chunk_size = 1000  # Read 1000 rows each time
with open('large_file.json', 'r', encoding='utf-8') as file:
    chunk = []
    for line in file:
        (line)
        if len(chunk) &gt;= chunk_size:
            process_chunk(chunk)
            chunk = []
    if chunk:
        process_chunk(chunk)

Method 4: Use the jsonlines library

jsonlinesThe library is specifically used to process files that are one JSON object per line:

import jsonlines

# Suppose our JSON file content is as follows, each line is an independent JSON object:# {"name": "Alice", "age": 30}
# {"name": "Bob", "age": 25}

with ('') as reader:
    for obj in reader:
        print(obj)

These methods can help you read JSON files by line and select the appropriate method to process the data according to the specific structure and size of the file.
The above are some common methods to read and write txt, Excel, and JSON files. Each method has its advantages and disadvantages, and which method to choose depends on the specific requirements and usage scenarios.

The above is the detailed content of Python's methods of reading and writing txt, Excel files and JSON files. For more information about Python's reading and writing txt, Excel and JSON, please pay attention to my other related articles!