Preface
Files are everywhere, and no matter which programming language we use, processing files is essential for every programmer
File processing is a process used to create files, write data and read data from them. Python has a wealth of packages for processing different file types, making it easier and easier to complete file processing.
Outline of this article:
- Open a file using a context manager
- File reading mode in Python
- Read text file
- Read CSV files
- Read JSON files
Open the file
Before accessing the contents of the file, we need to open the file. Python provides a built-in function that helps us open files in different modes.open()Functions accept two basic parameters: file name and pattern
The default mode is "r", which opens the file in a read-only manner. These patterns define how we access files and how we manipulate their contents.open()Functions provide several different modes, which we will discuss one by one later
Let’s study the following discussion through the ’Python Zen’ file
f = open('zen_of_python.txt', 'r')
print(())
()
Output:
The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
...
In the above code,open()The function opens a text file in read-only mode, which allows us to get information from the file without changing it. In the first line,open()The output of the function is assigned to an object representing the text file.f, In the second line, we useread()Method to read the entire file and print its contents.close()Method closes the file on the last line. It should be noted that we must always close open files after processing them to free our computer resources and avoid throwing exceptions
In Python, we can usewithContext Manager to ensure that the program releases the resource used after the file is closed, even if an exception occurs
with open('zen_of_python.txt') as f:
print(())
Output:
The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
...
The above code is usedwithThe statement creates a context and binds to the variablef, All file object methods can access file objects through this variable.read()Method reads the entire file on the second line and then usesprint()Function output file content
When the program reaches the end of the with statement block context, it closes the file to free up resources and ensures that other programs can call them normally. Usually when we deal with objects that are no longer needed and need to be closed immediately (such as files, databases, and network connections), it is highly recommended to use the with statement.
It should be noted here that we can access even after exiting the with context manager blockfvariable, but the file is closed. Let's try some file object properties to see if the variable still exists and is accessible:
print("Filename is '{}'.".format())
if :
print("File is closed.")
else:
print("File isn't closed.")
Output:
Filename is 'zen_of_python.txt'.
File is closed.
However, it is impossible to read or write the file from the file at this time. When closing the file, any attempt to access its content will lead to the following error:
()
Output:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_9828/ in <module>
----> 1 ()
ValueError: I/O operation on closed file.
File reading mode in Python
As we mentioned earlier, we need to specify the pattern when opening the file. The following table shows the different file patterns in Python:
Mode Description
- 'r' Open a read-only file
- 'w' Open a file for writing. If the file exists, it will be overwritten, otherwise a new file will be created
- 'a' Open a file for append only. If the file does not exist, the file will be created
- 'x' Create a new file. If the file exists, it fails
- '+' Open a file for update
We can also specify that the file is opened in text mode "t", default mode, or binary mode "b". Let's see how to copy image files using simple statements dataquest_logo.png:
with open('dataquest_logo.png', 'rb') as rf:
with open('data_quest_logo_copy.png', 'wb') as wf:
for b in rf:
(b)
The above code copies the Dataquest logo image and stores it in the same path. 'rb' mode opens the file in binary mode and reads, while 'wb' mode opens the file in text mode and writes in parallel
Read text files
There are many ways to read text files in Python. Below we introduce some useful methods to read text files contents
So far, we have learned that it is availableread()Method to read all contents of the file. What if we just want to read a few bytes from a text file, we canread()Specify the number of bytes in the method. Let's try it out:
with open('zen_of_python.txt') as f:
print((17))
Output:
The Zen of Python
The simple code above reads the first 17 bytes of the zen_of_python.txt file and prints them out
Sometimes it makes more sense to read the contents of a text file at a time, in which case we can use the readline() method
with open('zen_of_python.txt') as f:
print(())
Output:
The Zen of Python, by Tim Peters
The above code returns the first line of the file, and if we call the method again, it will return the second line in the file, etc., as follows:
with open('zen_of_python.txt') as f:
print(())
print(())
print(())
print(())
Output:
The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
This useful method can help us read the entire file incrementally.
The following code outputs the entire file by iterating line by line until the file pointer tracking the location where we are reading or writing to the file reaches the end of the file. whenreadline()When the method reaches the end of the file, it returns an empty string
with open('zen_of_python.txt') as f:
line = ()
while line:
print(line, end='')
line = ()
Output:
The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
The above code iswhileRead the first line of the file outside the loop and assign it tolineVariable. existwhileIn the loop, it prints and stores it inlineString in the variable and then read the next line of the file.whileIterate the process loop untilreadline()The method returns an empty string. Empty string inwhileThe calculation result in the loop isFalse, so the iteration process ends
Another useful way to read a text file isreadlines()Method, applying this method to a file object will return a list of strings containing each line of the file
with open('zen_of_python.txt') as f:
lines = ()
Let's check the data type of the lines variable and print it:
print(type(lines)) print(lines)
Output:
<class 'list'>
['The Zen of Python, by Tim Peters\n', '\n', 'Beaut...]
It is a list of strings where each item in the list is a line of the text file, and the ``\n` escape character indicates a new line in the file. Additionally, we can access each item in the list through an index or slice operation:
print(lines) print(lines[3:5]) print(lines[-1])
Output:
['The Zen of Python, by Tim Peters\n', '\n', 'Beautiful is better than ugly.\n', ... -- let's do more of those!"]
['Explicit is better than implicit.\n', 'Simple is better than complex.\n']
Namespaces are one honking great idea -- let's do more of those!
Read CSV files
So far, we have learned how to use regular text files. But sometimes the data is in CSV format, and data professionals usually retrieve the required information and operate the contents of the CSV file
Next we will use the CSV module, which provides useful ways to read comma-separated values stored in CSV files. Let's try it now
import csv
with open('') as f:
reader = (f, delimiter=',')
for row in reader:
print(row)
Output:
['Company', 'Bean Origin or Bar Name', 'REF', 'Review Date', 'Cocoa Percent', 'Company Location', 'Rating', 'Bean Type', 'Country of Origin']
['A. Morin', 'Agua Grande', '1876', '2016', '63%', 'France', '3.75', 'Â\xa0', 'Sao Tome']
['A. Morin', 'Kpime', '1676', '2015', '70%', 'France', '2.75', 'Â\xa0', 'Togo']
['A. Morin', 'Atsane', '1676', '2015', '70%', 'France', '3', 'Â\xa0', 'Togo']
['A. Morin', 'Akata', '1680', '2015', '70%', 'France', '3.5', 'Â\xa0', 'Togo']
...
Each line of the CSV file forms a list where each item can be easily accessed, as shown below:
import csv
with open('') as f:
reader = (f, delimiter=',')
for row in reader:
print("The {} company is located in {}.".format(row[0], row[5]))
Output:
The Company company is located in Company Location.
The A. Morin company is located in France.
The A. Morin company is located in France.
The A. Morin company is located in France.
The A. Morin company is located in France.
The Acalli company is located in ..
The Acalli company is located in ..
The Adi company is located in Fiji.
...
Many times, using column names instead of their indexes is often more convenient for professionals. In this case, we do not usereader()Method, instead, it uses the return dictionary object collectionDictReader()method
import csv
with open('') as f:
dict_reader = (f, delimiter=',')
for row in dict_reader:
print("The {} company is located in {}.".format(row['Company'], row['Company Location']))
Output:
The A. Morin company is located in France.
The A. Morin company is located in France.
The A. Morin company is located in France.
The A. Morin company is located in France.
The Acalli company is located in ..
The Acalli company is located in ..
The Adi company is located in Fiji.
...
Read JSON files
Another popular file format we mainly use to store and exchange data is JSON, which stands for JavaScript Object Notation, allowing us to store data using comma-separated key-value pairs.
Next we will load a JSON file and use it as a JSON object instead of as a text file, for which we need to import the JSON module. Then inwithIn the context manager, we use the json objectload()Method, which loads the contents of the file and stores it as a dictionary in a context variable.
import json
with open('') as f:
content = (f)
print(content)
Output:
{'Title': 'Bicentennial Man', 'Release Date': 'Dec 17 1999', 'MPAA Rating': 'PG', 'Running Time min': 132, 'Distributor': 'Walt Disney Pictures', 'Source': 'Based on Book/Short Story', 'Major Genre': 'Drama', 'Creative Type': 'Science Fiction', 'Director': 'Chris Columbus', 'Rotten Tomatoes Rating': 38, 'IMDB Rating': 6.4, 'IMDB Votes': 28827}
Let's check the data type of the content variable:
print(type(content))
Output:
<class 'dict'>
Its data type is a dictionary, so we can easily extract data from it
print('{} directed by {}'.format(content['Title'], content['Director']))
Output:
Bicentennial Man directed by Chris Columbus
Summarize
Today we discuss file processing in Python, focusing on reading the contents of files. We've got itopen()Built-in functions,withContext manager, and how to read common file types such as text, CSV, and JSON.
The above is a detailed content of the method of understanding Python's reading text, CSV, and JSON files. For more information about Python's reading files, please pay attention to my other related articles!