Preface
Golang provides many ways to read and write IO, and I currently know about the io library, os library, ioutil library, bufio library, bytes/strings library, etc.
Although more libraries are a good thing, which means more selective, one thing that confuses me is: Which library should be used in what scenario? Why?
Before giving a conclusion, I want to give the project structure of Golang's built-in IO library, which is mainly convenient for understanding and citation:
# Only the core directories and files are listedsrc: - bufio - - bytes - - - io - ioutil - - - os - - strings -
The library belongs to the underlying interface definition library. Its function is to define some basic interfaces and some basic constants, and to explain the functions of these interfaces. Common interfaces include Reader, Writer, etc. Generally, this library is used just to call some of its constants, for example.
The library is included in the io directory. Its main function is to act as a toolkit, which contains some more practical functions, such as ReadAll (read data from a certain source), ReadFile (read file content), WriteFile (write data to a file), ReadDir (get directory)
The library mainly deals with the operating system, so file operations are basically linked to the Os library, such as creating a file, opening a file, etc. This library is often used in conjunction with ioutil library, bufio library, etc.
The library can be understood as encapsulating another layer on the io library and adding a cache function. It may be confused with the ioutil library and.
4.1 bufio VS ioutil library: Both provide read and write functions to files. The only difference is that bufio has an additional layer of cache function. This advantage mainly reflects when reading large files (it is to load the content into memory at one time. If the content is too large, it is easy to explode the memory)
4.2 bufio VS: Both provide a layer of caching function. Their differences are mainly because bufio targets file-to-memory cache, while bufio targets memory-to-memory cache (I personally feel like a channel, you can also find that there is no interface to write data to files).
and strings library: These two libraries are a bit funny. First of all, they both implement the Reader interface, so their differences mainly lie in the different objects they are targeted, bytes are targeted, and strings are targeted at strings (their method implementation principles are very similar). Another difference is that bytes also has the function of Buffer, but strings are not provided.
Note: Regarding the Reader and Writer interfaces, it can be simply understood as a read source and a write source. That is, as long as the Read method in Reader is implemented, this thing can be used as a read source, which can contain data and read by us; the same is true for Writer.
The above are some of my personal conclusions. The following will further explain the above conclusions. If there are any errors, please leave a message to correct me. ❤️!
Peeping io library
There are three common interfaces in the io library, namely Reader, Writer and Close.
// The Read method will receive a byte array p, store the read data into the array, and finally return the number of bytes n read.// Note that n does not necessarily equal the length of the data read. For example, the capacity of byte array p is too small, n will equal the length of the arraytype Reader interface {
Read(p []byte) (n int, err error)
}
// The Write method also receives a byte array p and saves the received data to a file or standard output, etc. The returned n represents the length of the written data.// When n is not equal to len(p), an error is returned.type Writer interface {
Write(p []byte) (n int, err error)
}
// Close the operationtype Closer interface {
Close() error
}
Regarding the specific implementation of the Read method, you can see in the strings library:
// Define a Reader interface bodytype Reader struct {
s string
i int64 // current reading index
prevRune int // index of previous rune; or < 0
}
// Get the reader object through the NewReader method. There is a key point here that the passed string is assigned to the s variablefunc NewReader(s string) *Reader {
return &Reader{s, 0, -1}
}
// Read method: The core is the copy method. Although the parameter b is a slice, the copy method will affect its underlying arrayfunc (r *Reader) Read(b []byte) (n int, err error) {
if >= int64(len()) {
return 0,
}
= -1
// Core method n = copy(b, [:])
+= int64(n)
return
}
Spy the ioutil library
As mentioned above, the ioutil library is a toolkit, which mainly contains more practical functions, such as ReadFile, WriteFile, etc. The only thing to note is that they are all read and written at one time, so when reading, please note that the file should not be too large.
Read data from a file:
func readByFile() {
data, err := ( "./lab8_io/file/")
if err != nil {
("err:", err)
return
}
("data", string(data)) // hello world！
}
Write data to a file:
func writeFile() {
err := ("./lab8_io/file/write_test.txt", []byte("hello world!"), 0644)
if err != nil {
panic(err)
return
}
}
Traversal Directory: One thing you need to note when traversing a directory is that its sorting is not a natural way of sorting.
Peek at the bufio library
The bufio library has also been mentioned above. It mainly adds a layer of cache function to the io library. The following is an example of bufio reading large files:
func readBigFile(filePath string) error {
f, err := (filePath)
defer ()
if err != nil {
(err)
return err
}
buf := (f)
count := 0
for {
count += 1
line, err := ('\n')
line = (line)
if err != nil {
return err
}
("line", line)
// Here is to avoid printing all if count > 100 {
break
}
}
return nil
}
Note:
ReadLine/ReadBytes/ReadString/ReadSlice: ReadString and ReadBytes are equivalent, ReadBytes and ReadLine both call ReadSlice
Peek into bytes/strings library
As mentioned earlier, just implementing the Reader interface, the implementation methods of bytes and strings underlying functions are similar. You can check the source code to prove it:
// bytes/
// Read implements the interface.
func (r *Reader) Read(b []byte) (n int, err error) {
if >= int64(len()) {
return 0,
}
= -1
n = copy(b, [:])
+= int64(n)
return
}
// strings/
func (r *Reader) Read(b []byte) (n int, err error) {
if >= int64(len()) {
return 0,
}
= -1
n = copy(b, [:])
+= int64(n)
return
}
Reference/Recommended
Detailed explanation of the implementation principle of bufio package in golang
Two solutions for reading super large files
/suntong/032173e96247c0411140
This is the end of this article about the confusion about Golang IO reading and writing. For more information about Golang IO reading and writing, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!