bufio package introduction
The bufio package implements buffered I/O. It wraps one or interface object, creates another object that also implements the interface, and also provides buffering and some text I/O helper functions.
golang bufio
IO is consumed when reading and writing a small amount of data frequently, causing performance problems. golang'sbufioThe library uses cache to read and write large chunks of data at one time, thereby reducing IO system calls and improving performance.
In Transport, you can set a nameWriteBufferSizeThe parameter of the parameter specifies the underlying layer () Write the size of the buffer.
tr := &{
WriteBufferSize: 64 * 1024,
} = (pconn, ())
= (persistConnWriter{pconn}, ())
Write using bufio
AvailableInitialize a size of 4096 bytesWriter(see below), or useInitialize a specified sizeWriter。
WriterThe main parameters in the cache areabuf, data offset in the cache areanand write interfacewr：
type Writer struct {
err error
buf []byte
n int
wr
}
Methods can write data in cache at one time, usually in the following three situations:
- Full data in cache
- There is still room in the cache
- The data to be written is larger than the cache size
Full data in cache
When the cache is full of data, a write operation is performed.
There is still room in the cache
If there is still data in the cache, the write action will not be performed unless the callFlush()method.
The data to be written is larger than the cache size
Since the cache cannot cache enough data at this time, the cache will be skipped and the write operation will be directly performed.
type Writer int
func (*Writer) Write(p []byte) (n int, err error) {
("Writing: %s\n", p)
return len(p), nil
}
func main() {
w := new(Writer)
bw1 := (w, 4)
// Case 1: Writing to buffer until full
([]byte{'1'})
([]byte{'2'})
([]byte{'3'})
([]byte{'4'}) // write - buffer is full
// Case 2: Buffer has space
([]byte{'5'}) //At this time, the buffer cannot accommodate more data. Perform write operations and write to []byte{'1','2','3','4'} err = () // forcefully write remaining
if err != nil {
panic(err)
}
// Case 3: (too) large write for buffer
// Will skip buffer and write directly
([]byte("12345")) //Insufficient buffer, directly execute the write operation//result:Writing: 1234
Writing: 5
Writing: 12345Cache reuse
Applying for cache is depleting performance and can be usedResetThe method resets the cache, and its internally justWriterData offsetnSet 0.
wr := new(Writer) bw := (wr,2) (wr)
Get the free space in the cache
Available()The method can return the number of free space in the cache, i.e.len()-
Read with bufio
and for writing dataWriterSimilarly, there is also one reading dataReader, can be usedNewReaderInitialize a size of 4096 bytesReader, or useNewReaderSizeInitialize a specified sizeReader(Requires minimum 16 bytes).ReaderThere is also a variable that records the offsetr
type Reader struct {
buf []byte
rd // reader provided by the client
r, w int // buf read and write positions
err error
lastByte int // last byte read for UnreadByte; -1 means invalid
lastRuneSize int // size of last rune read for UnreadRune; -1 means invalid
}
Peek
This method returns the contents of the first n bytes in the buf, but unlike the Read operation, it does not consume the data in the cache, that is, it does not increase the data offset, so it is usually used to determine whether the read end (EOF). There are usually several situations:
- If the value of peak is smaller than the cache size, the corresponding content will be returned
- If the value of peak is greater than the cache size, an error is returned
- If the value of peak contains EOF and is smaller than the cache size, EOF is returned
Read
Read data top, involves copying data from cache top。
func (b *Reader) Read(p []byte) (n int, err error)
ReadSlice
This method reads data read from the cache until the first one is encountereddelim. If there is no cachedelim, then EOF is returned. If the length of the query exceeds the cache size, then returnmistake.
func (b *Reader) ReadSlice(delim byte) (line []byte, err error)
For exampledelimfor',', the following will return the content1234,。
r := ("1234,567")
rb := (r, 20)
((','))
// Result: [49 50 51 52 44] <nil>Notice:
ReadSliceWhat is returned is the content in the original cache. If a concurrent operation is performed on the cache, the returned content may be overwritten by other operations. Therefore, it is written in the official comments, it is recommended to useReadBytesorReadString. butReadBytesandReadStringMemory applications and copies are involved, and therefore performance is affected. In the pursuit of high performance, external use is recommendedto provide cache.// Because the data returned from ReadSlice will be overwritten // by the next I/O operation, most clients should use // ReadBytes or ReadString instead.
ReadLine
ReadLine() (line []byte, isPrefix bool, err error)
ReadLineUsed in the bottom layerReadSlice, but will be removed when returned\nor\r\n. It should be noted that if no newline is found in the slice, EOF will not be returned orError, instead, it willisPrefixSet astrue
ReadBytes
andReadSliceSimilar, but it returns a new slice, so it is easy to use concurrently. If not founddelim，ReadBytesWill return
func (b *Reader) ReadBytes(delim byte) ([]byte, error)
Scanner
The scanner can continuously read data to the cache (default 64*1024 bytes).
rb := ("12345678901234567890")
scanner := (rb)
for () {
("Token (Scanner): %q\n", ())
}
// result：Token (Scanner): "12345678901234567890"refer to
how-to-read-and-write-with-golang-bufio
This is the end of this article about golang bufio analysis. For more related golang bufio content, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!