Preface
What is the problem of TCP sticking packet and why TCP sticking packets are generated will not be discussed in this article. This article uses golang'sTo implement custom protocol unpacking.
I won’t say much below, let’s take a look at the detailed introduction together.
Protocol packet definition
This article simulates a log server, which receives data packets sent by the client and displays them.
type Package struct {
Version [2]byte // Protocol version, tentative V1 Length int16 // Data part length Timestamp int64 // Time stamp HostnameLength int16 // Host name length Hostname []byte // Host name TagLength int16 // Label length Tag []byte // Label Msg []byte // Log data}
There is nothing to talk about in the protocol definition part, just define it according to the specific business logic.
Data Packaging
Since the TCP protocol is a language-independent protocol, it is impossible to directly send the protocol packet structure to the TCP connection. It can only send byte stream data, so you need to implement data encoding by yourself. Fortunately, golang provides binary to help us implement network byte encoding.
func (p *Package) Pack(writer ) error {
var err error
err = (writer, , &)
err = (writer, , &)
err = (writer, , &)
err = (writer, , &)
err = (writer, , &)
err = (writer, , &)
err = (writer, , &)
err = (writer, , &)
return err
}
The output goal of the Pack method is to facilitate interface expansion, and as long as the interface is implemented, data can be encoded and written. It is byte order, which will not be discussed in this article for the time being. Readers in need can find information and study it themselves.
Data depacket
Unpacking requires parsing the TCP packet into the structure. Next, we will talk about why we need to add several data-independent length fields.
func (p *Package) Unpack(reader ) error {
var err error
err = (reader, , &)
err = (reader, , &)
err = (reader, , &)
err = (reader, , &)
= make([]byte, )
err = (reader, , &)
err = (reader, , &)
= make([]byte, )
err = (reader, , &)
= make([]byte, )
err = (reader, , &)
return err
}
Since data such as host name and tag are not fixed in length, two bytes are needed to identify the data length. Otherwise, only one total data length can be known when reading, and it is impossible to distinguish between host name, tag name, and log data.
Solve the problem of sticking packets in data packets
The above only solves the encoding/decoding problem, provided that the received data packet does not produce the sticking problem. Solving the sticking packet is to correctly divide the data in the byte stream. Generally, the following practices are:
- Fixed-length separation (the maximum length for each packet) The disadvantage is that if there is insufficient data, it will waste transmission resources.
- Specific character separation (such as rn) The disadvantage is that if there is rn in the body, it will cause problems
- Adding a length field to the data packet (used in this article)
Golang providesTo solve the sticking problem.
scanner := (reader) // Reader implements the interface object, such as(func(data []byte, atEOF bool) (advance int, token []byte, err error) {
if !atEOF && data[0] == 'V' { // Since the packet header we define is the first two byte version number, only packets starting with V are processed if len(data) > 4 { // If the received data is > 4 bytes (2 byte version number + 2 byte packet length) length := int16(0)
((data[2:4]), , &length) // Read bytes 3-4 of the data packet (int16)=> data part length if int(length)+4 <= len(data) { // If the read data body length + 2 byte version number + 2 byte data length does not exceed the read data (in fact, a package was successfully parsed completely) return int(length) + 4, data[:int(length)+4], nil
}
}
}
return
})
// Print the received data packetfor () {
scannedPack := new(Package)
((()))
(scannedPack)
}
The core of this article isMethod, which is used to parse TCP packets
Complete source code
package main
import (
"bufio"
"bytes"
"encoding/binary"
"fmt"
"io"
"log"
"os"
"time"
)
type Package struct {
Version [2]byte // Protocol version Length int16 // Data part length Timestamp int64 // Time stamp HostnameLength int16 // Host name length Hostname []byte // Host name TagLength int16 // Tag length Tag []byte // Tag
Msg []byte // Data part length}
func (p *Package) Pack(writer ) error {
var err error
err = (writer, , &)
err = (writer, , &)
err = (writer, , &)
err = (writer, , &)
err = (writer, , &)
err = (writer, , &)
err = (writer, , &)
err = (writer, , &)
return err
}
func (p *Package) Unpack(reader ) error {
var err error
err = (reader, , &)
err = (reader, , &)
err = (reader, , &)
err = (reader, , &)
= make([]byte, )
err = (reader, , &)
err = (reader, , &)
= make([]byte, )
err = (reader, , &)
= make([]byte, )
err = (reader, , &)
return err
}
func (p *Package) String() string {
return ("version:%s length:%d timestamp:%d hostname:%s tag:%s msg:%s",
,
,
,
,
,
,
)
}
func main() {
hostname, err := ()
if err != nil {
(err)
}
pack := &Package{
Version: [2]byte{'V', '1'},
Timestamp: ().Unix(),
HostnameLength: int16(len(hostname)),
Hostname: []byte(hostname),
TagLength: 4,
Tag: []byte("demo"),
Msg: []byte(("The present time is:" + ().Format("2006-01-02 15:04:05"))),
}
= 8 + 2 + + 2 + + int16(len())
buf := new()
// Write four times to simulate the TCP sticking effect (buf)
(buf)
(buf)
(buf)
// scanner
scanner := (buf)
(func(data []byte, atEOF bool) (advance int, token []byte, err error) {
if !atEOF && data[0] == 'V' {
if len(data) > 4 {
length := int16(0)
((data[2:4]), , &length)
if int(length)+4 <= len(data) {
return int(length) + 4, data[:int(length)+4], nil
}
}
}
return
})
for () {
scannedPack := new(Package)
((()))
(scannedPack)
}
if err := (); err != nil {
("Invalid packet")
}
}
Written at the end
As a powerful network programming language, it is very important to implement custom protocols. In fact, it is not difficult to implement custom protocols. The following steps:
- Packet encoding
- Packet decoding
- Handle TCP sticking problem
- Disconnection and reconnection (can be achieved using heartbeat) (not necessary)
Summarize
The above is the entire content of this article. I hope that the content of this article has certain reference value for everyone's study or work. If you have any questions, you can leave a message to communicate. Thank you for your support.