Introduction to thrift
Thrift should be the cross-language rpc service framework that supports the most types of programming languages./
Thrift implements a complete network service, so when using thrift, the service framework of thrift will be used. Of course, you can also use the network services you have implemented and use the io stream to connect the input and output streams of the thrift interface to achieve thrift access.
Whether it is implemented using thrift network or network services implemented by yourself, as long as you connect to thrift, when calling the thrift interface to implement rpc, it is a network transmission method that uses thrift network.
Thrift's network transmission implementation method is not suitable for and does not support pressure-based network transmission needs. In fact, calling the thrift interface once does not mean that the network io writes data is only adjusted once, but is split into multiple writes and data transmissions.
When calling a thrift interface to send data, thrift will split this operation into several operations:
When calling the thrift method: thrift will find the object where this method is located and callwritemethod,
In the write method, call each parameter in turn.writeFieldBegin，writeXXX(Specific parameter type),WriteFieldStopEnvironmental functions
Each call also calls network io to write corresponding data at the same time.
Take the latest thrift-0.18.1 implementation as an example
Implementation of go
func (p *ItnetPonMergeArgs) Write(ctx , oprot ) error {
if err := (ctx, "PonMerge_args"); err != nil {
return (("%T write struct begin error: ", p), err) }
if p != nil {
if err := p.writeField1(ctx, oprot); err != nil { return err }
if err := p.writeField2(ctx, oprot); err != nil { return err }
}
if err := (ctx); err != nil {
return ("write field stop error: ", err) }
if err := (ctx); err != nil {
return ("write struct stop error: ", err) }
return nil
}
//The first parameterfunc (p *ItnetPonMergeArgs) writeField1(ctx , oprot ) (err error) {
if err := (ctx, "pblist", , 1); err != nil { //WriteFieldBegin
return (("%T write field begin error 1:pblist: ", p), err) }
if err := (ctx, , len()); err != nil {
return ("error writing list begin: ", err)
}
for _, v := range {
if err := (ctx, oprot); err != nil {
return (("%T error writing struct: ", v), err)
}
}
if err := (ctx); err != nil {
return ("error writing list end: ", err)
}
if err := (ctx); err != nil {
return (("%T write field end error 1:pblist: ", p), err) }
return err
}
//The second parameter, operation is similar to the first parameterfunc (p *ItnetPonMergeArgs) writeField2(ctx , oprot ) (err error) {
if err := (ctx, "id", thrift.I64, 2); err != nil {
return (("%T write field begin error 2:id: ", p), err) }
if err := oprot.WriteI64(ctx, int64()); err != nil {
return (("% (2) field write error: ", p), err) }
if err := (ctx); err != nil {
return (("%T write field end error 2:id: ", p), err) }
return err
}Thrift transmission order that calls Pon(ItnetPonMergeArgs) method
Write->
- writeField1->WriteFieldBegin-> WriteByte-> io
- -> Write16 -> io
- ->WriteBinary-> Write32 -> io
- ->Write -> io
- writeField2->WriteFieldBegin-> WriteByte -> io
- ->Write16 -> io
- ->Write64-> Write -> io
- WriteFieldStop ->io
As you can see, if there are two parameters in the method, there will be at least 8 io stream writing data calls in a simple method call.
If there are many parameters, or if there are many variables in a structure in the parameter, there will be more io stream writing data calls.
In massive network transmission, such transmission method, network io stream write data calls increase exponentially, and massive network io data writing leads to a sharp decline in performance.
The transport layer designed by thrift provides zlib protocol compression. In the case of zlib compression and transmission, the data is compressed and sent in an overall manner. After zlib is sent in two times, the receiver decompresses it;
Take go as an example:
You can see the zlib write data in compress/flate to the final io write call:
func (d *compressor) syncFlush() error {
if != nil {
return
}
= true
(d)
if == nil {
(0, false) //The first call () //The second call =
}
= false
return
}
//twiceioData writingTherefore, when a large number of calls to the thrift method, the performance of the zlib mode is far beyond that of the non-zlib. However, zlib compression will consume more memory, and when used in large quantities, it may lead to frequent GC and may also lead to performance degradation. Of course, even so, in most cases zlib transmission still performs much better than non-zlib transmission.
Implementation of other languages
For example, java:
public void write( oprot, SelectByIdxLimit_args struct) throws {
();
(STRUCT_DESC);
if ( != null) {
(NAME_FIELD_DESC); //io call (); //io call ();
}
if ( != null) {
(COLUMN_FIELD_DESC); //io call (); //io call ();
}
if ( != null) {
(VALUE_FIELD_DESC); //io call {
(new (, ()));
for ( _iter49 : )
{
(_iter49); //io call }
();
}
();
}
(START_ID_FIELD_DESC); //io call oprot.writeI64(); //io call ();
(LIMIT_FIELD_DESC); //io call oprot.writeI64(); //io call ();
(); //io call ();
}The transmission methods are all similar
The implementation methods are similar in each language, and of course, the data writing order is definitely the same.
The above is the detailed analysis of the problem examples that need to be paid attention to when implementing the network and transmission performance of Go thrift. For more information about the network transmission of Go thrift, please pay attention to my other related articles!