SoFunction
Updated on 2025-04-11

Detailed explanation of the types and usage scenarios of java serialization

Overview of serialization

What is serialization?

Serialization is the process of converting the state of an object into a byte stream so that the object can be stored in a file, a database, or transmitted over a network. Deserialization is the process of converting a byte stream back to an object.

This process allows data to be passed between different computer systems or persisted between different runtimes of the program.

The role of serialization

  1. Persistence: Save the state of the object to the storage medium for recovery if needed.
  2. Network transmission: In a distributed system, objects are transmitted from one application to another through the network.
  3. Deep copy: Deep copying of objects is achieved through serialization and deserialization.
  4. cache: Serialize the object and store it in the cache for quick retrieval.
  5. Distributed computing: In the microservice architecture, complex data structures need to be passed between services, and serialization can effectively achieve this.

Java built-in serialization

interface

  • definition：Serializableis a tag interface that indicates that an object of a class can be serialized.
  • accomplish: Any class that needs to be serialized must implement this interface. There is no way to implement, just declare it.

Using ObjectOutputStream and ObjectInputStream

Serialize objects：

try (ObjectOutputStream out = new ObjectOutputStream(new FileOutputStream(""))) {
    (yourObject);
} catch (IOException e) {
    ();
}

Deserialize objects：

try (ObjectInputStream in = new ObjectInputStream(new FileInputStream(""))) {
    YourClass yourObject = (YourClass) ();
} catch (IOException | ClassNotFoundException e) {
    ();
}

Pros and cons analysis

advantage：

  • Simple and easy to use: By implementingSerializableThe interface can be serialized.
  • Built-in support: The Java standard library comes with no additional dependencies.

shortcoming：

  • Poor performance: The serialized data is larger in size and slower in speed.
  • Inflexible: The serialization process cannot be easily controlled, such as field exclusion.
  • Incompatibility: Changes in class structure (such as adding or deleting fields) may cause deserialization to fail.
  • Security Question: It may lead to deserialization vulnerabilities and needs to be handled with caution.

Custom serialization

Implement the Externalizable interface

definition：ExternalizableThe interface has been extendedSerializableInterface that allows developers to fully control the serialization and deserialization process.

method：

  • writeExternal(ObjectOutput out): Customize the serialization process of objects.
  • readExternal(ObjectInput in): Deserialization process of custom objects.

Custom serialization method

Implementation example：

public class CustomObject implements Externalizable {
    private String name;
    private int age;

    public CustomObject() {
        // A parameter-free constructor must be provided    }

    @Override
    public void writeExternal(ObjectOutput out) throws IOException {
        (name);
        (age);
    }

    @Override
    public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
        name = (String) ();
        age = ();
    }
}

Applicable scenarios

  • Full control of the serialization process is required: When fine control of the serialized format is required.
  • Performance optimization: You can reduce the serialized data size or increase the speed through custom serialization logic.
  • Compatibility requirements: When the class structure changes, compatibility can be maintained through custom logic.
  • Security Requirements: Through the custom serialization process, security checks can be added or sensitive information can be filtered.

Third-party serialization framework

Kryo

Features and Advantages：

  • high performance: Kryo provides fast serialization and deserialization speeds.
  • Efficient space utilization: The generated serialized data is small.
  • Supports multiple data structures: Can serialize complex object graphs.

Example of usage：

Kryo kryo = new Kryo();
Output output = new Output(new FileOutputStream(""));
(output, yourObject);
();

Input input = new Input(new FileInputStream(""));
YourClass yourObject = (input, );
();

Protobuf (Google Protocol Buffers)

Introduction：

  • Language neutral, platform neutralThe scalable mechanism for serializing structured data.
  • Suitable for data storage and communication protocols.

Example of usage：

definition.protodocument:

syntax = "proto3";

message Person {
  string name = 1;
  int32 age = 2;
}

Generate Java classes and use:

Person person = ().setName("John").setAge(30).build();
FileOutputStream output = new FileOutputStream("");
(output);
();

FileInputStream input = new FileInputStream("");
Person person = (input);
();

Jackson

JSON serialization and deserialization：

  • Provides a simple and easy-to-use API to process JSON data.
  • Supports a wide range of Java object types.

Example of usage：

ObjectMapper objectMapper = new ObjectMapper();

// SerializationString jsonString = (yourObject);

// DeserializationYourClass yourObject = (jsonString, );

Serialization in gRPC

Introduction to gRPC

definition:gRPC is a high-performance, open source remote procedure call (RPC) framework developed by Google.

Features：

  • Supports multiple languages.
  • Based on the HTTP/2 protocol, it supports bidirectional flow and concurrent requests.
  • Provides load balancing, authentication, tracking and other features.

Application of Protobuf in gRPC

Role:Protobuf is the default interface definition language (IDL) of gRPC, used to define service and message formats.

Steps to use：

Define services and messages：

syntax = "proto3";

service Greeter {
  rpc SayHello (HelloRequest) returns (HelloResponse);
}

message HelloRequest {
  string name = 1;
}

message HelloResponse {
  string message = 1;
}

Generate code:useprotocThe compiler generates client and server-side code.

Implement service logic：

public class GreeterImpl extends  {
    @Override
    public void sayHello(HelloRequest req, StreamObserver<HelloResponse> responseObserver) {
        HelloResponse response = ()
            .setMessage("Hello, " + ())
            .build();
        (response);
        ();
    }
}

Advantages and Disadvantages of gRPC Serialization

advantage：

  • High efficiency:Protobuf serialization format is compact and suitable for network transmission.
  • Cross-language support: Supports multiple programming languages, making it easier to build multilingual systems.
  • Strong type: IDL is well defined to reduce communication errors.

shortcoming：

  • Learning curve: Protobuf and gRPC need to be learned and configured.
  • Binary format: Not as easy as JSON to debug and read.
  • Dependency Generation Tool: Need to depend onprotocTools generate code.

gRPC combined with Protobuf provides an efficient and flexible remote calling solution for systems requiring high performance and cross-language support.

Dubbo's default serialization

Introduction to Dubbo

definition:Dubbo is Alibaba's open source high-performance Java RPC framework.

Features：

  • Provide service governance, load balancing, automatic service registration and discovery.
  • Supports multiple protocols and serialization methods.

Dubbo supports serialization methods

  • Hessian: The default serialization method supports cross-language.
  • Java serialization: Use Java's own serialization mechanism.
  • JSON: Used for lightweight data transmission.
  • Protobuf: Efficient binary serialization format.
  • Kryo: A serialization solution for high performance and efficient space utilization.

Default serialization mechanism and its application

Hessian serialization：

  • Features: Supports cross-language and compact serialized data.
  • application: Suitable for scenarios where cross-language calls are required, especially for communication from Java to other languages.

Example of usage：

In Dubbo, the configuration serialization method is very simple and can be specified in the configuration of the service provider or consumer:

<dubbo:protocol name="dubbo" serialization="hessian2"/>

advantage：

  • Cross-language support:Hessian supports multiple language implementations.
  • Ease of use:Dubbo default configuration, ready to use out of the box.

shortcoming：

  • performance: Compared with Protobuf or Kryo, the performance may be slightly inferior.
  • readability: The binary format is not easy to debug.

Dubbo's default serialization mechanism provides good cross-language support and ease of use through Hessian, which is suitable for the needs of most distributed systems.

Notes on serialization

The security of serialization

risk：

  • Deserialization vulnerability: An attacker may execute arbitrary code through a maliciously constructed byte stream.
  • Data Breach: Unencrypted serialized data may be stolen.

Protective measures：

  • Whitelist mechanism: Restrict deserialized classes.
  • Using the security library: Choose a serialization framework with higher security, such as Protobuf.
  • Data encryption: Encrypted transmission of serialized data.

Version compatibility issues

challenge：

  • Changes in serialization format may cause old client or server to fail to resolve the new format.

Solution：

  • Backward compatible: Used in ProtobufoptionalField.
  • Version Management: Maintain a good version control policy and use version numbers to manage different serialization formats.
  • default value: Provides default values ​​for new fields to avoid parsing errors.

Performance considerations

Factors：

  • Speed ​​of serialization and deserialization.
  • Serialize the size of the data.

Optimization strategy：

  • Select an efficient framework: Such as Kryo or Protobuf.
  • Reduce data volume: Serialize only the necessary data.
  • Batch processing: Merge multiple messages to reduce network overhead.

When designing and implementing serialization mechanisms, security, version compatibility and performance need to be comprehensively considered to ensure the stability and efficiency of the system.

Scenarios in practical applications

Network transmission

Scene: Exchange data between the client and the server.

application：

  • RPC framework: For example, Dubbo and gRPC use serialization to make remote method calls.
  • Message Queue: Kafka, RabbitMQ, etc. serialize the messages and transmit them.

consider：

  • Choose an efficient serialization method to reduce bandwidth usage and increase transmission speed.

Data persistence

Scene: Save the object state to the storage medium.

application：

  • Database storage: Serialize complex objects and store them in the database.
  • File storage: Serialize objects into file formats such as JSON or XML.

consider：

  • It is necessary to ensure the stability and readability of the serialized format to facilitate subsequent data recovery and processing.

Applications in distributed systems

Scene: Share data between different nodes.

application：

  • Cache system: For example, Redis, serialize objects and store them to improve access speed.
  • Microservice communication: Interaction between services by serializing data.

consider：

  • Compatibility and consistency of serialization formats are required to support communication between different versions of services.

High-performance RPC framework design

The basic principles of the RPC framework

definition: Remote procedure call (RPC) allows programs to call functions in different address spaces just like calling local functions.

Components：

  • Client and server: The client initiates a request, the server processes the request and returns the result.
  • Communication Protocol: Define message format and transmission rules (such as HTTP/2, gRPC).
  • Serialization mechanism: Convert request and response objects into byte streams (such as Protobuf).
  • Service registration and discovery: Manage and discover service instances through the Service Registration Center.

How to implement millisecond service calls under 100,000 QPS

Efficient network protocol: Use low-overhead protocols such as HTTP/2 or custom binary protocols to reduce network transmission time.

Asynchronous IO: Use Netty and other frameworks to achieve non-blocking IO and improve concurrent processing capabilities.

Connection pool: Maintain long connection pools to reduce the overhead of connection establishment and closing.

Load balancing: Allocate requests between the client and the server to avoid single point of overload.

cache: Caches commonly used data on the client or server side to reduce duplicate calculations and transmissions.

Performance optimization strategy

Serialization optimization：

  • Use efficient serialization formats (such as Protobuf, Kryo) to reduce the overhead of serialization and deserialization.
  • Serialize only necessary data to reduce packet size.

Thread model optimization：

  • Use thread pool to manage request processing to avoid frequent creation and destruction of threads.
  • An event-driven model (such as Reactor mode) is used to handle high concurrent requests.

Resource Management：

  • Memory management: Use object pool to reduce GC pressure.
  • Connection Management: Optimize connection multiplexing and disconnection strategies.

Monitoring and tuning：

  • Monitor system performance indicators in real time and discover bottlenecks in a timely manner.
  • Continuous optimization through stress testing and analysis.

Serialization in Netty

Netty is a high-performance network application framework that is widely used to build highly concurrent network services. Serialization plays an important role in Netty, helping to convert data objects into byte streams for network transmission. The following are the commonly used serialization methods and implementations in Netty.

Netty itselfThere is no default serialization method. It provides flexible mechanisms that allow developers to choose and implement their own serialization methods as needed. By rationally selecting and optimizing serialization methods, the performance and reliability of the application can be significantly improved.

Commonly used serialization methods

Java native serialization

  • accomplish:useObjectInputStreamandObjectOutputStream。
  • advantage: Simple and easy to use.
  • shortcoming: Low performance, large serialized data.

Protobuf（Protocol Buffers）

  • accomplish: By definition.protoFile generation Java classes.
  • advantage: Efficient, cross-language support, clear data structure.
  • shortcoming: Need to be written and maintained.protodocument.

JSON

  • accomplish: Use libraries such as Jackson or Gson.
  • advantage: Good readability and easy to debug.
  • shortcoming: The performance is relatively low and the data volume is large.

Kryo

  • accomplish: Serialization using the Kryo library.
  • advantage: Efficient and support complex objects.
  • shortcoming: The class needs to be registered manually, which may not be suitable for all objects.

Serialization implementation in Netty

Encoder and decoder：

  • Netty byChannelHandlerIn-houseEncoderandDecoderImplement serialization and deserialization.
  • For example,ProtobufEncoderandProtobufDecoderUsed to process data in Protobuf format.

Custom serialization：

  • Can be achieved byMessageToByteEncoderandByteToMessageDecoderInterface custom serialization logic.
  • This allows developers to optimize the serialization process according to specific needs.

Using Java native serialization

rely

Make sure your project contains Netty's dependencies.

Sample code

import ;
import .*;
import ;
import ;
import ;
import ;
import ;
import ;

import ;

// Define a serializable objectclass MyObject implements Serializable {
    private static final long serialVersionUID = 1L;
    private String message;

    public MyObject(String message) {
         = message;
    }

    @Override
    public String toString() {
        return "MyObject{" +
                "message='" + message + '\'' +
                '}';
    }
}

// Server Processorclass ServerHandler extends SimpleChannelInboundHandler&lt;MyObject&gt; {
    @Override
    protected void channelRead0(ChannelHandlerContext ctx, MyObject msg) throws Exception {
        ("Received: " + msg);
        // Echo the received object back to the client
        (msg);
    }
}

// Server startup classpublic class NettyServer {
    public static void main(String[] args) throws Exception {
        EventLoopGroup bossGroup = new NioEventLoopGroup(1);
        EventLoopGroup workerGroup = new NioEventLoopGroup();

        try {
            ServerBootstrap b = new ServerBootstrap();
            (bossGroup, workerGroup)
                .channel()
                .childHandler(new ChannelInitializer&lt;SocketChannel&gt;() {
                    @Override
                    protected void initChannel(SocketChannel ch) throws Exception {
                        ChannelPipeline p = ();
                        (new ObjectDecoder((null)));
                        (new ObjectEncoder());
                        (new ServerHandler());
                    }
                });

            ChannelFuture f = (8080).sync();
            ().closeFuture().sync();
        } finally {
            ();
            ();
        }
    }
}

Things to note

  • performance: Java native serialization performance is low and is suitable for simple testing and learning environments. In production environments, it is recommended to use more efficient serialization methods such as Protobuf or Kryo.
  • Security: There may be security issues with Java native serialization, especially when deserializing. Make sure to deserialize only data from trusted sources.

By Netty'sObjectEncoderandObjectDecoder, can easily implement serialization and deserialization of Java objects. Choose the appropriate serialization method according to your needs to optimize performance and security.

Summarize

The above is personal experience. I hope you can give you a reference and I hope you can support me more.