Java implementation of deduplication of custom object array using Stream stream

In Java, the core of using Stream streams to deduplicate custom object arrays is to ensure that the object can correctly judge "repeat". The following are the specific implementation methods and scenario analysis:

Method 1: Use distinct() directly (need to rewrite equals and hashCode)

If the custom object has been rewrite correctlyequals()andhashCode()Method, can be directly passeddistinct()Remove the heavy weight.
Applicable scenario: The uniqueness of an object is determined by all fields (such as the primary key of the database entity class).

Sample code

public class Person {
    private String id;
    private String name;

    //Construction method, Getter/Setter omitted
    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != ()) return false;
        Person person = (Person) o;
        return (id, ); //Judge whether it is equal based on id    }

    @Override
    public int hashCode() {
        return (id); // Generate hash based on id    }
}

// Use Stream to remove the reloadPerson[] people = ...; // Custom object arrayList<Person> uniqueList = (people)
        .distinct()
        .collect(());

Method 2: Deduplication based on a unique property of the object (no need to rewrite equals and hashCode)

If the object class cannot be modified (such as the class of a third-party library), or if it needs to be deduplicated according to some fields, it can be used.orTreeSetaccomplish.

(1) Use

Applicable scenarios: According to the unique key (such asid) Deduplication, retaining the first element that appears.

List<Person> uniqueList = (people)
        .collect((
                Person::getId, // Key extraction function (deduplication according to id)                p -> p,       // Value is the object itself                (existing, replacement) -> existing // Keep existing elements during conflict        ))
        .values()            // Get the Value collection after deduplication        .stream()
        .collect(());

(2) Use TreeSet to customize the comparator

Applicable scenarios: Deduplication is required based on multiple fields, or dynamically specify deduplication rules.

List<Person> uniqueList = (people)
        .collect((
            () -> new TreeSet<>((p -> () + ()))
        ))
        .stream()
        .collect(());

Method 3: Use filter + memory state to dereload

Applicable scenarios: Deduplication needs to be deduplicated according to dynamic conditions (such as retaining the last element after deduplication).

(1) Use ConcurrentHashMap to maintain status

Set<String> seenIds = ();
List<Person> uniqueList = (people)
        .filter(p -> (())) // If id has not appeared, keep it        .collect(());

(2) Keep the last element that appears

List<Person> uniqueList = (people)
        .collect((
                Person::getId,
                p -> p,
                (oldValue, newValue) -> newValue // Keep new elements during conflict        ))
        .values()
        .stream()
        .collect(());

Summary answer

• Deduplication directly: If the object is rewrittenequals()andhashCode(), use it directlydistinct()。
• Deduplication by attribute: UseorTreeSet, according to the unique key (such asid)filter.
• Dynamic Deduplication: ByfilterCombined with memory collections (such asConcurrentHashMap) Control the deduplication logic.

Key points: clarify the definition of "repeat" in the business (such as equal all fields or equal some fields), and choose a solution that takes into account performance and code simplicity.

This is the article about Java's implementation of using Stream stream to deduplicate custom object arrays. For more related Java custom object array deduplication content, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!