Summary of various ways to deduplicate JS object

ES5 method

In ES5, we can useforLooping and array method to realize deduplication of object arrays.

function uniqueArrayByProperty(array, property) {
  var seen = {};
  return (function(item) {
    var val = item[property];
    return (val) ? false : (seen[val] = true);
  });
}

var arr = [
  { id: 1, name: 'Alice' },
  { id: 2, name: 'Bob' },
  { id: 1, name: 'Alice' }, // repeat  { id: 3, name: 'Charlie' }
];

var uniqueArr = uniqueArrayByProperty(arr, 'id');
(uniqueArr);
// Output: [ { id: 1, name: 'Alice' }, { id: 2, name: 'Bob' }, { id: 3, name: 'Charlie' } ]

ES6 and after

With the introduction of ES6, there are more concise ways to deal with arrays and objects.

Use Set and /

This method is suitable for simple objects, but it needs attentionThe properties on the prototype chain will be lost, and an error will be thrown for objects containing circular references.

function uniqueArrayByPropertyES6(array, property) {
  const seen = new Set();
  return (item => {
    const val = (item[property]);
    return (val) ? false : (val);
  });
}

var uniqueArrES6 = uniqueArrayByPropertyES6(arr, 'id');
(uniqueArrES6);

Using Map

For more complex object deduplication, you can useMapTo store objects you have seen.

function uniqueArrayByPropertyWithMap(array, property) {
  const map = new Map();
  return (item => {
    const val = item[property];
    if (!(val)) {
      (val, true);
      return true;
    }
    return false;
  });
}

var uniqueArrWithMap = uniqueArrayByPropertyWithMap(arr, 'id');
(uniqueArrWithMap);

Use Map and custom comparison logic

This method is suitable for situations where an object has multiple properties, and the comparison logic can be customized.

function uniqueArrayByProperties(array, properties) {
  const seen = new Map();
  return (item =&gt; {
    const key = (prop =&gt; item[prop]).join('|');
    if (!(key)) {
      (key, true);
      return true;
    }
    return false;
  });
}

const arr = [
  { id: 1, name: 'Alice' },
  { id: 2, name: 'Bob' },
  { id: 1, name: 'Alice' }, // repeat  { id: 3, name: 'Charlie' }
];

const uniqueArr = uniqueArrayByProperties(arr, ['id', 'name']);
(uniqueArr);
// Output: [ { id: 1, name: 'Alice' }, { id: 2, name: 'Bob' }, { id: 3, name: 'Charlie' } ]

Using the Lodash library

Lodash provides a convenient function_.uniqBy, can easily remove weight.

const _ = require('lodash');

const uniqueArr = _.uniqBy(arr, 'id');
(uniqueArr);
// Output: [ { id: 1, name: 'Alice' }, { id: 2, name: 'Bob' }, { id: 3, name: 'Charlie' } ]

Use WeakMap

WeakMapIt can be used to store object references without preventing these objects from being garbage collected. AlthoughWeakMapThe key must be an object, but it can be used to store already processed objects.

function uniqueArrayByPropertyWithWeakMap(array, property) {
  const seen = new WeakMap();
  return (item =&gt; {
    const val = item[property];
    if (!(val)) {
      (val, true);
      return true;
    }
    return false;
  });
}

const uniqueArrWithWeakMap = uniqueArrayByPropertyWithWeakMap(arr, 'id');
(uniqueArrWithWeakMap);
// Output: [ { id: 1, name: 'Alice' }, { id: 2, name: 'Bob' }, { id: 3, name: 'Charlie' } ]

Using Set and WeakRef (experimental)

In the latest JavaScript version,WeakRefIt can be used to store weak references of objects, which can be used to assist in deduplication of objects, but please noteWeakRefis an experimental feature that may change in future versions.

const { WeakRef } = require('weakref'); // Assume this is an environment that supports WeakRef
function uniqueArrayByPropertyWithWeakRef(array, property) {
  const seen = new Set();
  return (item =&gt; {
    const ref = new WeakRef(item[property]);
    if (!(ref)) {
      (ref);
      return true;
    }
    return false;
  });
}

const uniqueArrWithWeakRef = uniqueArrayByPropertyWithWeakRef(arr, 'id');
(uniqueArrWithWeakRef);
// Output: [ { id: 1, name: 'Alice' }, { id: 2, name: 'Bob' }, { id: 3, name: 'Charlie' } ]

Use Set combined

For simple objects, you can useTo generate a string representation of the object, and then useSetDeduplication.

function uniqueArrayByPropertyWithSet(array, property) {
  const seen = new Set();
  return (item =&gt; {
    const val = (item[property]);
    return (val) ? false : (val);
  });
}

const uniqueArrWithSet = uniqueArrayByPropertyWithSet(arr, 'id');
(uniqueArrWithSet);
// Output: [ { id: 1, name: 'Alice' }, { id: 2, name: 'Bob' }, { id: 3, name: 'Charlie' } ]

Summarize

The above lists some different methods to implement JavaScript object array deduplication (this requires array object deduplication often occurs in projects). Which method to choose depends on your specific needs and environment. For simple objects, useSetandProbably the easiest way; for more complex situations, useMaporWeakMapMaybe more suitable. If using a third-party library, Lodash's_.uniqByFunctions provide convenient solutions. For experimental functions, such asWeakRef, compatibility and stability issues need to be considered.

The above is a detailed summary of various ways to deduplicate JS object. For more information about deduplication of JS object, please pay attention to my other related articles!