Detailed explanation of batch processing and caching in JavaScript

Scene

Recently, I encountered the following scenario in the production environment:
The background stores some enumeration values that were previously maintained by the front and back end in the dictionary table, and provides an API for obtaining the dictionary based on type/id. So when rendering a list, many of the fields of the list are directly the id of the dictionary without being assembled in the background.

At first, the process of solving the problem is as follows

Determine the dictionary field and add the converted object type interface
Convert the object list to get all values in the dictionary field
Deduplicate the dictionary id list
Get all dictionary data from the background according to the id list
Convert the obtained dictionary data to a map of id => dictionary
Iterate through the initial list and convert the specified dictionary fields in it

As you can see, although the above steps are not troublesome, they are very cumbersome. Not only do you need to define additional types, but errors are also prone to occur.

Ideas

Optimize performance using asynchronous batch processing + LRU cache
Support asynchronous formatter for better user experience

Implement asynchronous batch processing

Reference implementation:

import { wait } from '../async/wait'

/**
  * Merge multiple concurrent asynchronous calls into one batch
  * @param handle Batch function
  * @param ms The length of waiting (the longer the time is, the more calls may be merged, otherwise all calls executed synchronously will be merged using microtasks only once)
  */
export function batch&lt;P extends any[], R extends any&gt;(
 handle: (list: P[]) =&gt; Promise&lt;Map&lt;P, R | Error&gt;&gt;,
 ms: number = 0,
): (...args: P) =&gt; Promise&lt;R&gt; {
 //Parameter => Result Mapping const resultCache = new Map&lt;string, R | Error&gt;()
 //Parameter => Mapping of times const paramCache = new Map&lt;string, number&gt;()
 //Is it currently locked let lock = false
 return async function (...args: P) {
  const key = (args)
  (key, ((key) || 0) + 1)
  await ([wait(() =&gt; (key) || !lock), wait(ms)])
  if (!(key)) {
   try {
    lock = true
    (
     await handle((()).map((v) =&gt; (v))),
    ).forEach(([k, v]) =&gt; {
     ((k), v)
    })
   } finally {
    lock = false
   }
  }
  const value = (key)!
  (key, (key)! - 1)
  if (((key) || 0) &lt;= 0) {
   (key)
   (key)
  }
  if (value instanceof Error) {
   (key)
   throw value
  }
  return value as R
 }
}

The basic ideas for implementing batch processing are as follows

1. Use Map paramCache to cache incoming parameters => The remaining number of calls (the result needs to be queried for this parameter)
2. Use Map resultCache to cache parameters => results
3. Use lock to identify whether a function is currently executing
4. Waiting to meet the following conditions
The results are not included in the map
There are currently other calls being executed
The minimum waiting time has not yet been completed (the minimum time segment of the call is collected)
5. Use lock to identify execution
6. Determine whether there is a result
If it does not exist, perform batch processing to process all current parameters.
7. Get results from cached maps
8. Change the remaining number of calls to the corresponding parameter in paramCache -1
9. Determine whether the cache needs to be retained (the remaining number of calls corresponding to this parameter is 0)
If not needed, delete
10. Determine whether the cached result is an Error
If so, throw an error

LRU Cache

refer to:Wiki cache algorithm, accomplishMemoryCache

Q: Why is cache used here?
Answer: The dictionary interface here is highly likely to be idempotent, so you can use cache to improve performance.
Q: So why do you need to choose LRU for caching policy?
A: There is no doubt that FIFO is unreasonable
Q: Then why not choose the LFU algorithm? It seems to retain the most frequently visited resources
A: Because the dictionary table is not completely idempotent, we want to avoid a possibility - the most visited dictionary has not been deleted, and it has been updated in the database.

The general implementation idea is as follows

1. Use a Map record to cache key => last access time
2. Update the last access time every time you get the cache
3. Check the number of caches when adding new caches
If the maximum number exceeds, delete the cache with the longest last access time distance now
4. Add a new cache
Pass: Don't complain about the poor performance. There will be no cache of particularly many elements in this scenario, at most, there will be less than 1,000.

Combined with higher-order functions

Now, we can combine these two methods and use two higher-order functions of onceOfSameParam/batch to optimize the API for obtaining dictionary information based on id.

const getById = onceOfSameParam(
 batch&lt;[number], Dict&gt;(async (idList) =&gt; {
  if ( === 0) {
   return new Map()
  }
  // Batch processing of multiple ids at a time  const list = await (uniqueBy(()))
  return arrayToMap(
   list,
   (dict) =&gt; [],
   (dict) =&gt; dict,
  )
 }, 100),
)

Support asynchronous formatter

Originally wanted to support the asynchronous process of ListTableformatterFunction, but then think about it, ifslotWhat does it also contain dictionary id? Is thatslotShould it also support asynchronousness? This is a difficult problem, so it is better not to support it.

Finally, we added a *Service middle layer between the component and the API to handle data transformation.

The above is a detailed explanation of batch processing and caching in JavaScript. For more information about batch processing and caching in JavaScript, please pay attention to my other related articles!