When trying to write migration scripts, one of the most common patterns that one would want to solve for is not hitting API rate limits and also wait enough before processing the next item.
Problem
// list of tasks const tasks = [1,2,3,4,5,6,7,8,9,10] // simple processor to print the item, can by async const processor = (item) => { console.log(item) } async function main() { await tasks.forEach( async (item) => { processor(item) await new Promise(r => setTimeout(r, 2000)); }) console.log('complete') } main()
Why will above not work
forEachdoes not await.
awaitinprocessoris redundant and has no impact
Solution 1 - using a simple for loop
const tasks = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]; const processor = (item) => { console.log(item); }; async function main() { for (const item of tasks) { processor(item); await new Promise((r) => setTimeout(r, 2000)); // Wait 2 seconds before processing the next item } console.log('complete'); } main();
Solution 2 - create a batched function call
const tasks = [1,2,3,4,5,6,7,8,9,10] // simple processor to print the item, can by async const processor = (item) => { console.log(item) } const batchedCall = (list, cb, interval) => { async function processIndex(idx, list, cb, interval, r) { // resolve if the list is complete. if (idx >=list.length) { r(); return; } // process the task, await here is optional await cb(list[idx]); // wait for the mentioned timeout await new Promise(r => setTimeout(r, interval)); // process the next index processIndex(idx+1, list,cb,interval,r) } // call the processIndex function with first index return new Promise( r => processIndex(0, list,cb,interval, r)); } async function main() { await batchedCall(tasks,processor,1000) console.log('complete') } main();
Newsletter