批量请求

9881559271729_.pic_hd.jpg

请实现如下函数, 可以批量请求数据, 所有的URL在地址urls参数中, 同时可以通过max参数控制请求的并发度, 当所有请求结束, 需要执行callback回调函数, 发请求的函数使用fetch即可.
function sendRequest(urls: string[], max: number, callback: () => void) {};

题目比较粗糙, 请求发送就结束了, 对返回的数据并没有要求保存

用两种方式实现了一下, 明显async方式可读性更高, 逻辑更清晰, 如果异步请求不用async, 需要在异步的回调中再对自身进行调用

const mapUrlList = (urls: string[]) => urls.map(url => fetch(url));

function sendRequest(urls: string[], max: number, callback: () => void) {
  if (urls.length === 0) {
    callback();
    return;
  }
  
  const reqList = urls.splice(0, max);
  Promise.all(mapUrlList(reqList)).then((respList) => {
    sendRequest(urls, max, callback);
  });
}

async function sendRequest2(urls: string[], max: number, callback: () => void) {

  while(urls.length > 0) {
    const reqList = urls.splice(0, max);
    await Promise.all(mapUrlList(reqList));
  }

  callback();
}

Screen Shot 2019-05-31 at 12.23.54 PM.png

花了心思实现了一版并发密集型的解法, 用promise.all 还是要等最长的请求结束才能进入下一轮, 新的解法能在给定的max-pool计数内并发请求, 如果有1个长多个短的请求, 就很高效, 异步明显在编码上比同步麻烦一些, 思考的维度不一样

enum ReqState {
  Active = 0,
  Progress,
  Done,
}

interface Request {
  url: string,
  state: ReqState
}

// simulate fetch
function doFetch(url: string) {
  return new Promise((resovle, reject) => {
    const wait = Math.floor(Math.random() * 3000);
    setTimeout(() => {
      resovle(`${url} ${repeatRand(10)}`);
    }, wait);
  });
}

function sendRequest3(urls: string[], max: number, callback: () => void) {
  const mapUrl = (url: string): Request => ({url, state: ReqState.Active});
  let reqList = urls.splice(0, max).map(mapUrl);

  function addNewRequest() {
    reqList = reqList.filter(req => req.state != ReqState.Done);
    
    //no new request
    if (urls.length === 0) {
      // no progressing request
      reqList.length === 0 && doOnce(callback);
      return;
    };

    const sub = max - reqList.length;
    const subList = urls.splice(0, sub).map(mapUrl);
    [].push.apply(reqList, subList);

    startRequest(subList);
  }

  function startRequest(urlList: Request[]) {
    urlList.forEach(o => {
      o.state = ReqState.Progress;

      doFetch(o.url).then((resp) => {
        console.log(`req done ${resp}`);
        o.state = ReqState.Done;
        addNewRequest();
      })
    });
  }

  startRequest(reqList);
}

const urls = [
  'www.baidu.com?page=1',
  'www.baidu.com?page=2',
  'www.baidu.com?page=3',
  'www.baidu.com?page=4',
  'www.baidu.com?page=5',
  'www.baidu.com?page=6',
  'www.baidu.com?page=7',
  'www.baidu.com?page=8',
  'www.baidu.com?page=9',
  'www.baidu.com?page=10',
  'www.baidu.com?page=11'
];

sendRequest3(urls, 4, () => {
  console.log(`all requests finished!`);
});

Screen Shot 2019-05-31 at 2.58.18 PM.png