在快递中上传multer-s3是非常慢的

Question

在我的快递应用程序中通过multer-s3上传到S3的文件速度非常慢（200kb需要10秒，延迟时间与文件大小相关）。问题是multer-s3，但有人可以告诉你如何进一步调试库中发生的事情吗？不再支持将onFileUploadStart等传递给multer。我不知道如何在multer-s3正在做什么以及在哪个阶段引起的延迟达到峰值。

问题不是AWS S3，因为从CLI上传是超快的，也不是因为上传到本地磁盘也很快。任何直接的S3 javascript操作也可以按预期的那样快速运行。

尝试关闭multer的内容自动类型功能，但它没有做太多。

var upload = multer({
  storage: multerS3({
    s3: s3,
    bucket: bucketName,
    acl: 'public-read',
    contentType: multerS3.AUTO_CONTENT_TYPE,
    metadata: function (req, file, cb) {
      cb(null, {fieldName: file.fieldname});
    },
    key: function (req, file, cb) {
      cb(null, Date.now().toString() + "-" + file.originalname);
    }
  },)
})

router.post('/', upload.array('files'), function(req, res, next) {
  console.log('Successfully uploaded ' + req.files.length + ' files!');
});

Want to have sub-second latency as I do with direct AWS CLI upload or using multer to store to local disk, takes 10 seconds to minute or more on sub-MB files.

Answer 1

获得洞察力的一种方法是在响应套接字上代理write函数，并检查它的调用方式：

var upload = multer({
  storage: multerS3({
    s3: s3,
    bucket: bucketName,
    acl: 'public-read',
    contentType: multerS3.AUTO_CONTENT_TYPE,
    metadata: function (req, file, cb) {
      cb(null, {fieldName: file.fieldname});
    },
    key: function (req, file, cb) {
      cb(null, Date.now().toString() + "-" + file.originalname);
    }
  },)
})

router.post('/'
(req, res, next) => { // debug middleware
  const write = res.socket.write;
  res.socket.write = (function(...args) {
    // inspect args, look at the time between invocations of write
    // also check the size of the chunks being passed to write
    return write(...args)
  }).bind(res.socket);
  next();
}
, upload.array('files'), function(req, res, next) {
  console.log('Successfully uploaded ' + req.files.length + ' files!');
});