How to save and retrieve files in Zip with NodeJS & ExpressJS?

File Import & Export

In a Management Information Systems (MIS), we often need to import and export files, especially when migrating from another system.

Assume you need to upload several files in your new system. Once done, when required, the user will download them. Instead of downloading each file individually or all of them in one go, you can compress them into a single zip file.

The user will therefore only need to download a single file and extract it locally. I notice the same behaviour when you download multiple files on Gmail and Google Drive.

Process

In this post, I shall share my experience on how to upload files in the server and for download, compress them as a single zip file before sending to the frontend.

At the start, when looking for solutions on how to zip multiple files, my main problem is the excessive amount of information on the subject. With such a variety, it's difficult to pinpoint exactly what is working and otherwise too many factors to check if it's not working.

This consumes a lot of time but nevertheless also presents an opportunity for learning; such as lower level concepts like streams. That's one of the main hiccups for using OpenSource software - no information available or too much - often these not being precise or providing lack of examples. Yet I can't complain as we get all these amazing technologies for free enabling us to make a living out of them. So for me, a big thanks to the authors.

The aim is to upload the files on the server in their original format, for example as a PDF, image or CSV since you might need to perform certain operations. Afterwards, when you are going to retrieve them, you will compress them as needed.

Libraries

Coding the whole thing from scratch or even using bit of snippets - gathered on the Internet is time consuming - when you can use one library for each step. As usual with JavaScript development, we are going to use ExpressJS.

For file upload, we are going to use MulterJS. Another efficient solution is express-fileupload.

Multer adds a file or files object to the request object depending if you're going to upload one or several files. Once you install multer using npm install multer or yarn add multer, you set up the destination as follows:

const multer = require('multer');
const upload = multer({dest:'uploads/'}).single("image");

You upload several files by using the array function.

app.post("/files", upload.array("files[]", 4), (req, res) =>{
  try {
    // file manipulation
  } catch (error) {
    console.log(error);
    res.send(400);
  }
});

Archiver

You can use Archiver to zip your files and send to the frontend. This library uses streams and events. The codes are available in the Github page and covers most of the usual cases.

I use the append file function, the easiest add a file from local storage given it's path and a name:

archive.file('file1.txt', { name: 'file1.txt' });

The code in the front:

axios.post(file) => {
  const url = window.URL.createObjectURL(new Blob([file]));
  const link = document.createElement('a');
  link.href = url;
  link.setAttribute('download', 'download.zip');
  document.body.appendChild(link);
  link.click();
}

In your axios response, make sure of using responseType: 'blob'.

References

https://afteracademy.com/blog/file-upload-with-multer-in-nodejs-and-express