Zipadeedoodah š¤ - Download Multiple Files To Zip On Client Browser
Craig McNicholas
Posted on November 14, 2024
If you've ever had to set up a backend API just to zip and download files (Google Drive, OneDrive etc.), you know it can be a pain in the nacellesāand it costs you additional infrastructure. This article shows you a simpler way - how to let your users zip files right in their browser. No backend required. ššļø "moon lambo" your infrastructure cost saving.
Why did I write this? We're developing a new product over at Vismo that deals with a lot of files and a subset of users would like "Download as Zip" functionality for their GeoSpatial data. We didn't want to stand up more infrastructure to support this so we have an effective solution for our customer base. Be sure to check your users and especially browser support
A Traditional Approach
Let's say you've got a bunch of files sitting in an S3 bucket in AWS. Normally, to zip these files for download, you would:
- Fetch the file records from a database.
- Read them from S3.
- Zip them up on the backend.
- Stream the zipped file back to S3.
- Expose a pre-signed URL (or similar) to the user to download.
All that works but hereās the thing: every time someone requests a zip, you pay for compute and you pay for data egress when it leaves AWS. You can of course just try to download files individually and struggle with the browser blocking repeated downloads or a users download folder with 100's of files in it.
With client-side zipping, we can change all that. By handling the zipping right in the user's browser, we keep things simple(ish), cheaper, and easier to show progress to the user directly in page. Let's get into the pros and cons.
Pros of Client-Side Zipping
- No Additional Infrastructure Cost ā Skip the long-lived backend API and save on compute resources.
- No Zip Hosting Cost ā Since thereās no server-side zipping, you donāt have to worry about hosting temporary zip files.
- Better User Feedback ā Show the download progress directly in the browserāgreat for the user experience.
Cons of Client-Side Zipping
- Browser Compatibility ā This approach relies on the File System Access API, which only works in modern browsers.
- No Compression ā Files download in their original size unless you've already got them compressed on the server.
- File Availability ā You need a way to serve the files securely to the browser, like with presigned URLs or similar.
How It Works
In a compatible browser, we can progressively enhance our app to zip and download files directly on the client. Hereās the idea:
- Request Files Individually ā The client requests each file directly.
- Stream Files to Zip Archive ā Using the File System Access API, the browser writes the files into a zip archive on the user's device.
- No Storage/Infra Cost ā Since the zipping happens locally, you're not paying to store the zip files or the compute required to support this.
For cases where data transfer is free (there are providers other than AWS out there, shocking I know), this method is a big win. No backend processing, no zip storage, no hassle. Itās fast, itās efficient, and your users get their files in one clean download.
So let's dive in to what this looks like.
Tooling Up: Zip.js
To make this all work smoothly, weāre going to use one package: the excellent Zip.js. Why Zip.js? Zip.js can handle a response buffer in chunks, flushing data to disk periodically as it streams. This keeps memory consumption low because it doesn't try to load the entire file into memory before writing. Some libraries do that, if you're working with large files it can crash your browser tab as you overconsume.
With Zip.js, though, we get efficient zipping that plays nice with the browser.
Step 1: Getting a Writable File Handle
First we need to get a writable file handle from the browser to save our zip content. To do this, weāll use showSaveFilePicker
. Now, fair warning: showSaveFilePicker
doesnāt have built-in typing in TypeScript yet. So, weāll get a little creative and opt out of type safety for this part.
To check if a browser supports this, use the following compatibility check:
const compatible = typeof window['showSaveFilePicker'] === 'function';
If compatible is true, youāre good to go! If not, youāll want to fall back to a more basic download approach, where files are downloaded individually.
This check lets you provide client-side zipping as an optional, progressive enhancement for browsers that support it. For those that donāt, users still get their files, just without the one-click zip download.
// showSaveFilePicker is not part of the standard types yet
// (hence the comment of using a modern browser)
const showSaveFilePicker = (window as any).showSaveFilePicker;
// get the file handle for the zip file
const handle: FileSystemFileHandle = await showSaveFilePicker({
suggestedName: 'download.zip',
startIn: 'downloads',
types: [{
description: 'ZIP Files',
accept: { 'application/zip': ['.zip'] },
}],
});
// create a writable stream to the file handle
const writable: FileSystemWritableFileStream = await handle.createWritable({
keepExistingData: false,
});
Step 2: Setting Up the Zip Archive
Now letās set up the zip archive. Weāre going to configure it for low memory use and fast processing. Hereās how it works.
Set Chunk Size ā The
chunkSize
option determines how much data is downloaded and processed at once. Smaller chunks take a bit longer to download, but if a chunk fails, the smaller size makes it easier to retry without starting over.Configure Compression ā Weāll set
compression
to zero and enablepassThrough: true
(also known asSTORE
mode) on the zip instance. This disables compression. Why? Because compressing files uses a lot of memory and we don't want to keep large chunks of the file in memory to avoid crashing the page, we want to offload and stream to disk quickly.
This setup means we can keep memory use low. Hereās an example of what this looks like in code:
// configure the zip js library to use a chunk size of 5MB
configure({ chunkSize: 5000 * 1024 });
// create a new zip writer
const zipWriter = new ZipWriter(writable, {
compressionMethod: 0,
passThrough: true,
bufferedWrite: true,
});
Step 3: Download & Zip the Files
Now, letās gather the files we need to download and zip. Youāll typically fetch file metadata from your API e.g. file name, URL, and size. Your API should return the data in a format you can process like the example below:
const files = [
{
name: 'myfilebigfile.mp4',
url: 'https://myproduct.com/path/to/myfilebigfile.mp4',
size: 1073741824, // 1GB
},
{
name: 'anotherfile.jpg',
url: 'https://myproduct.com/path/to/anotherfile.jpg',
size: 10485760, // 10MB
}
];
Adding Files to the Zip Archive
With an array of file models in hand, we can now loop through each file and add it to our zip archive. Zip.js makes this process efficient by handling each file as a stream and sending requests with Range headers. This avoids loading the entire file into memory.
Hereās how the code looks:
for (const file of files) {
// Add each file to the zip archive
await zipWriter.add(file.name, new HttpRangeReader(file.url), {
uncompressedSize: file.size,
async onprogress(progress, total) {
// Update the progress for the current file
// (could be displayed in a progress bar)
console.log('entry progress:', file.name, progress, total);
},
});
}
In this example, each add
call waits until the previous file completes streaming otherwise we risk saturating the network (you could have a worker pool here to process multiple but that's beyond this article). Weāre also updating progress in real-timeāgreat for giving users feedback if you want to display progress bars or status indicators.
With this, your users can download a custom zip of files without touching a backend. Just efficient, on-the-fly zipping directly in the browser.
Step 4: Cleaning It All Up
To wrap things up, make sure you properly close the zip archive. This final step ensures all data is written correctly, producing a valid zip file for download.
Simply call:
await zipWriter.close();
This command flushes the zip data to the file, ensuring there are no loose ends. Without it, the zip file might end up incomplete or corrupted, which isnāt ideal for your users!
The Full Code
If you're ready to see it all together, hereās the complete code sample for client-side zipping:
import {
HttpRangeReader,
TextReader,
ZipWriter,
configure,
} from '@zip.js/zip.js';
// this is the file we want to download, it should be returned from your
// api/database or whatever you are using to store references to files
type DownloadFile = {
url: string;
name: string;
size: number;
};
// configure the zip js library to use a chunk size of 5MB
// you can tweak this if you want, a larger size means more
// risk of network failure and having to retry the chunk,
// a smaller size means more chunks and longer time
configure({ chunkSize: 5000 * 1024 });
export async function downloadAsZip(files: DownloadFile[]) {
// check if the browser supports `showSaveFilePicker`
const compatible = typeof window['showSaveFilePicker'] === 'function';
if (!compatible) {
// handle your fallback here, e.g. window.open each file
return;
}
// showSaveFilePicker is not part of the standard types yet
// (hence the comment of using a modern browser)
const showSaveFilePicker = (window as any).showSaveFilePicker;
// get the file handle for the zip file
const handle: FileSystemFileHandle = await showSaveFilePicker({
suggestedName: 'download.zip',
startIn: 'downloads',
types: [{
description: 'ZIP Files',
accept: { 'application/zip': ['.zip'] },
}],
});
// create a writable stream to the file handle
const writable: FileSystemWritableFileStream = await handle.createWritable({
keepExistingData: false,
});
// create a new zip writer
const zipWriter = new ZipWriter(writable, {
compressionMethod: 0,
passThrough: true,
bufferedWrite: true,
});
for (const file of files) {
// add each file to the zip
await zipWriter.add(file.name, new HttpRangeReader(file.url), {
uncompressedSize: file.size,
async onprogress(progress, total) {
// log the progress of the file download to the user
// e.g. in a progress bar
console.log('entry progress:', file.name, progress, total);
},
});
}
// cleanup the zip
await zipWriter.close();
}
Posted on November 14, 2024
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.