Uploading file in chunks from React to Node-Express

In almost every web application we get some use cases where we need to upload files to server.
One of the easiest way is using input type="file" and then sending to server as a blob.

But sometimes sending files in chunks can be useful, for performance, security and other stuffs like showing upload completion percentage.

In this tutorial we are going to upload a file from React application to Server in node and express.

Step 1 Setting up React App

I am using react-formvik for quick form setup, I will create input type="file".



import React from 'react';
import { Form } from 'react-formvik';
const App = () => {
    return <Form
        config={
            {
                fields: [
                    {
                        field: 'avatar',
                        css: {
                            containerClass: 'mt-4',
                            inputClass: 'form-control',
                            labelClass: 'form-label'
                        },
                        inputProps: {
                            type: 'file'
                        },
                        label: 'Upload'
                    }
                ],
                css: {
                    formContainerClass: 'container'
                }

            }
        }
        onChange={async ({ avatar }) => {
           //We will add code here in further steps
        }
}

export default App;

I have also installed bootstrap to style our input field.
Add below line in index.js or you can import it in css file as well.



import 'bootstrap/dist/css/bootstrap.css';

Preview

Step 2: Setting up server

In express app, we need multer,
npm i -S multer

Add following line in your server file
const multer = require('multer');
const storage = multer.memoryStorage();
const upload = multer({ storage });

Make sure you have body parser setup in express app.

Create one route for upload



app.post('upload', (req, res) => {
    //We will add code here in further steps
});

Step 3: Understanding the approach of File chunking on Client

So, we are going to follow below sub steps

we will click browse on UI and upload a file(image)
In React component - in onChange method we will get file blob.
We will keep one constant for chunk size (say: 1024 bytes)
Knowing the chunkSize and file Size, we can find out how many chunks we need to create.
We will create one loop and slice blob based on chunk size and in each iteration we will be sending next chunk to the server
That means we are going to call server multiple times for the same route '/upload'. For backend(node) approach we will discuss in upcoming steps.

Step 4: Implementing chunking in onChange method

Now that we understand how we are going to implement we can start writing code.
First we need to get file details in onChange event



 onChange={async ({ avatar }) => {
            const chunkSize = 1024;
            if (avatar) {
                  // we will proceed if we have files here
             }

}

Next, lets find out total chunks we need



 onChange={async ({ avatar }) => {
            const chunkSize = 1024;
            if (avatar) {
                   const totalChunks = Math.ceil(avatar[0].size/chunkSize);
             }

}

Create a loop to iterate for each chunk



 onChange={async ({ avatar }) => {
            const chunkSize = 1024* 100;
            if (avatar) {
                   const totalChunks = Math.ceil(avatar[0].size/chunkSize);

           for(let i=0; i<totalChunks; i++) {
                    const start = i*chunkSize;
                    const end = (i + 1) * chunkSize;
                    await sendChunk(start,end);
                }
             }

}

Lets create a sendChunk method to send chunk to server.



 onChange={async ({ avatar }) => {
            const chunkSize = 1024;
            if (avatar) {
                   const totalChunks = Math.ceil(avatar[0].size/chunkSize);


                const sendChunk = async (start,end) =>{
                    const formData = new FormData();
                    const blobSlice = avatar[0].slice(start, end);
                    formData.append('file', blobSlice, avatar[0].name);
                    return await fetch('http://localhost:3000/upload', {
                        method: 'POST',
                        body: formData
                    });
                }
           for(let i=0; i<totalChunks; i++) {
                    const start = i*chunkSize;
                    const end = (i + 1) * chunkSize;
                    await sendChunk(start,end);
                }
             }

}

Step 5: Implementing server



app.post('/upload', upload.single('file'), (req, res) => {
    const { buffer, originalname } = req.file;
    const writeStream = fs.createWriteStream(originalname, { flags: 'a' });
    writeStream.write(buffer);
    writeStream.end();

    writeStream.on('finish', () => {
        res.status(200).send('File received successfully.');
    });

    // Event listener for any errors during the write operation
    writeStream.on('error', (err) => {
        console.error(err);
        res.status(500).send('Internal Server Error');
    });

Lets understand above code,

upload.single('file') - is a middleware for multer which expects a form data with a file name 'file'
In fs.createWriteStream we have added a flag 'a' which will actually append the buffer at the end of file every time it gets a request.