Uploading files is one of the most common tasks performed by web developers. It’s a pretty simple operation when using PHP or other server side frameworks. However, it is not so straightforward when using Node.js, unless you understand how to user Buffer API for reading streams of binary files (i.e. uploading file content as binary code). Luckily for us, the Node.js community came up with a few solutions.
Also Read:- How to Implement Real Time Notification Using Socket.io and Node.JS?
Project set up
We are using React.js on the client, Express.js as the middle (orchestration) layer and external REST API, which is not part of React/Express architecture. One way to handle files is to upload them directly to a server through API from the browser, which may not be ideal in every scenario, for example, when the API is on a different domain or you want to modify the file before sending to API.
Node server is a perfect solution in this case. However, as I mentioned before, developers have to jump through some hoops here. Let’s see how we can do it.
Client
Here is a simple input field with an event handler:
// Redux action
export function uploadSuccess({ data }) {
return {
type: 'UPLOAD_DOCUMENT_SUCCESS',
data,
};
}
export function uploadFail(error) {
return {
type: 'UPLOAD_DOCUMENT_FAIL',
error,
};
}
export function uploadDocumentRequest({ file, name }) {
let data = new FormData();
data.append('file', document);
data.append('name', name);
return (dispatch) => {
axios.post('/files', data)
.then(response => dispatch(uploadSuccess(response))
.catch(error => dispatch(uploadFail(error));
};
}
handleFileUpload({ file }) {
const file = files[0];
this.props.actions.uploadRequest({
file,
name: 'Awesome Cat Pic'
})
}
<input type="file" onChange={this.handleFileUpload} />
Let’s go over what’s going on in the client. When the input changes (file is added), the event handler will fire the Redux action creator and pass the file and name as arguments. The action creator will build FormData object, which is required for handling multipart/form-data. After we append file and name to the form’s data object, it’s good to go to the server. Axios is a great library for making HTTP requests. In this case, we’re POST`ing data to Node.js route. Let’s look at what’s happening on the server.
Also Read:- How to Create Calculator Using HTML, CSS and JavaScript
Server
Once the client gives the server a file and other relevant meta info associated with the file, the server has to know how to handle the binary file. Few libraries can help with this. We’ll be using Multer, which is maintained by the Express.js team.
import express from 'express';
import axios from 'axios';
import multer from 'multer';
const app = express();
/**
... express.js boilerplate
routes, middlewares, helpers, loggers, etc
**/
// configuring Multer to use files directory for storing files
// this is important because later we'll need to access file path
const storage = multer.diskStorage({
destination: './files',
filename(req, file, cb) {
cb(null, `${new Date()}-${file.originalname}`);
},
});
const upload = multer({ storage });
// express route where we receive files from the client
// passing multer middleware
app.post('/files', upload.single('file'), (req, res) => {
const file = req.file; // file passed from client
const meta = req.body; // all other values passed from the client, like name, etc..
// send the data to our REST API
axios({
url: `https://techmekrz.com/uploads`,
method: 'post',
data: {
file,
name: meta.name,
},
})
.then(response => res.status(200).json(response.data.data))
.catch((error) => res.status(500).json(error.response.data));
});
Let’s go over what’s going on in the server. First, we’re configuring multer to use local /files/ directory to store uploaded files from the client. It’s important to hold the file somewhere before we send it to REST API, because in most cases we’ll have to provide full path to the files. Another option is to hold it in memory, but that may cause the server to crash. Next, we want to make sure we generate unique file names with file extensions (not provided by default).
Also Read:- How to Build a Chrome Extension in JavaScript
The next step is creating the actual route where the client sends FormData and where we pass multer middleware. When the route receives a file, it goes through the middleware first and is stored in our /files directory with a newly generated file name. So, when we get to the callback (which can be refactored to custom middleware), the file is available as part of req object. From here, we can do whatever is needed, in this case, calling our external API (could be S3 or any API that handles files) and passing file and meta info.
Update 06/25/2017. Using streams to upload directly to cloudinary CDN.
Another way to upload files is to use Node.js streams instead of storing temporary files before sending them to CDN. Here is how the middleware looks like.
import axios from 'axios';
import cloudinary from 'cloudinary';
export default function fileUploadMiddleware(req, res) {
cloudinary.uploader.upload_stream((result) => {
axios({
url: '/api/upload', //API endpoint that needs file URL from CDN
method: 'post',
data: {
url: result.secure_url,
name: req.body.name,
description: req.body.description,
},
}).then((response) => {
res.status(200).json(response.data.data);
}).catch((error) => {
res.status(500).json(error.response.data);
});
}).end(req.file.buffer);
}
Once you have this middleware to handle streaming files to CDN, you can add new route which will trigger the middleware
import multer from 'multer';
import cloudinary from 'cloudinary';
import fileUploadMiddleware from './fileUploadMiddleware';
/* your servrer init and express code here */
cloudinary.config({
cloud_name: 'xxx',
api_key: 'xxxx',
api_secret: 'xxxxx',
});
/**
* Multer config for file upload
*/
const storage = multer.memoryStorage();
const upload = multer({ storage });
app.post('/files', upload.single('file'), fileUploadMiddleware);
/* the rest of your routes app.get('*', () => {}) */
/* the rest of your server code */
Now you can upload files from React (or any other client side framework). Here is an example:
import React, { Component } from 'react';
import axios from 'axios';
class uploadMyFile extends Component {
handleUploadFile = (event) => {
const data = new FormData();
data.append('file', event.target.files[0]);
data.append('name', 'some value user types');
data.append('description', 'some value user types');
// '/files' is your node.js route that triggers our middleware
axios.post('/files', data).then((response) => {
console.log(response); // do something with the response
});
render() {
<div>
<input type="file" onChange={this.handleUploadFile} />
</div>
}
}
export default uploadMyFile;
Also Read:- How to build Snake using only JavaScript, HTML & CSS: Think like a Developer
Conclusion
It may look like an over-engineered solution to a common problem. Nevertheless, there will be cases where you’ll need to use a node server in the middle before sending files to another source. It’s especially tricky when file uploads are not part of the form submit method, and there is a need to handle file uploads independently.
