Our Work

Reading large files asynchronously in ReactJS using chunks(Hapijs and Papaparse)

Updated 4 days ago


Node is an open source development platform for executing javascript code at server-side.
It is useful for applications need persistent connection between server and browser.It is mostly used for the real time applications such as chats and web push notifications.

Hapi js is a rich and open source node js framework for building applications and services.It enables developers to focus on writing reusable logic instead of building infrastructure.

You can refer the files provided in this 

https://github.com/KtreeOpenSource/NodejsExamples/tree/master/AsyncRead_LargeFiles_HapiJs

Step 1: Install Hapi js Framework

After running existing node js applications, install hapi js framework into it using below command  npm i hapi

Step 2: Define Route using Hapi js

Define a route inside an controller like below.Define method,path which must be a string and in the config define handler which will handles the request and send response to the client.whenever any request touches the server you will find http methods,request url and request payloads etc.In this route payload contains the content coming from the request payload.You can restrict the file maxsize using maxBytes and all.

  server.route({
        method: 'POST',
        path: '/api/dashboard',
        config: {
            handler: async (req, res) => {
//
},
            payload: {
                output: 'stream',
                parse: true,
                allow: 'multipart/form-data',
                maxBytes: 10 * 1024 * 1024     //10 mb,
            }
        }
    });

Step 4: Reading large files using papa-parse

Before reading the files install papa-parse into your existing application using below command npm i papaparse.Inside the handler function read the files in the form of step and chunks using papa-parse.In order to read the files step by step means single row at once, you can use step function in the papaparse.Once reading rows in file is done,it will call the complete method, where you can return the data. Check the below example

Papa.parse("http://example.com/big.csv", {
download: true,
step: function(row) {
console.log("Row:", row.data);
},
complete: function() {
console.log("All done!");
}
     });

Chunk is a callback function, identical to step. However, this function is executed after every chunk of the file is loaded and parsed rather than every row.Do not use both chunk and step callbacks together, the result will be undefined.
 

Papa.parse(fs.createReadStream(__dirname + relativePathToFile, 'utf8'), {
delimiter: delimiter,
    	header: true,
skipEmptyLines: true,
    	beforeFirstChunk: function(chunk) {
     return chunk;
},
chunk: function chunk(results, parser){
}
});


 

Looking for Meanstack Developer?

Please Contact us if you have any Mean Stack Implementation requirements. Hire dedicated Mean Stack developers or Mean Stack Development services from KTree. KTree is Best offshore Web & Mobile development company with extensive experience in NodeJS Backend Development

Request For Quote

Tags

    No tag results found for this post