Node js - Reading and Writing Files

Reading and Writing Files in Node.js

Introduction

Node.js provides powerful and flexible APIs for interacting with the file system. Whether you're building a logging utility, a data storage layer, or simply manipulating configuration files, reading from and writing to files is a critical operation in many applications. The core module that facilitates file system operations in Node.js is the fs module.

In this document, we will explore how to read and write files using both synchronous and asynchronous methods, how to work with streams for large files, and understand the differences and best practices. We will also look into real-world use cases, error handling, encoding, working with JSON, and modern promise-based APIs.

The fs Module

Node.js ships with a built-in module called fs (short for file system) that provides both synchronous and asynchronous functions for interacting with the file system. It allows you to read, write, rename, delete, and manipulate files and directories.

const fs = require('fs'); // CommonJS import

// Modern ES module syntax (Node 14+ with type="module")
// import fs from 'fs'; 

Reading Files

Asynchronous Reading

The fs.readFile() method is used to asynchronously read the entire contents of a file.

const fs = require('fs');

fs.readFile('example.txt', 'utf8', (err, data) => {
    if (err) {
        console.error('Error reading file:', err);
        return;
    }
    console.log('File content:', data);
});

Parameters

• Path: Path to the file
• Encoding: Optional, usually 'utf8'
• Callback: Function called with error and data

Synchronous Reading

The fs.readFileSync() method reads a file synchronously, blocking execution until complete.

const fs = require('fs');

try {
    const data = fs.readFileSync('example.txt', 'utf8');
    console.log('File content:', data);
} catch (err) {
    console.error('Error reading file:', err);
}

When to Use Synchronous vs Asynchronous

• Synchronous: Use only in startup scripts or scripts where blocking I/O is acceptable
• Asynchronous: Use in production for non-blocking performance

Writing Files

Asynchronous Writing

fs.writeFile() writes data to a file asynchronously. If the file doesn't exist, it's created.

const fs = require('fs');

fs.writeFile('output.txt', 'Hello, Node.js!', 'utf8', (err) => {
    if (err) {
        console.error('Error writing to file:', err);
        return;
    }
    console.log('File written successfully');
});

Synchronous Writing

fs.writeFileSync() writes data synchronously.

const fs = require('fs');

try {
    fs.writeFileSync('output.txt', 'Synchronous write', 'utf8');
    console.log('Synchronous write successful');
} catch (err) {
    console.error('Write error:', err);
}

Appending Data

To append content without overwriting the file, use fs.appendFile().

const fs = require('fs');

fs.appendFile('log.txt', 'New log entry\n', (err) => {
    if (err) throw err;
    console.log('Log updated.');
});

Working with JSON Files

Reading JSON

const fs = require('fs');

fs.readFile('data.json', 'utf8', (err, data) => {
    if (err) throw err;
    const obj = JSON.parse(data);
    console.log(obj);
});

Writing JSON

const fs = require('fs');

const person = {
    name: 'Alice',
    age: 30
};

fs.writeFile('person.json', JSON.stringify(person, null, 2), (err) => {
    if (err) throw err;
    console.log('JSON file saved.');
});

Using Promises with fs/promises

Modern Node.js supports promise-based file system operations via the fs/promises module.

const fs = require('fs/promises');

async function writeReadFile() {
    try {
        await fs.writeFile('hello.txt', 'Hello with Promises!', 'utf8');
        const content = await fs.readFile('hello.txt', 'utf8');
        console.log('Content:', content);
    } catch (err) {
        console.error(err);
    }
}

writeReadFile();

Working with Streams for Large Files

When dealing with very large files, loading the entire file into memory is inefficient. Instead, Node.js streams can read or write in chunks.

Read Stream

const fs = require('fs');

const readStream = fs.createReadStream('bigfile.txt', 'utf8');

readStream.on('data', (chunk) => {
    console.log('Chunk:', chunk);
});

readStream.on('end', () => {
    console.log('Finished reading file.');
});

Write Stream

const fs = require('fs');

const writeStream = fs.createWriteStream('output.txt');

writeStream.write('Line 1\n');
writeStream.write('Line 2\n');
writeStream.end('End of file\n');

Pipe Method

You can directly pipe read streams to write streams.

const fs = require('fs');

const readStream = fs.createReadStream('input.txt');
const writeStream = fs.createWriteStream('copy.txt');

readStream.pipe(writeStream);

Checking File Existence

Using fs.existsSync

const fs = require('fs');

if (fs.existsSync('check.txt')) {
    console.log('File exists');
} else {
    console.log('File not found');
}

Deleting Files

const fs = require('fs');

fs.unlink('delete_me.txt', (err) => {
    if (err) {
        console.error('Failed to delete file:', err);
    } else {
        console.log('File deleted');
    }
});

Renaming and Moving Files

const fs = require('fs');

fs.rename('oldname.txt', 'newname.txt', (err) => {
    if (err) throw err;
    console.log('File renamed');
});

Changing File Permissions

const fs = require('fs');

fs.chmod('example.txt', 0o777, (err) => {
    if (err) throw err;
    console.log('Permissions changed');
});

Error Handling

Always check for errors when performing file system operations, especially asynchronous ones.

fs.readFile('nonexistent.txt', 'utf8', (err, data) => {
    if (err) {
        if (err.code === 'ENOENT') {
            console.error('File does not exist');
        } else {
            console.error('An error occurred:', err);
        }
        return;
    }
    console.log(data);
});

Best Practices

• Use asynchronous methods to avoid blocking the event loop
• Use fs.promises with async/await for cleaner code
• Always handle errors gracefully
• Use streams for large files to improve performance
• Use path module for cross-platform path manipulation

Real-World Use Cases

Logging to a File

const fs = require('fs');

function log(message) {
    const timestamp = new Date().toISOString();
    fs.appendFile('app.log', `${timestamp} - ${message}\n`, (err) => {
        if (err) console.error('Failed to write to log');
    });
}

log('Application started');

Configuration Loader

const fs = require('fs');

fs.readFile('config.json', 'utf8', (err, data) => {
    if (err) {
        console.error('Failed to load config');
        return;
    }

    const config = JSON.parse(data);
    console.log('App configuration loaded:', config);
});

Node.js provides a comprehensive and flexible set of tools to read, write, and manage files efficiently. Understanding the differences between synchronous and asynchronous methods, using streams for large files, leveraging promise-based APIs, and ensuring proper error handling are essential skills for any Node.js developer.

Whether you are developing command-line utilities, backend services, or real-time applications, mastering the fs module and its capabilities can significantly improve your productivity and code quality. Remember to always choose non-blocking methods for better performance and responsiveness, and embrace modern patterns like async/await to write cleaner, more maintainable code.