Working with files, paths and streams in Node.js

Reading a config file, appending to a log, processing a large CSV: these are week-one tasks in any Node.js role, and screenings often ask you to write one from scratch. The mistakes are predictable, so this lesson shows the safe version of each task.

Read and write with promises

Use fs/promises with async/await for ordinary files. Always pass an encoding, or you get a Buffer instead of a string.

const fs = require("fs/promises");
const path = require("path");

async function readConfig() {
  const file = path.join(__dirname, "config", "app.json");
  const text = await fs.readFile(file, "utf8");
  return JSON.parse(text);
}

async function appendLog(line) {
  const dir = path.join(__dirname, "logs");
  await fs.mkdir(dir, { recursive: true });
  await fs.appendFile(path.join(dir, "app.log"), line + "\n");
}

path.join inserts the correct separator for the operating system. Using __dirname makes the path relative to the file that contains the code, not to wherever the process was started. In an ES module there is no __dirname; build it from import.meta.url with fileURLToPath.

Handling a missing file

File errors carry a code. Check it rather than matching the message text.

try {
  await fs.readFile("missing.txt", "utf8");
} catch (err) {
  if (err.code === "ENOENT") {
    console.log("File not found, using defaults");
  } else {
    throw err;
  }
}

Common codes: ENOENT (no such file), EACCES (permission denied), EEXIST (already exists).

Streams for large files

readFile loads the whole file into memory. For a multi-gigabyte log that is a crash. Streams process chunks as they arrive. pipeline connects them and forwards errors properly, which plain .pipe() does not.

const { createReadStream, createWriteStream } = require("fs");
const { pipeline } = require("stream/promises");
const zlib = require("zlib");
const readline = require("readline");

async function compress(src) {
  await pipeline(
    createReadStream(src),
    zlib.createGzip(),
    createWriteStream(src + ".gz")
  );
}

async function countLines(src) {
  const rl = readline.createInterface({ input: createReadStream(src) });
  let count = 0;
  for await (const line of rl) {
    if (line.trim() !== "") count++;
  }
  return count;
}

Common mistakes and a practice habit

• Calling fs.readFileSync inside a request handler. It blocks the event loop, so every other request waits.
• Using a relative path such as "./data.json", which breaks when the app is started from another folder.
• Forgetting "utf8" and then calling JSON.parse on a Buffer. It works by accident in some cases and fails in others.
• Writing to a file from several concurrent requests without a queue, which interleaves lines.

Practice habit: write a small script that reads a folder, prints each file's size using fs.stat, and gzips any file over one megabyte with a stream. Run it from two different working directories to confirm your paths hold.