Skip to content
← All posts

Nov 18, 2026

Your scheduled job ran twice — and nobody noticed

Every dashboard eventually grows a scheduled job. A low-stock digest that emails shop owners every morning. A nightly report. A cleanup that archives old orders. You write it once, you schedule it, it runs on time for weeks — and then one morning a shop owner gets the same digest twice, or the same invoice is generated two times, and nobody can explain why.

Cron does not check whether the previous run finished before it starts the next one. That is the whole bug, and it hides until the job gets slow. A digest that normally takes ninety seconds is fine on a five-minute schedule — until the day it takes six minutes, because the table grew, or a downstream API was slow, or there were simply more rows than usual. Now the next scheduled fire starts while the first one is still working, and two copies of the same job are reading and writing the same data at the same time.

The second way it happens is scaling. One server, one crontab, one job — no problem. Add a second server for a zero-downtime deploy, or run the app in cluster mode, and every instance runs the same schedule independently. Cron was designed for a single machine. It has no idea it has siblings.

The fix, for a single server, is a flag that lives outside the job itself:

let isRunning = false;

cron.schedule("*/5 * * * *", async () => {
  if (isRunning) {
    console.log("Previous run still going, skipping this one");
    return;
  }
  isRunning = true;
  try {
    await sendLowStockDigest();
  } finally {
    isRunning = false;
  }
});

The finally matters more than the flag. If the job throws and the flag never resets, every future run skips itself and the job silently dies. But a flag only protects one process. The moment there are two servers, the flag has to live somewhere both of them can see — and that is where a lock comes in.

If your data already lives in PostgreSQL, you probably do not need a new service for this. Advisory locks are application-level locks: you pick a number that means "this job", and Postgres enforces that only one session holds it at a time.

import { Pool } from "pg";

const pool = new Pool({ connectionString: process.env.DATABASE_URL });

async function runJobWithLock(jobKey, jobFn) {
  const client = await pool.connect();
  try {
    const { rows } = await client.query(
      "SELECT pg_try_advisory_lock($1) AS got_it",
      [jobKey]
    );
    if (!rows[0].got_it) {
      console.log("Another instance is already running this job, skipping");
      return;
    }
    await jobFn();
  } finally {
    await client.query("SELECT pg_advisory_unlock($1)", [jobKey]);
    client.release();
  }
}

cron.schedule("0 8 * * *", () =>
  runJobWithLock(1001, sendLowStockDigest)
);

pg_try_advisory_lock returns immediately — it does not wait. If another instance holds the lock, you get false, you skip the run, and you move on. That skip-not-wait behaviour is the point: a scheduled job should never queue up behind a slow copy of itself, because the delayed runs would just pile up and fire back-to-back later. One detail worth knowing: a session-level advisory lock is held until the connection closes or you release it, so take it and release it on the same checked-out connection, and never use it through a transaction pooler that hands the session to somebody else afterwards.

If the job runs on more than Postgres — or on infrastructure where a Redis is already there — the same shape works with a Redis key, with one extra rule: the lock has to expire on its own. A Postgres lock releases itself when the connection dies. A Redis key does not know the process that set it just crashed, so a lock without an expiry is a job that stops running forever after one bad deploy. Set a TTL of roughly two to three times the job's worst realistic runtime, and treat a job that actually takes that long as a performance problem worth an alert of its own.

Two honest limits. A lock stops two copies from running at the same time. It does not stop the same job from being triggered twice through another path — a scheduler retry after a network blip, or someone firing the job's endpoint by hand during an incident. That is what idempotency is for: make the work itself safe to repeat, so a second run is harmless instead of a second charge. I wrote about that one last week, and the two ideas belong together — the lock is the gate, idempotency is the backstop. And a lock only works if every code path that does the work takes the same lock. If one path locks "digest for shop 12" and another locks "digest", you still get duplicates; name the thing you are protecting, and use that exact name everywhere.

The rule I use now: a scheduled job that touches shared state gets a lock at the top, a skip when the lock is held, and a log line every time it skips — because the day it starts skipping is the day you learn the job got slow, and that is information you want before the shop owner tells you about the duplicate email.