My temporary folder was 63 GB of RAM. It killed my own assistant three times before I checked what it really was.
On this box, /tmp is not a disk. It's a tmpfs — a folder backed by memory — and it reports 63 GB free while borrowing from the same 16 GB the whole machine runs on. It looks like a filesystem path. It behaves like one. It is not one.
Back in August, my video pipeline wrote narration WAVs and assembly files there. Large temp files, render after render, and memory pressure climbed. Three times in a single day, the out-of-memory killer reached up and took down the agent gateway — the process that runs my crons, my schedules, my whole night crew. Every automation went dark with it. I restarted. It died again. Only on the third pass did I think to ask what /tmp actually was.
The mistake was mine, not the machine's. I treated a path as a path. I never checked the filesystem type under a folder that conveniently reported 63 GB free — the very number that should have smelled wrong on a 16 GB host.
This week I rebuilt the last pipeline still writing heavy temp files — the narration and assembly stages of the digest channel — to write to project-local disk instead. One small change per script; the audit took an afternoon. And I wrote the rule down so it outlives my memory: heavy temp work goes to disk, never to RAM. The numbers: three gateway deaths in a day, every scheduled job down until a manual restart, one production video delayed. The money cost: zero. The trust cost: more, because my own automation had to be watched before I could watch it.
Last issue I trusted a version number. This issue I trusted a folder. Check the substrate: type mount, read the filesystem type, and never let a reported size tell you where your temp files actually live. On containers and cloud boxes, temporary often means memory you will miss when it is gone. When a number looks too good to be true on a box you know the limits of, it is a clue about the architecture, not a free lunch.
Tools I leaned on: Hermes (the agent gateway that kept dying), ffmpeg (the RAM hog that taught me the lesson), cron+agent (the schedule that went silent when the gateway died).
Next week: the quarter so far — what the numbers say about what stays and what gets cut.