The incident report usually reads the same way. The database disappeared for ninety seconds, came back empty-cached, and the error log shows nothing useful — just a restart. The evidence is one line in dmesg or the system journal:
Out of memory: Killed process 1183 (mysqld) total-vm:...
MySQL did not crash. The kernel killed it, because the host promised more memory than it had. On RDS and Aurora you see the managed equivalent: a swap spike, FreeableMemory falling to the floor, and a forced restart you did not request.
The cause is almost never the buffer pool alone. It is the buffer pool plus everything that scales with connection count, plus everything that scales with query shape — three budgets that teams size independently and then add up only after the fact.
The three budgets
1. Global, allocated once. The InnoDB buffer pool (innodb_buffer_pool_size) dominates. Alongside it: the log buffer, the adaptive hash index, the dictionary cache, and the Performance Schema, which on default settings commonly takes several hundred megabytes and is frequently forgotten entirely. These are stable; you can reason about them on a whiteboard.
2. Per connection, allocated on demand. sort_buffer_size, join_buffer_size, read_buffer_size, read_rnd_buffer_size, tmp_table_size, plus a few hundred kilobytes of thread and net buffer per session. A session allocates these only when a statement needs them, and some of them can be allocated more than once in a single statement — one join buffer per joined table that needs one, one sort buffer per sort.
3. Per statement, unbounded in practice. Internal temporary tables. In 8.0 the default internal_tmp_mem_storage_engine = TEMPTABLE draws from a shared pool capped by temptable_max_ram (1 GB by default), with overflow going to mmap'd files or to InnoDB on disk. A query plan with Using temporary; Using filesort over a wide result set can consume a surprising amount of that pool, and several concurrent copies of the same bad report query are how a healthy instance turns into a dead one at 09:05 on a Monday.
The classic sizing mistake is multiplying budget 2 by max_connections and concluding you need 400 GB. That worst case is not reachable in normal traffic — buffers are allocated per statement, not per idle connection. The opposite mistake is more dangerous: assuming budget 2 is negligible, setting sort_buffer_size to 32 MB because a blog post said bigger is faster, and then discovering what 300 concurrent sorts cost.
Measure before you tune
Start with what the server reports about itself rather than with a formula. On 8.0, memory instrumentation in the Performance Schema is already on for the major allocators:
SELECT event_name,
ROUND(current_alloc / 1024 / 1024) AS current_mb
FROM sys.x$memory_global_by_current_bytes
ORDER BY current_alloc DESC
LIMIT 15;
That ranks the actual allocations: buffer pool, Performance Schema internals, temptable, connection-level buffers. Then look at per-thread usage to find the offender during an event:
SELECT thread_id, user, current_allocated, current_statement
FROM sys.memory_by_thread_by_current_bytes
ORDER BY current_allocated DESC
LIMIT 10;
Two supporting counters tell you whether budget 3 is in play:
SHOW GLOBAL STATUS LIKE 'Created_tmp_%tables';
A high and rising Created_tmp_disk_tables relative to Created_tmp_tables means queries are spilling. The reflex is to raise tmp_table_size; the better first move is to find the queries creating temporary tables at all — statement digests sorted by SUM_CREATED_TMP_DISK_TABLES in performance_schema.events_statements_summary_by_digest name them in one query. A GROUP BY that can be served by an index creates no temporary table and needs no memory at all. That is a tuning fix with a negative cost.
On RDS and Aurora, FreeableMemory in CloudWatch is the number that matters, and the shape of its curve tells you which budget is leaking: a step down after a deploy is global, a sawtooth tracking traffic is per-connection, a cliff at a reporting hour is temporary tables.
A sizing method that holds up
For a dedicated database host, work downward from total RAM rather than upward from the buffer pool:
- Reserve the operating system's share. Roughly 1 GB plus 5% of RAM on a Linux host, more if backups, agents, or a proxy run locally. This reservation is not optional; it is what keeps the kernel from having to choose a victim.
- Reserve the connection budget. Take your observed peak concurrent active threads (
Threads_runningat peak, notThreads_connected), multiply by the sum of your per-connection buffers, and add 50% headroom. With modest buffer settings and 100 active threads this is single-digit gigabytes, not hundreds. - Reserve the temporary-table pool.
temptable_max_ramas configured, because under a bad hour you will use it. - Add the Performance Schema and dictionary. Read it from
sys.x$memory_global_by_current_bytesrather than guessing. - Give the remainder to the buffer pool — but not more than the working set needs. On a 64 GB host with a 20 GB dataset, a 45 GB buffer pool buys nothing over 24 GB, and the unused memory is more valuable as a safety margin.
The widely repeated "buffer pool = 75–80% of RAM" rule is sound on a large dedicated host with a dataset much bigger than memory and disciplined per-connection settings. It is actively dangerous on a small instance, on a shared host, or with generous sort and join buffers. Rules of thumb assume the rest of the budget is zero.
The settings most often set wrong
sort_buffer_sizeabove a few megabytes. Large values can make sorts slower, not faster, and the allocation is per sort. Leave it near default and fix the query that sorts a million rows.join_buffer_sizeraised to mask a missing index. A join buffer is the optimizer's consolation prize for a join with no usable index. The index is the fix; the buffer is a tax on every session that hits the same plan.max_connectionsraised to survive a connection storm. Each extra connection is permission to allocate more memory, and the server was already past its concurrency sweet spot. Pool in front of the database instead; we wrote about where that limit actually binds in the post on connection limits and pooling.innodb_buffer_pool_sizeset from the old host's value after a downsize. This is the single most common cause of the post-migration OOM we get called about.performance_schemaleft entirely at defaults on a small instance. You can trim consumers and sizing without turning it off; turning it off blinds you exactly when you need the data.
Guardrails worth having
Two settings turn a server-wide outage into one failed query. max_execution_time caps runaway SELECTs at the server or session level. In 8.0.28 and later, connection_memory_limit and global_connection_memory_limit let you cap memory per connection and across all connections, so a pathological statement gets an error instead of the kernel's attention. Set them generously at first, watch for a few weeks, then tighten; the goal is a ceiling nobody normally touches.
On Linux, leave a small swap configured even on a database host. It is not there to be used for performance — a MySQL that swaps actively is already in trouble — but a few gigabytes gives you a slow degradation and an alert instead of an instant kill.
What good looks like
An instance sized this way shows a flat memory curve under traffic, Created_tmp_disk_tables near zero, a buffer pool hit rate that stays high without consuming the whole host, and enough free memory at peak that a single unusual query cannot end the day. Getting there is usually a half-day of measurement and two configuration changes, not a bigger instance.
If your database restarted without a crash entry in the error log, the evidence above will say why within an hour. Send us the output of the two sys-schema queries and your current global variables, and we can usually name the budget that overflowed.