When you are trying to solve a hard technical problem, looking at how other people in your specific industry solved it is usually a trap. You end up inheriting their assumptions. When we set out to build a graph engine capable of serving AI agents at scale, we didn't look at traditional databases. We looked at John Carmack.
In 1993, DOOM achieved something that the rest of the software engineering world considered impossible: real-time, texture-mapped 3D rendering on consumer hardware. The processors of the era simply did not have the compute budget to render a full 3D world pixel by pixel at 35 frames per second.
Carmack didn't solve this by waiting for faster hardware or writing slightly better C code. He solved it by fundamentally rethinking the architecture of the problem. He realized that the engine didn't need to render the whole world—it only needed to render the exact polygons the player could see. And to figure out what the player could see in microseconds, he adapted a data structure from academic computer science: the Binary Space Partitioning (BSP) tree.
The Lesson of the Hot Loop
There are two profound lessons in the DOOM architecture that we consider foundational to how we built pgGraph.
The first is the sanctity of the hot loop. In a game engine, the hot loop is the code that must execute every single frame. If it pauses, the game stutters. Carmack was famously ruthless about stripping away any abstraction that didn't serve the immediate frame. He bypassed the operating system. He wrote direct-to-metal assembly. He discarded 'best practices' of object-oriented programming if they introduced even a millisecond of overhead.
AI agents have a hot loop too. It's the reasoning cycle. When an agent is trying to determine why a supply chain failed, it has to traverse relationships—from the delay, to the warehouse, to the shift manager, to the policy, to the exception. If each hop takes 50 milliseconds because a traditional graph database is translating a high-level query language into a multi-stage execution plan, the agent's hot loop collapses.
Standard databases are full of abstractions: query planners, transaction managers, buffer pools, and lock managers. These are wonderful for ensuring data integrity, but they are fatal in the hot loop. Just like Carmack bypassed the OS to write to the VGA buffer, we realized we had to bypass the Postgres query planner to walk relationships.
Memory Layout is Everything
The second lesson is that compute is rarely the bottleneck; memory access is. Carmack's use of BSP trees meant that the engine could determine visibility simply by walking a pre-computed tree structure in memory. The CPU wasn't doing complex math on the fly; it was just reading the right memory addresses incredibly fast.
When we looked at how Apache AGE and other traditional graph databases executed traversals, we saw the exact opposite. They emulated relationships using relational tables or linked lists of pointers. To traverse a single edge, the CPU had to bounce around random access memory, chasing pointers and loading entirely new cache lines just to find the next node.
We applied Carmack's philosophy directly: compute the structure ahead of time, and lay it out in memory so the CPU never has to think.
The pgGraph Translation
This is why pgGraph compiles your Postgres relationships into a Compressed Sparse Row (CSR) array. In a CSR array, all relationships are packed into contiguous blocks of memory. Finding a node's neighbors does not require a B-tree index lookup or a SQL join. It requires a single integer addition to find an array offset.
Once the agent begins a traversal, the CPU is no longer executing a database query. It is walking a tightly packed integer array in a raw Rust loop. There are no pointers. There are no garbage collection pauses. The cache hit rate is astronomically high because the data is perfectly linear.
By discarding the heavy abstractions of relational emulation and obsessing over memory layout, we achieved graph traversals at speeds that standard Postgres query planners physically cannot match. We regularly see 20-hop deep traversals execute in microseconds.
Choosing Your Abstractions
We look up to John Carmack not just because DOOM was a technical marvel, but because he proved that the right data structure beats the right hardware. He proved that if you truly understand the constraints of your system, you can build software that feels like magic to everyone else.
Traditional graph databases chose to optimize for flexibility. They wanted to answer any possible query using a beautiful, sprawling query language. In doing so, they sacrificed the hot loop.
We chose differently. We chose to build an Unreal Engine for operational data. We let Postgres handle the heavy lifting of storage and transactions, but when it comes time for an AI agent to traverse the graph, we drop down to the bare metal. Because in the agent era, latency isn't just a performance metric. It's the difference between a system that thinks, and a system that stalls.