Farhan Ali Khan Build logs
Farhan Ali Khan · build logs4 logs · 2026

Build logs

Build logs and notes for myself on distributed systems: deterministic simulation, databases, and the guts underneath.

Contents

  1. 02Git-style branching on an LSM key-value storeStorage Engines17 Aug 2026
  2. 03Building a filesystem on a databaseFilesystems10 Aug 2026
  3. 04Who else is here? Cluster membership from first principlesDistributed Systems14 Jul 2026
  4. 05Learning DST for testing our distributed transactional KV storeDeterministic Simulation Testing9 Jun 2026

←→ turn the pages, or press a bookmark

P.02 · Storage Engines17 August 2026

Git-style branching on an LSM key-value store

How a fork, a diff and a merge are built out of one sequence number.

Agents need somewhere to work that they can break. So I started with an experiment to build a filesystem on a database (SurrealDB) so that every action could say who wrote it and be rolled back on its own. That worked, somewhat. But giving each agent its own copy of the…

Read the log ~9,225 words · 11 widgets · 35 diagrams

Written with AI assistance

Fig 00 · A branch as a ceiling on the shared commit sequence
what it is notmainits own clock, its own HEADchild-ba second, independent history✗ there is no second clock, and no per-branch headwhat it isone global commit sequence1020253040fork_seq = 25invisible to the child, permanentlythe child inheritsThe child’s own commits draw numbers from the same counter, so they land to the right of 25 — above everything itinherited.
Storage Engines
P.03 · Filesystems10 August 2026

Building a filesystem on a database

How syscalls, inodes, blocks, journals and FUSE work, and what each one becomes when the storage underneath is a database.

An agent worked a repository overnight. In the morning, one file is wrong.

Read the log ~9,892 words · 12 widgets · 9 diagrams

Written with AI assistance

Fig 00 · What each of the four structures holds
superblockthe filesystem itself:size, block size,where the tables startdirectory entryname → inode number"report.md" → 12inodemode · uid · gid · sizelink count · block pointersNO NAMEdata blockswhere the bytes actually areopen file descriptionthe cursor: offset + flagsshared across dup() and fork()names itThe name and the file are separate objects.Two directory entries may name one inode — that is a hard link. The inode counts its names,and the data is freed only when that count reaches zero.
Filesystems
P.04 · Distributed Systems14 July 2026

Who else is here? Cluster membership from first principles

Heartbeats, gossip, SWIM, Serf, Lifeguard, and Rapid — how a cluster learns who is in it, and the separate problem of changing the voter set.

Consider a cluster of three servers. One of them stops answering.

Read the log ~7,152 words · 19 widgets · 15 diagrams

Written with AI assistance

Fig 00 · Four decades of asking “who else is here?”
Demers et al.epidemics / anti-entropy1987van Renessegossip failure detection1998SWIMdetection ≠ dissemination2002φ-accrualsuspicion as a level2004FireFliesByzantine membership2006Censusconsistent, location-aware2009memberlist / Serfimplemented + extended2013Lifeguardthe accuser is the problem2017Rapidone agreed view, at scale2018Matchmakerreconfigurable consensus2021failure detectiondisseminationconsistent viewsreconfiguration
Distributed Systems
P.05 · Deterministic Simulation Testing9 June 2026

Learning DST for testing our distributed transactional KV store

Building a deterministic simulation tester.

Over the past year, we've been building our distributed transactional key-value store at SurrealDB, alongside our own embedded key-value engine, SurrealKV. Working on both systems has reinforced just how important correctness and reproducibility are for these kinds of systems.

Read the log ~5,277 words · 22 widgets · 4 diagrams

Written without AI

Fig 00 · Simulation first: the whole cluster in one process, time compressed
one OS processn0node taskn1node taskn2node tasknetwork backplanedriver + clockseed -> RNGentire cluster, one drivertime compressionwall clocksim timemonth -> hour
Deterministic Simulation Testing