Polymath Engineer Weekly #61
Great reads to end the week
Hello again.
Links of the week
After much experimentation, we have settled on the approach of visualizing our projects as a series of concentric layers, where each layer can depend only on further inner layers and each layer should be executable on its own.
Supavisor: Scaling Postgres to 1 Million Connections
One of the most widely-discussed shortcomings of Postgres is it's connection system. Every Postgres connection has a reasonably high memory footprint, and determining the maximum number of connections your database can handle is a bit of an art.
A common solution is connection pooling. Supabase currently offers pgbouncer which is single-threaded, making it difficult to scale. We've seen some novel ways to scale pgbouncer, but we have a few other goals in mind for our platform.
Manage your priorities and energy.
People are complex, and they get energy in complex ways. Some managers get energy from writing some software. That’s great, particularly if you avoid writing software with production dependencies. Some managers get energy from coaching others. That’s great. Some get energy from doing exploratory work. Others get energy from optimizing existing systems. That’s great, too. Some get energy from speaking at conferences. Great. Some get energy from cleaning up internal wiki’s. You get the idea: that’s great. All these things are great, not because managers should or shouldn’t program/speak at conferences/clean up wiki’s/etc, but because folks will accomplish more if you let them do some energizing work, even if some of that work itself isn’t very important.
How Google Unlocks and Measures Developer Productivity
Since 2018, another powerful measuring tool at Google has been the quarterly engineering satisfaction survey, which goes out to about a third of the engineering force at a time. Green admitted that executives were reticent about this measurement at first because it’s “just people’s opinions.” During the pandemic lockdowns of 2020, the survey first revealed an uptick in productivity, followed by a big dip the next quarter, as time at home often alone continued.
Inside Disney’s Site Reliability Engineering practice
Disney’s servers had nicknames after Snow White’s dwarves — grumpy, sleepy, and dopey to reflect server behaviors — but this amusing behavior revealed a more significant issue of difficulty staying on top of server configurations
😅 Servers began to take on the personality they were named after
Grumpy regularly showed errors
Sleepy kept suffering from high latency
Bashful would disappear from the network for days
Distributed Transactions at Scale in Amazon DynamoDB
In the TSO algorithm, Each transaction, is assigned a timestamp on start, which is taken to define its position in the serial order. Every object x is tagged with the timestamp of the last transaction that successfully read/write it. As long as transactions appear to execute at their assigned time, serializability is achieved. To realize this, if/when a transaction tries to access (read/write) an object from its future it is aborted. And, that's all folks.
Book of the Week
Escape into Meaning: Essays on Superman, Public Benches, and Other Obsessions
Do you have any more links our community should read? Feel free to post them on the comments.
Have a nice week. 😉
Have you read last week's post? Check the archive.