Polymath Engineer Weekly #94
Distributed Databases, Makers vs Managers, Parquet, Faster LLM Decoders, Buying a House, Illicit Meat and Cyberwarfare
Hello again.
Comic of the week
Links of the week
Deno KV internals: building a database for the modern web
We chose to build Deno KV atop of FoundationDB, Apple’s open source distributed database used in iCloud and by Snowflake, since it’s perfect for building scalable database solutions: it’s thoroughly verified with deterministic simulation, scalable and efficient, and provides the transactional key-value storage API.
Maker vs. Manager: How Your Schedule Can Make or Break You
The numerous articles about the routines of successful people miss the point. Waking early doesn’t make you a novelist. Segmenting your day doesn’t make you an entrepreneur.
The lesson isn’t what time to set your alarm, which journalling prompt to use, or how long to wait before drinking your coffee. The lesson is that different work requires different schedules.
Chapter I: The birth of Parquet
During that time, the data platform team at Twitter had a paper reading group. People would read papers and present them to the group if they were interesting. At the time I re-read the Dremel paper and presented to the group. I also read interesting papers that described how those MPP databases work: C-Store, Vertica, Monet-DB and others. This led to a better understanding of distributed query engines and the benefits of column stores.
Consistency Large Language Models: A Family of Efficient Parallel Decoders
LLMs have been traditionally regarded as sequential decoders, decoding one token after another. In this blog, we show pretrained LLMs can be easily taught to operate as efficient parallel decoders. We introduce Consistency Large Language Models (CLLMs), a new family of parallel decoders capable of reducing inference latency by efficiently decoding an n-token sequence per inference step.
The Rise of the Forever Renter Class
I am a part of what you might call the Forever Renter class. This is a cohort of people in their mid-20s to late-30s who are either unwilling or unable to buy a home under current market conditions. Unfortunately, those conditions aren’t moving in the right direction either.
Tracking Illicit Brazilian Beef from the Amazon to Your Burger
With grassroots support from labor unions and Indigenous communities, he had mapped the complex networks of cattle farms responsible for illegal deforestation. He then tracked the often-illicit beef through JBS’s slaughterhouses and packing plants to the freezers, shelves, and customer trays of retail outlets and fast-food restaurants around the world. When his sleuths were done, the fingerprints of forest destruction were plain to see. Six of Europe’s biggest retail chains reacted by halting purchases of JBS beef.
I have created a survey to get feedback from you. It takes only 2 minutes.
Book of the Week
The Art of Cyberwarfare: An Investigator's Guide to Espionage, Ransomware, and Organized Cybercrime
Do you have any more links our community should read? Feel free to post them on the comments.
Have a nice week. 😉
Have you read last week's post? Check the archive.