Polymath Engineer Weekly #35
Get ready for a heavy reading week
Links of the week
“This paper appeared in CIDR 2023 recently. It is a thought provoking big picture paper, coming from a great team.
The paper draws attention to the divide between conventional wisdom on building scalable OLTP databases (shared-nothing architecture) and how they are built and deployed on the cloud (shared storage architecture). There are shared nothing systems like CockroachDB, Yugabyte, and Spanner, but the overwhelming trend/volume on cloud OLTP is shared storage, and even with single writer, like AWS Aurora, Azure SQL HyperScale, PolarDB, and AlloyDB.“
“The beautiful part, though, is when you finally connect all these systems and your database is talking to your streaming platform and your model is reading from the database and you have a new model every day and, finally, one day, someone from sales will come to you and say, we just prevented a customer from churning because we offered them a hypoallergenic organic mattress based on their previous browsing behavior and we kept the customer, and then you can look back through the decision trees, to see the forest of what you have built.“
“Transaction log is an essential part of database, because all of the database management system is required not to lose any data even when a system failure occurs. It is a history log of all changes and actions in a database system so as to ensure that no data has been lost due to failures, such as a power failure, or some other server failure that causes the server crash. As the log contains sufficient information about each transaction executed already, the database server should be able to recover the database cluster by replaying changes and actions in the transaction log in case of the server crash.“
“In the preparation for the course, I was re-reading some old articles about always-valid domain models, including this one form Jeffrey Palermo and the response to it from Greg Young. I highly recommend that you read them both if you haven’t already. This post is another response to Jeffrey’s article, which will hopefully complement Greg’s one.“
“Currently, there is an astonishing amount of toil and guesswork involved in actually getting deep neural networks to work well in practice. Even worse, the actual recipes people use to get good results with deep learning are rarely documented. Papers gloss over the process that led to their final results in order to present a cleaner story, and machine learning engineers working on commercial problems rarely have time to take a step back and generalize their process. Textbooks tend to eschew practical guidance and prioritize fundamental principles, even if their authors have the necessary experience in applied work to provide useful advice.“
“Interfaces are one of the big ways of creating and defining APIs in Go. Go famously encourages these interfaces to be very minimal; the widely used and implemented io.Reader and io.Writer are each one method. Minimal APIs such as this have the advantage that almost anything can implement them, which means that Go code that accepts an io.Reader or io.Writer can work transparently with a huge range of data sources and destinations.“
Book of the Week
Do you have any more links our community should read? Feel free to post them on the comments.
Have a nice week. 😉