Polymath Engineer Weekly #92
AI Mass Assassination, Kafka, Quant Developers, Columnar Storage, Option Pricing, Processor State Machine and Breathing
Hello again.
Comic of the week
Links of the week
‘Lavender’: The AI machine directing Israel’s bombing spree in Gaza
The Israeli army has marked tens of thousands of Gazans as suspects for assassination, using an AI targeting system with little human oversight and a permissive policy for casualties.
“At 5 a.m., [the air force] would come and bomb all the houses that we had marked,” B. said. “We took out thousands of people. We didn’t go through them one by one — we put everything into automated systems, and as soon as one of [the marked individuals] was at home, he immediately became a target. We bombed him and his house.”
The architecture of Kora is one of a light proxy layer over a fast fault-tolerant cache that offloads data asynchronously to cloud object storage as the primary data store. As I described in the series introduction, cloud object storage presents both a large opportunity for reducing storage costs but also comes with the significant downside of high request latency and an economic model that punishes small requests
Quant trading's most lucrative programming language (you never heard of it)
In today's video I interview Jeremy Lucid. Jeremy programs in one of the most niche and lucrative programming languages in the world of quantitative trading. While most have never heard of it, all large banks and cutting-edge financial institutions are demanding ever larger amounts of developers proficient in this language (despite their continued shortage).
The columnar roadmap: Apache Parquet and Apache Arrow
Vertical integration from storage to execution greatly improves the latency of accessing data by pushing projections and filters to the storage layer, reducing time spent in IO reading from disk, as well as CPU time spent decompressing and decoding. Standards like Arrow and Parquet make this integration even more valuable as data can now cross system boundaries without incurring costly translation. Cross-system programming using languages such as Spark, Python, or SQL can becomes as fast as native internal performance.
Comparing option pricing methods in q
Within the financial industry, there is a need to price complex financial instruments. Despite this need, there are a lack of analytical solutions to do so. MC methods are used with the financial industry to mimic the uncertainty associated with the underlying price of an instrument and to subsequently generate a value based on the possible underlying input values. One example of where MC is used in finance, is in evaluating an option on an equity.
Talking to memory: Inside the Intel 8088 processor's bus interface state machine
The bus state machine is an example of how the 8088's design consists of complications on top of complications. While the four-state bus cycle seems straightforward at first, it gets more complicated due to prefetching, wait states, the
HALTinstruction, and the bus hold feature, not to mention the interactions between these features. While there were good motivations behind these features, they made the processor considerably more complicated. Looking at the internals of the 8088 gives me a better understanding of why simple RISC processors became popular.
I have created a survey to get feedback from you. It takes only 2 minutes.
Book of the Week
Do you have any more links our community should read? Feel free to post them on the comments.
Have a nice week. 😉
Have you read last week's post? Check the archive.