- "A better compressed bitset. Roaring bitmaps are compressed bitmaps. They can be hundreds of times faster."
- Used by a ton of impressive projects (Google Procella, Apache Lucene, etc)
Scientists rename human genes to stop MS Excel from misreading them as dates
Excellent post on how TikTok became what it is.
The Sorting Hat is perhaps the most curious plot device from the Harry Potter universe. Is it a metaphor for genetic determinism? Did Draco have any hope of not being a Slytherin? By sorting Draco into that house, did it shape his destiny? Is the hat a metaphor for the U.S. college admissions system, with all its known biases? Is Harry Potter, sorted into Gryffindor, a legacy admit?
Instagram is some strange hybrid mix of social and interest graph, and now it’s also a jumble of formats, with a Stories feed relegated to a top bar in the app while the more stagnant and less active original feed continues to run vertically as the default. Messaging is pushed to a separate pane and also served by a separate app. Longer form videos bounce you to Instagram TV, which is just an app for videos that exceed some time limit, I guess? And soon, perhaps commerce will be jammed in somehow? Meanwhile, they have a Discover tab, or whatever it is called, which seems like it could be the default tab if they wanted to take a more interest-based approach like TikTok. But they seem to have punted on making any hard decisions for so long now that the app is just a Frankenstein of feeds and formats and functions spread across a somewhat confused constellation of apps.
See tweet section for more Chrisopher Alexander
By Tristan Hume
Fun series of lectures I've been watching when I have some free time and feel like thinking a bit but not too hard. Companion website
- adversarial examples often transfer between models
- leverage three key ideas:
- "we use the recent black box adversarial example construction techniques that create adversarial examples for a target model without access to the model’s architecture or parameters"
- "we adapt machine learning models to mimic the initial visual processing of humans, making it more likely that adversarial examples will transfer from the model to a human observer"
- "we evaluate classification decisions of human observers in a time-limited setting, so that even subtle effects on human perception are detectable"
- "We find that adversarial examples that transfer across computer vision models do successfully influence the perception of human observers, thus uncovering a new class of illusions that are shared between computer vision models and the human brain"
- Adversarial examples that transfer between models also affect human response time and accuracy
- Terry Tao story about his poor study habits through his second year of grad school, almost failed general exams, and learned better habits as a result.
- Write-up of his generals
"We build and maintain an open repository of web crawl data that can be accessed and analyzed by anyone."
T5: Text-To-Text Transfer Transformer
Apparently neopanamax and post-neopanamax are ship size standards.
Abstract: The method consists in extracting of soluble substances in dried tea leaf, containing in a porcelain or earthenware pot, by means of freshly boiling water, pouring of the liquor into a white porcelain or earthenware bowl, examination of the organoleptic properties of the infused leaf, and of the liquor with or without milk or both.
- reddit (video -- Daniel Selsam presentation)
- HN
- training data
- define a distribution SR(n): problems come in pairs: differ by negating one literal in one clause, one SAT and one UNSAT. n variables
- prevent superficial analysis differentiating the two
- network architecture
- SAT problems as character strings
- SAT problems are permutation invariant
- also negation invariant
- don't want model to have to learn these invariants
- build a graph representing the problem
- maintain vector embedding at each node
- pass messages along the edges of the graph
- iterative
- 85% accuracy on SR(40)
- can read off a satisfying assignment from the literal votes
- two-clustering recovers satisfying assignment
- solves more and more problems over time (almost 100% by 1000 rounds)
- never becomes confident in unsat (when training on SR(40))
- author thinks it's synthesized a local search algorithm
- "NeuroSAT (trained on SR(U(10, 40))) can find satisfying assignments but is not helpful in constructing proofs of unsatisfiability. When it runs on an unsatisfiable problem, it keeps searching
for a satisfying assignment indefinitely and non-systematically"
- Abram Demski, Sam Eisenstat, Scott Garrabrant
- Naively updating a probability distribution can lead to "trollability" -- the ability to push beliefs arbitrarily high or low by feeding it true / false observations.
- The distinction is about updating on
A -> B
vs observe[A -> B]
- "Logical induction (which is untrollable but not exactly a Bayesian probability distribution) is still the gold standard for logical uncertainty
- "Just as biology uses nature and nurture cooperatively, we reject the false choice between “hand-engineering” and “end-to-end” learning, and instead advocate for an approach which benefits from their complementary strengths."
- combinatorial generalization: constructing new inferences, predictions, and behaviors from known building blocks
- "important critiques have highlighted key challenges it faces in complex language and scene understanding, reasoning about structured data, transferring learning beyond the training conditions, and learning from small amounts of experience. These challenges demand combinatorial generalization, and so it is perhaps not surprising that an approach which eschews compositionality and explicit structure struggles to meet them"
- https://github.com/deepmind/graph_nets
- (2d) square grids don't contain any regular polygons (except for squares)
- this is easy to show by a shrink proof
- 3d (and higher dimensions) cubical grid has only equilateral triangles, squares, and regular hexagons
Tweet section