Category Theory: A Principled Framework for AI Computation

Key Quotes

"When you change a single digit in a long string of numbers, the pattern breaks because the model lacks the internal "machinery" to perform a simple carry operation. It either chokes and makes up some nonsense or it says it's one with a bunch of zeros anyway like it definitely can't add in the in the the basic way that we know how to do algorithmically that humans learn."

Andrew Dudzik explains that large language models fail at basic arithmetic like addition because they rely on pattern recognition rather than true computational understanding. This limitation means they cannot reliably perform operations that require internal "machinery" for steps like carrying over numbers. The author highlights this as a fundamental flaw in current AI architectures.

"deep learning is currently in its "alchemy" phase--we have powerful results, but we lack a unifying theory. Category Theory is proposed as the framework to move AI from trial-and-error to principled engineering."

The text posits that deep learning is currently in an "alchemy" phase, characterized by empirical success without a foundational theoretical understanding. Category Theory is presented as a potential unifying framework that could transition AI development from an experimental, trial-and-error approach to a more rigorous, principled engineering discipline.

"To make Category Theory accessible, the guests use brilliant analogies--like thinking of matrices as magnets with colors that only snap together when the types match. This "partial compositionality" is the secret to building more complex internal reasoning."

The guests utilize analogies, such as colored magnets, to explain Category Theory. This analogy illustrates "partial compositionality," where elements can only combine if their "colors" or types match. The author suggests this concept is crucial for enabling more sophisticated internal reasoning capabilities in AI models.

"We looked at geometric deep learning from a group symmetry point of view which is a very nice way to describe spatial regularities and spatial symmetries but it's not necessarily the best way to talk about say invariants of generic computation which you would find in algorithms. However, it is something that perhaps we could express more nicely using the language of category theory."

The text notes that while geometric deep learning effectively uses group theory to describe spatial symmetries, it falls short when addressing the invariants found in general algorithmic computations. Category Theory is proposed as a more suitable mathematical language for expressing these broader computational concepts.

"Category theory is very much in the eye of the beholder... in the first instance for me category theory categories are a very mundane thing from pure mathematics where I come from and category theory means you know when you study categories for their own sake but everybody uses categories the question is what exactly are they and I really come from algebra and a lot of my motivation comes through studying algebra and one way you can think about categories is algebra with colors."

The speaker describes Category Theory as a fundamental concept from pure mathematics, akin to "algebra with colors." This analogy suggests that categories provide a structured way to understand mathematical relationships, particularly when dealing with elements that have specific matching criteria, similar to how matrices with matching dimensions can be multiplied.

"There is a historical analogy worth keeping in mind before the periodic table--before we understood protons and electrons practitioners of alchemy made real advances but without a principled foundation deep learning today may be in a similar position we have powerful empirical results but we lack the fundamental theory that would let us derive new architectures rather than just stumbling upon them categorical deep learning is an attempt to find that periodic table for neural networks."

The text draws a parallel between the current state of deep learning and the historical practice of alchemy. Just as alchemists achieved results without a scientific foundation, deep learning has produced powerful outcomes without a unifying theory. Categorical deep learning is presented as an effort to establish this foundational "periodic table" for neural networks, enabling principled architecture design.

"The problem with geometric deep learning is that as I said it talks about symmetries so permutations or circular shifts those are generally things that have very specific and rigid behaviors in typically one of the things we assume about symmetries is that they are invertible so basically that whenever I permute nodes I can always permute them back I haven't lost any information... Now why is this a problem for me who is really interested in aligning models to classical algorithmic computation well as any computer scientist will know many programs you write will delete some of the data or destroy some of the data so that is no longer a symmetry you cannot invert it."

The author points out a limitation of geometric deep learning: its reliance on invertible symmetries. This is problematic because many classical algorithms involve computations that inherently destroy or alter data, making them non-invertible. Category Theory, by accommodating non-invertible operations, offers a more suitable framework for modeling such algorithmic processes.

"The claim is quite straightforward at the end of the day deep learning has two languages constraints and implementation and we lack a single framework that cleanly links them together categorical deep learning produces the bridge right using a universal algebra in a two category of parametric maps it recovers geometric deep learning as a special case while naturally expressing things like recursion weight tying and non invertible computation."

The central argument is that deep learning currently operates with separate "languages" for constraints and implementation, lacking a unifying framework. Categorical deep learning aims to bridge this gap by providing a theoretical structure that links these two aspects. This framework, by using universal algebra within a two-category of parametric maps, can encompass geometric deep learning and naturally express concepts like recursion and non-invertible computations.