Blockchain and Verifiable AI for Personal Autonomy and Trust

Key Quotes

"there's biases that go into it and like they are changing how we see the reality depending on this and so the point here is like whoever controls that effectively controls how you perceive the information how do you make decisions and censorship from which information you can see and so money obviously is like a one of the core primitives of society but information is becoming more and more important and more valuable than even money can be and more powerful"

Illia Polosukhin argues that control over information, influenced by AI biases, grants significant power. He highlights that information is becoming more valuable and influential than money, suggesting that those who control AI's perception of reality effectively control decision-making and access to information.

"it's all about how to ensure you own your ai your ai should be working on your side ensuring that your well being and success are accounted for"

Illia Polosukhin emphasizes the importance of personal AI ownership. He believes that AI should be aligned with individual interests, acting as a personal assistant that prioritizes the user's well-being and success.

"the challenge was like if you use this neural network method back in 2014 2015 2016 they were too slow right because there was this method called recurrent neural network and the way to think about it it's how we read right it reads one word at a time and so you give it you know 10 articles from search results it will read one word at a time and somewhere two minutes later it will try to answer the question right but you know if you're at google nobody wants to wait for two minutes for it to read the text right it wants to answer right away"

Illia Polosukhin explains a key technical challenge in early AI development: the slowness of recurrent neural networks. He likens their sequential processing to reading word-by-word, which was too slow for real-time applications like Google Search, where immediate responses are critical.

"what if we dropped the recurrence what if we doesn't read every word at one time sequentially what if it just reads everything in parallel and then tries to make sense of it over you know few layers of kind of processing and this is again this is goes back to nvidia gpus became pretty kind of available and i was like hey we have this massive parallel computing and when we're reading one word at a time we're utilizing it for like 10 right there's like 90 of gpu just sitting there and not being used"

Illia Polosukhin describes the conceptual shift that led to the transformer architecture. He explains the idea of processing text in parallel, leveraging the power of GPUs, rather than sequentially, which was a significant departure from previous methods and unlocked greater computational efficiency.

"transformers kind of treats text in parallel it has this mechanism of attention where every word effectively looks at all the words around it and tries to make sense of itself in within the context then you do another step of transformers you do that again and again and again and so what happens is inside it effectively builds the relationship map between every word and everything around it but not in a one to one way but in this like multi hop way because every layer of this transformation effectively adds another hop of reasoning to this uh mental representation"

Illia Polosukhin details the core mechanism of transformers: attention. He explains that this allows each word to consider all other words in the context, building a complex relationship map through multiple processing layers, which enables a deeper understanding of text.

"the reality is it's like where you are in the ecosystem is always changing and similar right now right google brain merged into deepmind and so things things are always changing well but just take for example google research i think very few people probably appreciate how i think it's it's wildly unique in the history of organizations that google has prioritized research as a core function"

Illia Polosukhin notes the dynamic nature of organizational structures within large tech companies like Google. He emphasizes that Google's unique commitment to research as a core function, distinct from typical product development, is often underestimated.

"what blockchain introduced is actually an ownership that's not relies on them kind of violence it relies on code it relies on effectively a social consensus around everybody agreeing that this is the rules and that is like you know as an engineer as a nerd it's a very like fundamental like hey okay well this is like really interesting and really new"

Illia Polosukhin explains the fundamental innovation of blockchain regarding ownership. He contrasts it with traditional ownership, which relies on governmental enforcement and potential violence, highlighting that blockchain's ownership is based on code and social consensus, which he finds fundamentally compelling as an engineer.

"the deeper understanding you have the better you can use the tool and he was of course referring to engineers who can have the context and architecture of a system you know far that far surpasses a model's kind of context window but i think that's actually a principle that we have heard is probably one of the most recurring principles on this show from all of our conversations the deeper understanding you have the better possibilities you can experience in collaboration with ai"

The podcast hosts emphasize that deeper human understanding enhances AI collaboration. They argue that individuals with a profound grasp of a system's architecture and context, exceeding an AI's limitations, can achieve superior outcomes when working with AI tools.