AI-Driven Interfaces Demand New Backend Strategies and Context Management

Key Quotes

"I actually went to school for photography and then for media studies so that's like writing about movies and video games very fun things to do but not a super employable skill that's true yeah I know I started in photography I liked doing kind of wedding shooting events kind of a technical endeavor but also a creative endeavor but ultimately I don't know it was hard to kind of run a business separating kind of the art from the business side of it somehow I got into am news radio production so kind of finding interesting stories and guests for am news talk shows so I learned a ton from this job I think like translating kind of complex ideas into interesting stories and problems that the people could get invested in and then for whatever reason I moved into content marketing so for a tech startup combined some of like visual design skills from my photo days with kind of communication from my radio days and I really kind of knew nothing about tech at that time"

Wesley Yu explains that his path into technology was not direct, involving early interests in photography and media studies before transitioning through radio production and content marketing. Yu highlights that these diverse experiences, particularly learning to translate complex ideas into engaging stories, provided a valuable foundation for his eventual role in tech. This demonstrates how seemingly unrelated fields can equip individuals with transferable skills applicable to new domains.

"I think that's changed a lot over the last 20 years I mean you mentioned the cellphone the iPhone coming out in 2007 that was a big boom that kind of changed people's expectations of how they interact with technology I think there was like a productivity app boom in like 2010s that set new expectations there is of course crypto and now there's AI so with mobile like you kind of built this expectation with consumers that there are adaptive screen sizes that you have gestural inputs like scrolling and pinch to zoom swiping and all that replacing point and click the interface became this thing that directly manipulate you can kind of drag things around expand things long press on things and then of course you also get access to like location cameras and sensors that unlock new possibilities so I think like many expectations have changed just from the mobile device itself not to mention even just like the ability to notify people you know mobile phones are in your pocket so now you start to build habit loops notifications in the applications that draw you back into the device and demand your attention in some ways that I feel like are less inviting these days and more invasive but those are kind of the concepts that have built some foundation in interface"

Wesley Yu discusses how mobile technology, particularly the iPhone, fundamentally altered user expectations for digital interfaces over the past two decades. Yu points out that features like adaptive screen sizes, gestural inputs, and access to device sensors created a more interactive and personalized user experience. He notes that this evolution has led to new expectations for how users engage with technology, including the use of notifications to build habit loops.

"I think like this change this kind of trend in technology productivity has far more of an impact in the technical implementations of of applications because now more things are stateful more things are real time with having like a lot more state and a lot more sources of state to sort of track what's the technological programmatic thing that you have to sort of adapt to handle that yeah there's lots of different strategies to resolve changes but essentially you kind of would want to track kind of individual changes and be able to merge and come back to a state that makes sense for everyone depending on the application there are different strategies to do this but that is definitely a technical challenge depending on kind of the interface you're working with"

Wesley Yu explains that the rise of productivity applications has significantly impacted the technical implementation of software by increasing the need for stateful and real-time systems. Yu highlights that tools like Google Docs and Figma necessitate tracking numerous changes from multiple users, creating a complex state management challenge. He emphasizes that resolving these changes and merging them into a coherent state for all users is a core technical hurdle in developing such applications.

"You know everyone is talking about AI today you know we had an article a while ago that the AI isn't the app it's the interface it's the UI to other applications information and I wonder if you see AI that way or you see it as another intermediary app when we think about products that innovate AI we see them as kind of two buckets one is like AI that enables businesses that couldn't exist without AI and then there is AI that is used as an accelerator or another modality to accomplish tasks that you could otherwise do manually"

Wesley Yu categorizes AI-driven products into two main groups: those that enable entirely new business models and those that accelerate existing manual tasks. Yu uses the example of an AI application for teaching children to read, which relies heavily on AI for personalized content generation and speech analysis, illustrating an AI-enabled business. He contrasts this with AI used to structure and query vast amounts of policy data, which acts as an accelerator for tasks previously done manually by users.

"I think that right now humans are best at designing that but in the future maybe we can teach an LLM to do this but I think that this type of problem is challenging to verify and so therefore challenging to teach an LLM to do correctly I think an LLM can certainly verify whether or not a function was written correctly maybe you can infer if a feature was kind of done correctly but to verify whether an application meets the needs of the consumer that's extremely hard to verify it needs to be verified with the market and I don't think that LLMs are going to get there anytime soon"

Wesley Yu expresses that humans currently excel at designing interfaces that effectively solve user problems, citing the example of a complex travel arrangement task for a reality TV show. Yu believes that humans possess a superior ability to externalize memory, manage progressive disclosure, and hide irrelevant information, which are crucial for intuitive interface design. While acknowledging the potential for LLMs to assist in this area in the future, Yu asserts that verifying an application's market fit and consumer needs remains a significant challenge for AI.