Event Sourcing Reframes Software Development Beyond CRUD Illusions

The following blog post is an analysis of the SE Radio episode featuring Martin Dilger on Event Sourcing. It synthesizes his core arguments, applying consequence mapping and systems thinking to highlight non-obvious implications for software development. The insights presented are derived solely from the provided transcript.

The hidden cost of immediate solutions and the power of delayed payoff: How event sourcing reframes software development.

This conversation with Martin Dilger reveals a fundamental misunderstanding of how software systems evolve and the hidden costs associated with conventional approaches. While many teams default to CRUD architectures, Dilger argues that event sourcing, when combined with event modeling and slice-based architectures, offers a more robust and ultimately simpler path to building complex information systems. The non-obvious implication is that what appears more difficult upfront--embracing a historical view of data and focusing on business processes--leads to dramatically reduced development costs, increased agility, and a more resilient system over time. This analysis is crucial for engineering leaders and architects seeking to break free from the exponential cost growth of traditional development and gain a significant competitive advantage through a more principled approach to system design. By understanding the cascading effects of architectural choices, teams can move from reactive problem-solving to proactive, sustainable development.

The Illusion of Simplicity: Why CRUD Becomes a Costly Trap

The default approach to software development for many teams is a CRUD (Create, Read, Update, Delete) architecture, often centered around relational databases. This path seems intuitive because it mirrors a perceived "current state" of data. However, Martin Dilger argues that this apparent simplicity is a dangerous illusion. By storing only the current state and discarding the history of changes, these systems lose critical context. This loss of historical data means that understanding how a system arrived at its current state becomes incredibly difficult, if not impossible. This lack of historical perspective is not merely an academic concern; it directly translates into tangible problems down the line.

Consider a typical user registration process. In a CRUD system, a users table might be updated. If details change, the existing record is overwritten. What’s lost is the sequence of events: when the user registered, who registered them, how their information was updated, and by whom. This historical "gold" is intentionally discarded. Dilger highlights that this leads to significant coupling across the system, as multiple use cases often depend on the same, monolithic data tables. Changing one aspect of a user’s data might inadvertently break functionality in seemingly unrelated parts of the application, forcing extensive retesting and increasing development time.

"The longer you work on those projects, the more coupling accumulates if you don't invest heavily to get rid of that. So if you don't do constant refactorings and keep the coupling very contained, coupling just grows in the system. And this means those systems, they get slower in terms of development speed, and they get more expensive."

This escalating complexity and cost are a direct consequence of prioritizing immediate data representation over historical truth. The "simplicity" of CRUD is a short-term gain that leads to long-term pain, manifesting as slower development cycles, increased bug fixing efforts, and a brittle architecture that resists change.

Event Sourcing: Embracing Time for Clarity and Agility

Event sourcing fundamentally shifts this paradigm by treating every change as an immutable event. Instead of storing the current state, the system stores a sequence of events that represent every action that has ever occurred. Dilger emphasizes that this is not just a technical storage difference; it’s a shift in perspective that aligns with how humans naturally think about processes. Businesses operate on timelines of events: "this happened, then that happened, then the user was registered." Event sourcing directly maps to this natural way of thinking, making systems more understandable and easier to model collaboratively.

The core of event sourcing, as Dilger explains, can be broken down into two primary types of "slices": state-change slices (how information gets into the system, typically via commands leading to events) and state-fuse slices (how existing events are used to present information, often via projections or read models). This slice-based architecture, combined with event modeling, allows complex systems to be broken down into small, independent, and manageable units.

This approach offers a powerful competitive advantage. Because each slice is independent, adding new features often means adding new slices without modifying existing ones. This creates a flat cost curve for development, meaning that adding the 50th feature is not exponentially more expensive than adding the 5th. This contrasts sharply with the "hockey stick" cost curve of traditional systems where complexity and dependencies cause development costs to skyrocket over time.

"No matter what requirements you add to the system, you just add new slices. It's almost like an append-only architecture, just adding new slices, which means since those slices are independent from each other, you don't touch them. They almost behave like small, tiny little microservices within your system."

This architectural resilience and predictable cost structure are significant advantages, especially in rapidly evolving markets. Furthermore, Dilger points out that this architecture is highly conducive to integrating AI coding agents, as they can operate on small, self-contained slices without needing to understand the entire codebase, reducing the complexity for AI interactions.

The Unlearning Curve: Overcoming Conventional Wisdom

Adopting event sourcing requires a significant mindset shift, which Dilger describes as "unlearning." Many established "best practices" in traditional development, such as aggressive code reuse to avoid repetition (DRY - Don't Repeat Yourself), can actually lead to undesirable coupling in an event-sourced, slice-based architecture. In this context, copying code or data between independent slices is not necessarily a bad thing; it can be a deliberate choice to maintain slice autonomy.

"So the whole DRY principles, don't repeat yourself, for most developers, that means, oh, I, I have functionality implemented in one part of the system, and I will just reuse it from everywhere. But this creates a lot of coupling. And in a slice-based architecture, that doesn't work, because you want those slices to be independent."

This requires developers to re-evaluate deeply ingrained habits. The fear of repetition must be overcome by the understanding that independent slices are paramount for agility and maintainability. The use of AI can further aid this process, as it can help manage the implications of such "copy-pasting" by identifying necessary adjustments across slices when logic changes.

The adoption process, therefore, is less about mastering new technologies and more about cultivating a new way of thinking. Dilger strongly advocates starting with collaborative modeling techniques like event modeling, which help teams establish a shared understanding of business processes before diving into technical implementation. This collaborative approach, he notes, is where most project problems truly lie--in communication breakdowns, not technology. By focusing on understanding business processes first, teams can naturally transition to event sourcing, building small proofs of concept and incrementally modernizing legacy systems or developing new ones with a predictable, flat cost trajectory.

• Immediate Action: Embrace Collaborative Modeling

  • Initiate event modeling workshops to map core business processes. Focus on understanding what happens and when, not how it will be technically implemented initially.
  • Time Horizon: Begin immediately. This foundational step requires no new technology.

• Short-Term Investment: Build a Proof of Concept (PoC)

  • Select a small, well-understood business process or workflow from a legacy system or for a new feature.
  • Implement this process using event sourcing and a slice-based architecture. This could be a single microservice or a component within an existing system.
  • Time Horizon: 1-2 weeks. The goal is to gain practical experience and validate the approach.

• Medium-Term Investment: Develop Slice Cycle Time Measurement

  • Begin tracking "slice cycle time"--the duration it takes a team to build a complete, production-ready slice (including UI, events, projections, and tests).
  • Time Horizon: Over the next quarter. The aim is to observe a reduction in this metric as teams gain experience.

• Medium-Term Action: Foster "Unlearning" through Training and Practice

  • Encourage developers to question traditional assumptions about code reuse (DRY) and embrace controlled repetition (copying) for slice independence.
  • Time Horizon: Ongoing. This is a cultural and skill-development shift.

• Long-Term Investment: Incremental Modernization/Development

  • Continue to identify and extract business processes as independent slices, either modernizing legacy components or building new functionality incrementally.
  • Time Horizon: 6-18 months. This allows for gradual adoption and risk mitigation.

• Immediate Action: Re-evaluate Technology Choices

  • Critically assess if current tools, particularly event streaming platforms like Kafka, are being used as event stores. If so, correct course and select appropriate event store technologies.
  • Time Horizon: Immediately address any misapplication of tools.

• Short-Term Action: Integrate AI for Development Assistance

  • Explore how AI coding agents can assist in generating code for slices, identifying dependencies, and managing adjustments across multiple slices.
  • Time Horizon: Over the next 1-3 months. Leverage AI to accelerate the adoption of slice-based development.