Modern CLIs Evolve with Object Command Model and API Integration
TL;DR
- Modern CLIs leverage the object-command pattern, organizing functionality around nouns and verbs, which enables robust code isolation and reuse, unlike traditional CLIs that can lead to code duplication.
- The rise of modern CLIs is driven by their integration with APIs, shifting focus from local OS functions to networked communication, demanding designers consider remote interactions in their workflows.
- Modern CLIs should offer JSON output as a standard option to enhance interoperability, especially with environments like PowerShell, which treats output as objects rather than just text streams.
- Git's innovations in managing local and remote repositories at scale, handling millions of users without data loss, set a precedent for modern CLIs designed for distributed collaboration.
- CLIs should not act as credential managers; instead, they should integrate with dedicated secure systems, adhering to the Unix philosophy of doing one thing well and playing nicely with others.
- Autocompletion in CLIs is crucial for user experience and LLM discoverability, with frameworks like Cobra baking this functionality in, reducing developer effort and increasing accessibility.
- Future CLIs will need to handle non-deterministic returns from LLM interactions and capture detailed audit logs for regulatory compliance, requiring upfront design to manage increased complexity.
Deep Dive
Command-line interfaces (CLIs) are experiencing a significant resurgence, evolving from niche tools for power users to integral components of modern development workflows. This evolution is driven by innovations in command modeling, API integration, and language ecosystems, making CLIs more powerful, accessible, and essential for productivity across diverse applications, from software development to AI tooling.
The historical trajectory of CLIs reveals a pattern of rise, fall, and resurgence. Initially foundational, CLIs faced a decline with the advent of GUIs in the 1990s, only to re-emerge with renewed sophistication. Modern CLIs, unlike their predecessors, are characterized by the "object command model," popularized by tools like Git and WordPress CLI (wp). This model organizes commands around nouns (objects) and verbs (actions), creating a consistent and scalable structure, as seen in wp plugin list. This contrasts with older, more traditional monolithic commands like user add or user delete, which often lead to code duplication and management challenges. The object command model, often referred to as a command and sub-command pattern, allows for complex interfaces to be built with isolated code, facilitating reuse and robust organization. This structure is further enhanced by built-in self-documentation features, providing users with immediate access to available options and configurations directly within the command line, reducing reliance on external documentation.
The utility of modern CLIs has expanded dramatically with their integration into API-first architectures. Instead of merely processing local data streams via standard input/output, CLIs now frequently interact with remote APIs, enabling them to orchestrate complex distributed computing tasks. This shift necessitates careful consideration of long-running tasks, which require robust design patterns like chunking data, leveraging multi-core processing, efficient memory management, and implementing breakpoints and retries for fault tolerance, similar to how rsync operates. For tasks that depend on external provider execution times, CLIs must employ non-blocking wait mechanisms, often using concurrency primitives like goroutines and wait groups in languages like Go, to manage multiple asynchronous operations and track their completion without halting the user's interaction.
The development of modern CLIs is significantly influenced by the choice of implementation language and its associated ecosystem. While traditional CLIs were often written in C, newer, popular tools are increasingly built with languages like Go and Rust, which offer strong system programming capabilities with reduced overhead compared to C/C++. Go, in particular, provides robust libraries such as pflags for argument parsing and cobra for implementing the object command model. These libraries simplify the creation of sophisticated CLIs, including features like autocompletion, which is crucial not only for user experience but also for enabling AI-driven tools to discover and utilize CLI functionalities. Output formatting is another critical aspect, with JSON emerging as a de facto standard for machine-readable output, facilitating interoperability across different systems and scripting environments, especially in platforms like PowerShell that natively handle object-based data.
Looking ahead, CLIs are poised to integrate more deeply with AI and advanced security protocols. The ability of CLIs to be "crawled" by LLMs, leveraging their self-documentation, will become increasingly important for discoverability and AI-driven automation. Furthermore, handling non-deterministic returns from AI interactions and ensuring regulatory compliance through detailed logging of actions, authorization, and execution context will add new layers of complexity. Authentication and state management remain critical challenges; while stateful CLIs can offer convenience, like AWS CLI's short-duration tokens, the Unix philosophy encourages CLIs to avoid becoming credential managers themselves, instead playing well with dedicated credentialing systems. The future likely holds a balance between CLIs and custom APIs for AI interaction, with longevity favoring CLIs that adhere to the Unix philosophy of simplicity and interoperability.
Derick Schaefer's book, "CLI: A Practical Guide to Creating Modern Command-Line Interfaces," is available at moderncli.dev. He can also be found on LinkedIn and GitHub.
Action Items
- Audit authentication flow: Implement checks for common vulnerabilities (e.g., injection, XSS, CSRF) across 5-10 critical endpoints.
- Create runbook template: Define 5 essential sections (setup, failures, rollback, monitoring) to standardize operational documentation and prevent knowledge silos.
- Design CLI output strategy: Ensure JSON output is an option for all new CLIs to improve interoperability, especially with PowerShell and LLM consumption.
- Implement autocompletion: Provide shell autocompletion files for CLIs to enhance user experience and facilitate LLM discovery of CLI capabilities.
- Develop state management guidelines: For CLIs handling sensitive configuration or credentials, establish clear rules for state persistence and integration with dedicated credential management systems.
Key Quotes
"A command line interface is an interface into a computing environment right that is predicated on a terminal and generally a shell and you know the terminal gives access to the computer the shell enables both scripts and programs and these programs or these clis are literally built to do discrete pieces of work inside of an operating system whether it be at the operating system level or with a business application."
Derick Schaefer explains that a CLI is fundamentally a gateway into a computing environment, accessed via a terminal and shell. This interface allows users to execute discrete tasks within an operating system or application, distinguishing it from the underlying commands themselves.
"So historically we went from punch cards as the interface to invoke functionality and commands to a terminal which was a gateway into a shell that enabled access to scripting environments and commands and then obviously in the modern world we use graphical gateways such as applications and browser based apps and mobile apps."
Schaefer traces the evolution of user interfaces, starting from punch cards to terminals and shells, and finally to modern graphical applications. This historical perspective highlights how the CLI, through the terminal and shell, served as a crucial intermediary for interacting with computing systems before the widespread adoption of GUIs.
"So first of all if you think about scale the original cil mantra was within side of an operating system and it might venture its way something such as r sync to where it's going to communicate between another machine but when you look at git we are talking about local and remote connectivity and the ability to manage both sides of the repository equation at scale across hundreds of millions of users it really thought this out in a way that though people will run into their challenges with their repo you don't see the challenges of repository management of the past to where code is lost and documents are you know overwritten you know they really got that right at scale and i think that that was a very modern forethought on behalf of the designers and and the maintainers today is to make this work at scale."
Derick Schaefer highlights Git's innovation in handling scale and remote connectivity, a significant advancement over earlier CLIs. He points out that Git's design effectively manages repositories across millions of users, preventing common issues like lost code or overwritten documents, which demonstrates a modern approach to large-scale distributed systems.
"So the modern command line interface finds itself more organized around making these remote calls versus calling an operating system function or versus calling a function from a local package that might be compiled into it though that's still fair game for the function of a cli we see a lot more remote communication with apis and kind of a more modern sense and to be honest any cli designer really needs to think about leveraging once again the unix philosophy do something well and play well with others and part of that play well with others has extended from standard in standard out to being able to make an api call out and receive data back in as a part of that work stream if that makes sense."
Schaefer explains that modern CLIs are increasingly focused on making remote API calls rather than just local system functions. He emphasizes that this shift aligns with the Unix philosophy of doing one thing well and playing well with others, extending this principle to include network communication and data exchange with APIs.
"So first of all you want to shorten it as much as possible in a responsible manner and then to the extent that it's going to have to continue to be longer or run over periods of time that might be subject to interruptions or failures you're going to want to go back to tried and true design tactics such as is having breakpoints recovery points being able to do retries and not lose that work that has been in process."
Derick Schaefer advises that for long-running tasks in CLIs, the primary goal is to shorten their duration responsibly. He stresses the importance of incorporating robust design tactics like breakpoints, recovery points, and retries to ensure that work in progress is not lost, even if interruptions or failures occur.
"So first of all huge fan of json and i do think that any modern cli should have json as an output option not as the default but as an output option I'm going to throw a twist into the conversation that in the unix or linux world the expectation has always been exiting an exit code and that your data is going to be in the you know standard out or standard error pipes in the powershell world that changes powershell returns an object a json object and though it sets an exit code an administrator has to go look for that so if you write a cli today that does not support json you are immediately handicapping yourself in terms of your interoperability in the powershell world."
Schaefer strongly advocates for JSON as an output option in modern CLIs, though not necessarily as the default. He explains that while traditional Unix/Linux CLIs rely on exit codes and standard output/error, PowerShell's object-based return, including JSON, necessitates JSON support for effective interoperability in that environment.
Resources
External Resources
Books
- "CLI: A Practical Guide to Creating Modern Command Line Interfaces" by Derick Schaefer - Mentioned as the primary subject of the discussion and the source of the author's expertise.
People
- Derick Schaefer - Author of "CLI: A Practical Guide to Creating Modern Command Line Interfaces" and guest on the podcast.
- Robert Blumen - Host of the podcast.
- Linus Torvalds - Introduced Git into the ecosystem.
- Steve Francia - Developed the PFLAGS and Cobra packages for Go.
Organizations & Institutions
- Microsoft - Derick Schaefer's former employer.
- GitHub - Developed the GH CLI and the Git version control system.
- IEEE Computer Society - Sponsor of Software Engineering Radio.
- IEEE Software Magazine - Sponsor of Software Engineering Radio.
- WordPress - Ecosystem featuring the WP-CLI.
- Google - Holds a patent on the "Swiss Table" data structure.
- Splunk - A platform for logging and analytics.
- Loggly - A platform for logging and analytics.
- Datadog - A platform for logging and analytics.
Tools & Software
- Git - Version control system discussed for its innovations in the CLI space.
- GH (GitHub CLI) - GitHub's command-line interface, highlighted for its design and scale.
- Docker - Command-line tool mentioned as powering productivity.
- Kubernetes - Command-line tool mentioned as powering productivity.
- AWS CLI - Amazon Web Services command-line interface, discussed in the context of authentication and statefulness.
- SSH - Protocol that solved authentication issues through public and private keys.
- MySQL - Command-line interface discussed as an example of a REPL.
- WP-CLI - WordPress's command-line interface, used as an example for the object command model.
- PFLAGS - A flag and argument package for Go.
- Cobra - An object command model package for Go.
- Hugo - A static command-line driven website editor that inspired PFLAGS and Cobra.
- Bubble Tea - A Go package for creating terminal graphics and dashboard-like interfaces.
- Table Writer - A popular Go package for outputting data into tables.
- jq - A tool for processing JSON output.
- Cloud Code - A tool that presents challenges with non-deterministic returns.
- Warp Terminal - A smart terminal written in Rust that can crawl CLIs.
Articles & Papers
- "CLI: A Practical Guide to Creating Modern Command Line Interfaces" (Book) - Mentioned as the primary subject of the discussion.
Websites & Online Resources
- se radio.net - Website for Software Engineering Radio.
- computer.org - Website for IEEE Computer Society.
- moderncli.dev - Website for Derick Schaefer's book.
- linkedin.com/in/derickshafer - Derick Schaefer's LinkedIn profile.
- github.com - Platform for code hosting and collaboration.
Other Resources
- Unix Philosophy - Guiding principle for CLIs: "do one thing well and play well with others."
- Object Command Model - A command and sub-command model for CLIs, often organized around a noun-verb or verb-noun structure.
- API First Architecture - A design approach where CLIs are organized around making remote API calls.
- REPL (Read-Eval-Print Loop) - An interactive shell environment.
- JSON - A data format recommended for CLI output for interoperability.
- Exit Code - A status indicator for command execution, traditionally used in Unix/Linux.
- Environment Variables - A method for storing credentials, discussed as a security concern.
- Credential Repositories - Systems for storing and managing credentials.
- Public and Private Keys - Used by SSH for authentication.
- Stateful CLIs - CLIs that rely on stored state to function.
- System of Record - The definitive source of truth for data.
- Autocompletion - A feature that assists users by suggesting commands and arguments.
- Non-deterministic Returns - Outputs from LLMs that can vary between calls.
- Regulatory Compliance - Requirements for logging and authorization in sensitive industries.