Sign in Subscribe
Technology

New Visions

Rewiring the operating assumptions behind computer interfaces

Brian Sholis

Brian Sholis

— 7 min read
New Visions
Illustration by Paul Twa

Housekeeping note: the bonus Good Links digest sent to paying subscribers last Friday is now available for everyone. If you appreciate what we’re publishing and have the means, please consider upgrading to a paid subscription. Your support keeps everything we publish free for everyone.

Hi everyone,

You likely don’t need me to tell you about the Vision Pro headset that Apple announced last week. But I did read a lot of responses to the presentation, and would point you to John Gruber’s description of using one; Andy Matuschak’s notes on how it enables “desktop-class computing in any workspace”; or M.G. Siegler’s discussion of how the Vision Pro shapes public awareness and what it means for other companies’ AR/VR ambitions. Yet after all that reading, I still feel ambivalent: astounded by the technological achievement and impressed by the design considerations, but conflicted about what I take to be some of its social implications.

I came away from the WWDC keynote more excited about smaller, long wished-for updates (user profiles in Safari, password sharing, collaborative playlists, offline Maps) and others I didn’t know I wanted (live voicemail, NameDrop). Unlike Vision Pro, which arrives next year and will be well beyond my budget, each is a quality-of-life improvement that I can begin using the day it’s released. And many of them promise the kinds of interoperability and customization that too often remain a pipe dream—and which can enable even greater personalization of “personal computing.” As Alexander Obenauer, an independent researcher focused on these issues, has put it in one of his regular lab notes, “It might be difficult to fully consider the ways you would be able to reflect your own true thinking or uses in a well-suited software concept because we are so used to the metaphors our software has been based on for the last 50 years.”

This combination, of a paradigm-shifting innovation and a laundry list of incremental improvements, got me thinking about that half-century persistence of the desktop and file folder interface, and of the kinds of experiments I’ve encountered that have most excited me in recent years. So I thought I’d share some with you. This is a longer post than most, but that’s because it’s a catalog of enthusiasms. Click the links liberally!

Obenaeur’s own solution is to treat every packet of data—for example, from application views to inboxes to emails to recipients—as an item that can be customized or recombined: “With provided building blocks, you could start the OS from scratch and build up any interface system you wanted. Or starting with a selected default set, you could swap components [in and] out. Or following the lead of someone with working styles similar to your own, you might use their published view definitions and hierarchy to get started.” His first experiment is One Long Log OS (OLLOS), which proceeds from the idea of keeping everything in one chronological timeline.

MercuryOS interface

Perhaps the first of these experience to catch my eye was Jason Yuan’s MercuryOS, which was released in 2019 and which holds up incredibly well. Yuan put it plainly in an article that May: “The desktop metaphor must die.” His describes his experiment, designed for people “living with ASD, ADHD, and other neurological differences who are frequently overwhelmed by the flood of sensory information,” as fluid, focused, and familiar. That is, it’s meant to respond to a person’s intentions, keep things out of the way until they’re necessary, and builds on interaction habits made common in iPadOS. This structure, coupled with an elegant design, gives the concept its staying power. (In fact, another talented design engineer, Rauno Freiberg, has spent the past few weeks creating a working prototype of it.)

Obenauer and Yuan’s work is speculative. What existing tools have caught my eye? One is Screenotate, by Omar Rizwan, a little tool that extracts text from screenshots and saves it with contextual info (essentially, where it came from), so you don’t have to re-type what’s “trapped” in the image. (Apple partially “sherlocked” this with the Live Text feature it recently added to iOS and iPadOS.) Rewind, which launched last November, “records anything [your computer has] seen, said, or heard,” arrays it on a timeline you can “rewind” through, and makes it searchable. (The wizardry is the file compression that allows you to keep all this on your hard drive as much as the “perfect memory” it affords.) In a related vein, I’m intrigued by Spacedrive, a newly announced file explorer “for creators, hoarders and the painfully disorganized.” And though it’s not publicly available, I’ll include Linus Lee’s Monocle, a “universal personal search engine,” because it’s an existing piece of software and the tool that comes closest to what I dream of having for myself: a fast, private search engine for the important digital detritus in my life: published writing, notes, emails, calendar events, contacts, photos. Each of these tools, in their own ways, try to make things more accessible—to turn a hunch or a recollection into something tangible and sharable with as little friction as possible.

I’ve written before about how I am decidedly not a programmer, just an enthusiastic early adopter who’s curious about how I can better use the machine that occupies so many of my hours. So let me acknowledge there is a whole other group of explorations that I’d describe intriguing but esoteric. Here I’d include Hundred Rabbits’s Uxn, a “little personal computing stack” that creates “portable graphical applications” and evolved from the need to minimize power usage and avoid network connections … because its creator lives off the grid, on a boat. Another is Urbit, which I tried to use in April 2021 without much success, but which is still interesting as an idea: “a new OS and peer-to-peer network” meant to be “an effective, customizable productivity tool for collaborators, and a calm noninvasive communication tool for friends and families.” (Given a recent blog post about “Using Urbit in 2023,” perhaps I wasn’t the only one who couldn’t connect with it a few years ago.) And lastly, there is Rasmus Andersson’s Playbit, an “operating system for software creatives” that “encourages playful learning, building & sharing” of said software. To be honest, I’m sharing this based on Andersson’s stellar reputation and vibes. Seriously, just go to his Twitter profile and scroll.

But why do I, a simple Mac user, keep tabs on all this stuff? Because I know enough to know that I want more—that my “personal computer” is not yet personal enough. Three weeks ago, Rizwan, who made Screenotate, published a mini manifesto (as a Google Doc!) with writer Tim Hwang. They asked: “Do you dream about your smartphone? Does it feel like a place that you can inhabit and shape and reconfigure? Does it give you a sense of possibility? Computers are a feeling, not a device.” Or, as Jacky Zhao put it in an argument for “agentic computing”: “The best pieces of software … let me fully construct a world for myself in which I operate. They set the rules, but I am the one who decides how I want to play.”

Last week, in response to Apple’s Vision Pro announcement, a well-known software entrepreneur tweeted about “two distinctly different visions of the future,” juxtaposing a text-heavy ChatGPT interaction with the (ostensibly visual) Apple headset. Yuan, who created MercuryOS, responded to this either/or stance by saying, “I literally do not understand why people feel the need to segregate language driven interfaces and object/tactile interfaces???” I take Yuan’s viewpoint seriously, both for the example of MercuryOS and because, a month earlier, he had quit his job … at Apple. Why’d he make that decision? Because he’s launched a startup named, appropriately enough, new.computer.

I share Yuan’s both/and attitude because I know that for my computer to become more personal, for it to serve my unique needs as well as those of billions of other computer users, it needs to be more flexible. That’s why I appreciated Apple’s strategic use of the open-ended term “spatial computing” (as opposed to Facebook’s—er, Meta’s—use of “metaverse.”) But, of course, everyone I’ve cited here knows the example of Bret Victor’s Dynamicland, a “physical computing” medium in which people collaboratively manipulate real objects like paper and toys to create and run programs.

This brings me back to the social implications of the Vision Pro. Both Apple, with its announcement last week, and Dynamicland offer us a look at the “adjacent possible.” And though I didn’t fully realize it until the end of the Apple keynote, and for all my admiration for and reliance on Apple’s products, I find Dynamicland’s communal, accessible vision the more compelling one. But until the arrival of Victor’s “humane dynamic medium,” I’ll keep working to explore and customize the devices I own. I want personal, not isolated, computing.

Love all ways,
Brian

PS—The title of this essay, “New Visions,” is a reference to photographer László Moholy-Nagy’s 1932 manifesto “A New Instrument of Vision,” which discussed how then-new technologies were reshaping culture. I’d be remiss if I did not mention an exhibition of the same name now on view at The Henie Onstad Kunstsenter. (Full disclosure: I worked as an editor on the accompanying catalogue.) That show examines “today’s artists us[ing] photography to explore [our] entanglements with machines and processes of automation and extraction.”

Subscribe

Read more