larlet-fr-david-cache/cache/3443a780540007bda6c4333d4325113d/index.md

title: Futures of text | Whoops by Jonathan Libov
url: http://whoo.ps/2015/02/23/futures-of-text
hash_url: 3443a780540007bda6c4333d4325113d

<p>When the weather is bad I take a bus to work. I’m forever grateful to the person at the bus stop who informed me that you can text New York’s MTA service to find out exactly where the bus is and when it’s going to arrive. Sure, an app that put the bus on a map would be more rich in information, but when I got to texting Bus Time I thought, “Thank god I don’t need to download another f------ app for this.”</p>

<p><span class="img-border"><img src="https://s3.amazonaws.com/whoops-images/images/futures-of-text/bus-time.gif" alt="Texting with Bus Time"/></span>
<span class="caption">Texting with Bus Time</span></p>

<p>In contrast to a GUI that defines rules for each interaction — rules which, frustratingly, change from app to app — text-based, conversational interactions are liberating in their familiarity.  There's only really only one way to skin this cat: The text I type is displayed on the right, the text someone else typed is on the left, and there's an input field on bottom for me to compose a message. </p>

<p><img src="https://s3.amazonaws.com/whoops-images/images/futures-of-text/Messengers.png" alt="There are only so many ways to skin this messaging cat"/>
<span class="caption">There are only so many ways to skin this messaging cat</span></p>

<p>The other primary alternatives to the “There’s an app for that” paradigm are Google Now and Siri today. However, I’m skeptical of a future where we communicate with computers primarily by voice. The visions in 2001: A Space Odyssey and the Her portray voice as the most effortless interaction, but <a href="http://whoo.ps/2014/04/07/exactly-why-is-it-better-when-there-s-an-app-for-that">voice actually requires a lot more cognitive and physical effort</a> than pointing with a mouse, typing on a keyboard, or tapping on app icon and then navigating the UI. Consider all those times you’ve exchanged a million texts with someone while making plans when voice would have resolved it much more quickly. Text is often more comfortable even if it’s less convenient.</p>

<p>I believe comfort, not convenience, is the most important thing in software, and text is an incredibly comfortable medium. Text-based interaction is fast, fun, funny, flexible, intimate, descriptive and even consistent in ways that voice and user interface often are not. <a href="http://graydon2.dreamwidth.org/193447.html">Always bet on text</a>:</p>

<blockquote>
<p><span class="blockquote-long">Text is the most socially useful communication technology. It works well in 1:1, 1:N, and M:N modes. It can be indexed and searched efficiently, even by hand. It can be translated. It can be produced and consumed at variable speeds. It is asynchronous. It can be compared, diffed, clustered, corrected, summarized and filtered algorithmically. It permits multiparty editing. It permits branching conversations, lurking, annotation, quoting, reviewing, summarizing, structured responses, exegesis, even fan fic. The breadth, scale and depth of ways people use text is unmatched by anything.</span></p>
</blockquote>

<p>Convenient and comfortable as Bus Time may be, this interaction is still suboptimal: The bot’s language is unnatural and, having interacted with it for months, it never learns that this is my bus, even though it’s the only bus I ever ask about. This actually highlights one of the fundamental differences between apps and services: Whereas we view services as mere endpoints for input/output, we expect apps to retain state from session to session. I’m looking forward to the day when the service graduates from service to app and begins to retain state. I take my seat on the bus and the conversation keeps going:</p>

<p><span class="img-border"><img src="https://s3.amazonaws.com/whoops-images/images/futures-of-text/bus-time-2.png" alt="A smarter Bus Time"/></span>
<span class="caption">A smarter Bus Time</span></p>

<p>The other obvious problem is that this interaction is as strict as the command line: If I type “23&amp; 8ht 23M” it might not work. Natural Language Processing still isn’t good enough, or ubiquitous enough, to power an app that primarily interacts via messaging. </p>

<p>Until NLP gets there, though, there are a few ways to alleviate its current shortcomings, and we can see a few of them happening right now.</p>

<h3>GUI-Aided Chat</h3>

<p><a href="http://www.lark.com/">Lark for iPhone</a> is a virtual health coach that interfaces with HealthKit on the iPhone. They do an excellent job at weaving free-form chat with GUI.</p>

<p><video id="myVideo" muted="" autoplay="" loop="">
                                        <source src="http://lark.com/videos/preview.webm" type="video/webm">
                                        <source src="http://lark.com/videos/preview.ogv" type="video/ogv">
                                        <source src="http://lark.com/videos/preview.mp4" type="video/mp4">
                                    </source></source></source></video>
</p>

<p>Lark is also excellent at message design. The tone is natural and the tempo is fast but not so fast as to make you feel like your responses are perfunctory. </p>

<p>Some of these smarts are already built into the iOS Messages app. QuickType, which is new to iOS 8 but not far removed from typing assistants like T9 and Swype, is quite good at turning a prompt (“Send: 1 or 2”) from SMS into a series of one-tap inputs (“1”, “2”, and “Not sure”). This isn’t Photoshopped.</p>

<p><img src="https://s3.amazonaws.com/whoops-images/images/futures-of-text/quicktype.png" alt="QuickType hints at a smarter Messages app"/>
<span class="caption">QuickType hints at a smarter Messages app</span></p>

<p>I’m slightly surprised that we haven’t yet seen a chat agent that leverages the convenience of QuickType. The primary obstacle is likely that SMS is slow and expensive, which is why the App as Personae model, via one of the OTT messengers, makes so much sense.</p>

<h3>Apps as Personas</h3>

<p>Here in Western markets, if you want to interact with a service from your phone, you either visit its mobile website or, more likely, you download the app. In China’s WeChat and other services across Asia, the services you may want to interact with are right there in your messenger. There’s no need to download an app: It’s as if you could just tap on an app in the App Store and start using it within the App Store app. </p>

<p><img src="http://dangrover.com/img/content/chineseapps/oas.png" alt="The array of messaging experiences in China"/>
<span class="caption">The array of messaging experiences in China</span></p>

<p>App-as-Personae is a more elegant solution to the problem I described earlier about Bus Time (i.e., it never retains a state that M23 is my bus). One easy alternative for Bus Time that would be to offer M23 as a unique PTSN that I can message with instead. </p>

<p><span class="img-border"><img src="https://s3.amazonaws.com/whoops-images/images/futures-of-text/bus-time-3.png" alt="A PTSN for my bus"/></span>
<span class="caption">A PTSN for my bus</span></p>

<p>The most obvious candidate to become the <a href="https://medium.com/@tedlivingston/the-race-to-become-the-wechat-of-the-west-3fe52c8db946">WeChat of the West</a> is Facebook Messenger, which <a href="http://www.bloomberg.com/news/2014-11-11/what-s-next-for-facebook-messenger-look-to-asia.html">brought on David Marcus</a> to do something along those lines (“Look East, young man” is the new “Look West, young man”). They also recently purchased <a href="https://wit.ai/blog/2015/01/05/wit-ai-facebook">Wit.ai</a>, an API provider that helped app developers parse natural language. Perhaps Facebook has already burned developers too badly in the past to even attempt this. Regardless, one can easily imagine our services sitting right alongside our contacts in a messenger experience.</p>

<p><span class="img-border"><img src="https://s3.amazonaws.com/whoops-images/images/futures-of-text/FB-Messenger.png" alt="Facebook Messenger apps"/></span>
<span class="caption">Facebook Messenger apps</span></p>

<p>Meanwhile, Path (reportedly an acquisition target for Apple) <a href="http://techcrunch.com/2014/06/20/path-talk-talkto/">acquired TalkTo</a> to build a messenger where services and venues sit side by side in the contact list with friends. Kik and Snapchat have eyes on the same market. Arriving on the scene this weekend is <a href="http://getmagicnow.com">Magic</a>, is a virtual assistant that you interact with purely by SMS:</p>

<p><span class="img-border"><img src="https://s3.amazonaws.com/whoops-images/images/futures-of-text/IMG_4360.PNG" alt="Magic as a service"/></span>
<span class="caption">Magic as a service</span></p>

<p>Were an App-as-Personae model to emerge in the West, it would be at least somewhat disruptive to Google (because it could <a href="http://alexiskold.net/2015/02/22/ive-seen-the-new-face-of-search-and-it-aint-google/">cut away from Search</a>) and Apple (because it could cut away from the App Store). That’s especially true if <a href="http://blog.intercom.io/why-cards-are-the-future-of-the-web/">the rise of cards</a> comes to fruition, and all the content locked in apps and on the web can be quickly consumed from within a chat. Here’s that vision realized with the Wildcard SDK:</p>

<p>Look at how well <a href="http://luka.ai">Luka</a>, which popped up only today <a href="http://www.producthunt.com/posts/luka">on Product Hunt</a>, meshes chat with cards.</p>

<p>You’d think that Apple and Google would see this coming, so it’s likely that they would either 1) Try to stifle Facebook’s efforts to bundle all services inside Messenger (seemingly impractical) or 2) Try to beat them to the punch. </p>

<p class="width: 100%; text-align:center;">
</p>

<p>The most obvious path for Google and Apple to beat one of the messengers to the punch is to open up their own messengers: Hangouts and Messages. That would entail disrupting their own models to some degree, but there’s yet another alternative that might preempt a runaway messenger: embed services within all text across the OS.</p>

<h3>Deeper Semantics</h3>

<p>All the examples I’ve shown to this point depict a largely discrete mode of interaction. GUI-aided chat presents the end user with explicit responses, while Apps as Personas merely shifts the origin of the “launch” action from the springboard to the contact list in your messenger. As enumerated earlier, the primary interaction models since the beginning of computing have been very discrete.</p>

<p>What if the next model were significantly more fluid and conversational? Rather than the OS defining the rules of launching an app, users essentially drive their interaction with services according to their needs and context. </p>

<p>We can see the beginnings of this model on the market today. On the desktop, <a href="http://chatgrape.com">ChatGrape</a> obviates the need to open a separate tab or app to get you to do your data and documents: Just type ‘#’ and then follow it with the data or documents you’re looking for.</p>

<p><span class="img-border"><img src="https://ug-cdn.com/static/chatgrape/static/videos/chatgrape-autocomplete-v2.gif" alt="On ChatGrape, your files come to you"/></span>
<span class="caption">On ChatGrape, your files come to you</span></p>

<p>Sure, the examples you see here entail opening files within third-party apps, but from here you’re only a hop, skip and a jump from accessing data without ever launching an app. There are tremendous productivity gains to be made by embedding within messaging the data that is currently locked in files.</p>

<p>Perhaps no one has embraced messaging as an input method as well as Slack. In Slack, for example, you never fill out your profile information in what would <a href="https://medium.com/@einkoenig/batman-onboarding-999d19f0cab9">otherwise be a traditional, dull form</a>. You just chat it out with Slackbot.</p>

<p><span class="img-border"><img src="https://s3.amazonaws.com/whoops-images/images/futures-of-text/Slack.png" alt="Slack replaces the boring ol' input form with an informal chat"/></span>
<span class="caption">Slack replaces the boring ol' input form with an informal chat</span></p>

<p>The integrations in Slack are also amazing. For example, you can <a href="http://blog.appear.in/post/105623810020/appear-in-integration-for-slack">just type /appear</a> and you’ve launched an appear.in video chat. </p>

<p><span class="img-border"><img src="https://s3.amazonaws.com/whoops-images/images/futures-of-text/appearin.gif" alt="Want to launch an app? Just type it out"/></span>
<span class="caption">Want to launch an app?</span></p>

<p>As before, that launches the appear.in app, but you’re only a hop, skip and a jump from an interaction model that more closely mirrors your omnipresent virtual assistant: You state, in fairly plain language aided by an escape character, that you want to launch a video call, and then you’re in a video call. </p>

<p>It’s not hard to imagine how this works on the phone. In fact, it already kinda does:</p>

<p><span class="img-border"><img src="https://s3.amazonaws.com/whoops-images/images/futures-of-text/calendar.gif" alt="The OS already recognizes calendar- and contact-related keywords"/></span>
<span class="caption">The OS already recognizes calendar- and contact-related keywords</span></p>

<p>Now expand the horizons of the keywords that the OS can identify:</p>

<p><span class="img-border"><img src="https://s3.amazonaws.com/whoops-images/images/futures-of-text/video-chat.png" alt="There’s still quite a lot of low-hanging fruit for semantics"/></span>
<span class="caption">There’s still quite a lot of low-hanging fruit for semantics</span></p>

<p>Here the OS recognizes “talk this out” as a keyword for placing a call and immediately initializes a call. This saves one from launching FaceTime, or tapping into the person’s contact view to find that little “Call on Facetime” icon. </p>

<p>Now expand the horizons even further and imagine that apps and services can respond to all kinds of objects: Dates, Actions, Names, Brands and so on. Say the OS knows that I’m a Foursquare user, I could ask Foursquare directly for a recommendation:</p>

<p><span class="img-border"><img src="https://s3.amazonaws.com/whoops-images/images/futures-of-text/Message+Square.png" alt="What if we could interact directly with text?"/></span>
<span class="caption">What if we could interact directly with text?</span></p>

<p>Imagine if services could even respond directly to my input:</p>

<p><img src="https://s3.amazonaws.com/whoops-images/images/futures-of-text/QuickType+Apps.png" alt="What if we could interact directly with text?"/>
<span class="caption">What if the OS facilitated interaction with services via text?</span></p>

<p>Or, outside of messengers, services that respond to text selection with semantically relevant content. </p>

<p><img src="https://s3.amazonaws.com/whoops-images/images/futures-of-text/Instapaper.png" alt="What if we could interact directly with text?"/>
<span class="caption">What if these services weren’t even limited to messaging?</span></p>

<p>What’s intriguing about this is how it shifts the responsibility of explicitly linking content from the content creator to the OS and the services themselves. Or perhaps even, in the example above, the person whose name is highlighted might have some say in the matter. The possibilities are as broad as language itself.</p>

<h3>The end of the beginning</h3>

<p>Messaging is the only interface in which the machine communicates with you much the same as the way you communicate with it. If some of the trends outlined in this post pervade, it would mark a qualitative shift in how we interact with computers. Whereas computer interaction to date has largely been about discrete, deliberate events — typing in the command line, clicking on files, clicking on hyperlinks, tapping on icons — a shift to  messaging- or conversational-based UI's and implicit hyperlinks would make computer interaction far more fluid and natural.</p>

<p>What's more, messaging AI benefits from an obvious feedback loop: The more we interact with bots and messaging UI's, the better it'll get. That's perhaps true for GUI as well, but to a far lesser degree. Messaging AI may get better at a rate we've never seen in the GUI world. Hold on tight.</p>