larlet-fr-david-cache/cache/ac15b12059caac5a141edb44f039a6b6/index.md

title: The Firefox Experiments I Would Have Liked To Try
url: http://www.ianbicking.org/blog/2019/03/firefox-experiments-i-would-have-liked.html
hash_url: ac15b12059caac5a141edb44f039a6b6

<p>I have been part of the Firefox Test Pilot team for several years. I had a long list of things I wanted to build. Some I didn’t personally want to build, but I thought they were interesting ideas. I didn’t get very far through this list at all, and now that <a href="https://medium.com/firefox-test-pilot/adios-amigo-51bec2a00072">Test Pilot is being retired</a> I am unlikely to get to them in the future.</p>
<p>Given this I feel I have to move this work out of my head, and publishing a list of ideas seems like an okay way to do that. Many of these ideas were inspired by something I saw in the wild, sometimes a complete product (envy on my part!), or the seed of an idea embedded in some other product.</p>
<p>The experiments are a spread: some are little features that seem potentially useful. Others are features seen elsewhere that show promise from user research, but we could only ship them with confidence if we did our own analysis. Some of these are just ideas for how to explore an area more deeply, without a clear product in mind.</p>
<p>Test Pilot’s purpose was to find things worth shipping in the browser, which means some of these experiments aren’t novel, but there is an underlying question: would people actually use it? We can look at competitors to get ideas, but we have to ship something ourselves if we want to analyze the benefit.</p>

<hr/>

<p><img alt="mockup of Sticky Reader Mode" src="/media/mockups/sticky-readable-start.gif"/></p>
<p>Give <a href="https://support.mozilla.org/en-US/kb/firefox-reader-view-clutter-free-web-pages">Reader Mode</a> in Firefox a preference to make it per-domain sticky. E.g. if I use Reader Mode on nytimes.com and then if I visit an article on nytimes.com in the future it’ll automatically convert to reader mode. (The nytimes.com homepage would not be a candidate for that mode.)</p>
<p>I made an experiment in <a href="https://github.com/ianb/sticky-reader-mode"><code>sticky-reader-mode</code></a>, and I think it works really nicely. It changes the browsing experience significantly, and most importantly it doesn’t require frequent proactive engagement to change behavior. Lots of these proposed ideas are tools that require high engagement by the user, and if you don’t invoke the tool then they do nothing. In practice no one (myself included) remembers to invoke these tools. Once you click the preference on a site Sticky Reader Mode then you are opted in to this new experience with no further action required.</p>
<p>There are a <a href="https://github.com/ianb/sticky-reader-mode/issues/5">bunch of similar add-ons</a>. Sticky Reader Mode works a bit better than most because of its interface, and it will push you directly into Reader Mode without rendering the normal page. <strong>But</strong> it does this by using APIs that are not public to normal WebExtensions. As a result it can’t be shipped outside Test Pilot, and can’t go in addons.mozilla.org. So… just trust me, it’s great.</p>
<p>Recently I’ve come upon <a href="https://brave.com/blog/speed-reader/">Brave Speed Reader</a> which is similar, but without per-site opt-in, and using machine learning to identify articles.</p>

<p><img alt="mockup of a Cloud Browser" src="/media/mockups/cloud-browser-start.gif"/></p>
<p>Run a browser/user-agent in the cloud and use a mobile view as a kind of semantic or parsed view on that user agent (the phone would just control the browser that is hosted on the cloud). At its simplest we just take the page, simplify it in a few ways, and send it on - similar to what Opera Mini does. The approach lends itself to a variety of task-oriented representations of remote content.</p>
<p>When I first wrote this down I had just stared at my phone while it took 30 seconds to show me a 404 page. The browser probably knew after a couple seconds that it was a 404 but it acted as a rendering engine and not a user agent, so the browser insisted on faithfully rendering the useless not found page.</p>
<p>Obviously running a full browser instance in the cloud is resource hungry and finicky but I think we could ignore those issues while testing. Those are hard but solved operational issues.</p>
<p>Prior art: Opera Mini does some of this. Puffin is specifically cloud rendering for mobile. Light Point does the same for security reasons.</p>
<p>I later encountered <a href="https://www.brow.sh">brow.sh</a> which is another interesting take on this (specifically with <a href="https://html.brow.sh">html.brow.sh</a>).</p>
<p>This is a very big task, but I still believe there’s tremendous potential in it. Most of my concepts are not mobile-based, in part because I don’t like mobile, I don’t like myself when using a mobile device, and it’s not something I want to put my energy into. But I still like this idea.</p>

<p><img alt="mockup of Modal Page Actions" src="/media/mockups/modal-page-actions-start.gif"/></p>
<p>This was tangentially inspired by <a href="https://help.vivaldi.com/article/image-properties/">Vivaldi’s Image Properties</a>, not because of the interface, but thinking about how to fit high-information inspection tools into the browser.</p>
<p>The idea: instead of context menus, <a href="https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/user_interface/Page_actions">page actions</a>, or other interaction points that are part of the “chrome”, implement one overlay interface: the do-something-with-this-page interface. Might also be do-something-with-this-element (e.g. replacing the 7 image-related context menu items: View Image, Copy Image, Copy Image Location, Save Image As, Email Image, Set As Desktop Background, and View Image Info).</p>
<p>The interface would be an overlay onto the page, similar to what happens when you start Screenshots:</p>
<p><img alt="Screenshots interface" src="/media/mockups/screenshot-interface-example-small.png"/></p>
<p>Everything that is now in the Page Action menu (the <code>...</code> in the <span class="caps">URL</span> bar), or in the context menu, would be available here. Some items might have a richer interface, e.g., Send Tab To Device would show the devices directly instead of using a submenu. Bookmarking would include some inline <span class="caps">UI</span> for managing the resulting bookmark, and so on.</p>
<p>There was some pushback because of the <a href="https://textslashplain.com/2017/01/14/the-line-of-death/">line of death</a> – that is, the idea all trusted content must clearly originate from the browser chrome, and not the content area. I do not believe in the Line of Death, it’s something users <em>could</em> use to form trust, but I don’t believe they <em>do</em> use it (further user research required).</p>
<p>The general pattern is inspired by mobile interfaces which are typically much more modal than desktop interfaces. Modal interfaces have gotten a bad rap, I think somewhat undeserved: modal interfaces are also interfaces that guide you through processes, or ask you to explicitly dismiss the interface. It’s not unreasonable to expect someone to finish what they start.</p>

<p><img alt="mockup of Find + 1" src="/media/mockups/find-plus-1-start.gif"/></p>
<p>We have find-in-page but what about find-in-anything-linked-from-this-page?</p>
<p>Hit Cmd-Shift-F and you get an interface to do that. All the linked pages will be loaded in the background and as you search we show snippets of matching pages. Clicking on a snippet opens or focuses the tab and goes to where the search term was found.</p>
<p>I started experimenting in <a href="https://github.com/ianb/find-plus-one"><code>find-plus-one</code></a> and encountered some challenges: hidden tabs aren’t good workers, loading pages in the background takes a lot of grinding in Firefox, and most links on pages are stupid (e.g., I don’t want to search your Careers page). An important building block would be a way to identify the important (non-navigational) parts of a page. Maybe lighter-weight ways to load pages (in other projects I’ve used <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP"><span class="caps">CSP</span></a> injection). The Copy Keeper concept did come about while I experimented with this.</p>
<p>A simpler implementation of this might simply do a text search of all your open tabs, which would be technically simpler and mostly an exercise in making a good representation of the results.</p>

<p><img alt="mockup of Your Front Page" src="/media/mockups/your-front-page-start.gif"/></p>
<p>Create a front page of news from the sites you already visit. Like an <span class="caps">RSS</span> reader, but prepopulated with your history. This creates an immediate well-populated experience.</p>
<p>My initial thought was to use ad hoc parsers for popular news sites, and at run an experiment with just a long whitelist of news providers.</p>
<p>I got the feedback: why not just use <span class="caps">RSS</span>? Good question: I thought <span class="caps">RSS</span> was kind of passé, but I hadn’t looked for myself. I went on to do some analysis of <span class="caps">RSS</span>, and found it available for almost all news sites. The <a href="https://developer.mozilla.org/en-US/docs/Archive/RSS/Getting_Started/Syndicating#Adding_the_.3Clink.3E">autodetection</a> (<code>&lt;link rel=alternate&gt;</code>) is <em>not</em> as widely available, and it requires manual searching to find many feeds. Still <span class="caps">RSS</span> is a good way to get an up-to-date list of articles and their titles. Article content isn’t well represented and other article metadata is inaccurate or malformed (e.g., there are no useful tags). So using <span class="caps">RSS</span> would be very reasonable discovery mechanism, but an “<span class="caps">RSS</span> reader” doesn’t seem like a good direction on the current web.</p>

<p>This is bringing back old functionality from Page Shot, a project of mine which morphed into <a href="https://screenshots.firefox.com">Firefox Screenshots</a>: save full <span class="caps">DOM</span> copies of pages. What used to be fairly novel is now well represented by several projects (e.g., <a href="https://blog.webmemex.org/">WebMemex</a> or <a href="https://worldbrain.io/">World Brain Memex</a>).</p>
<p>Unfortunately I have never been able to really make this kind of tool part of <em>my own</em> day-to-day behavior, and I’ve become skeptical it can work for a general populace. But maybe there’s a way to package up this functionality that is more accessible, or happens more implicitly. I <a href="https://github.com/ianb/pagearchive">forked a version of Page Shot as pagearchive</a> a while ago, with this in mind, but I haven’t (and likely won’t) come back to it.</p>

<p>This isn’t really a product idea, but instead an approach to developing products.</p>
<p>One can imagine many tools that directly interact or learn from the content of your browsing. There is both a privacy issue here and a privacy opportunity: looking at this data is creepy, but if the tools live in your user agent (that belongs to you and hosts your information locally) then it’s not so creepy.</p>
<p>But it’s really hard to make experiments on this because you need a bunch of data. If you build a tool that starts watching your browsing then it will only slowly build up interesting information. The actual information that is already saved in browser history is interesting, but in my experience it is too limited and of poor quality. For instance, it is quite hard to build up a navigational path from the history when you use multiple tabs.</p>
<p>A better iterative development approach would be one where you have a static set of all the information you might want, and you can apply tools to that information. If you find something good then later you can add new data collection to the browser, secure in the knowledge that it’s going to find interesting things.</p>
<p>I spent quite a bit of effort on this, and produced <a href="https://github.com/ianb/personal-history-archive">`personal-history-archive</a>. It’s something I still want to come back to. It’s a bit of a mess, because at various times it was retrofitted to collect historical information, or collect it on an ongoing basis, or collected it when driven by a script. I also tried to build tools in parallel for doing analysis on the resulting database.</p>
<p>This is also a byproduct of experimentation with machine learning. I wanted to apply things I was learning to browser data, but the data I wanted wasn’t there. I spent all my time collecting and cleaning data, and ended up spending only a small amount of time analyzing the data. I suspect I’m not the only one who has done this.</p>

<p><img alt="mockup of Navigational Breadcrumbs" src="/media/mockups/navigational-breadcrumbs-start.gif"/></p>
<p>When I click on a link I lose the reminder of why I clicked on it. What on the previous page led me to click on this? Was I promised something? Are there sibling links that I might want to continue to directly instead of going back and selecting another link?</p>
<p>This tool would give you additional information about the page you are on, how you got there, and given where you came from, where you might go next. Would this be a sidebar? Overlay content? In a popup? I’m not sure.</p>
<p>Example: using this, if I click on a link from Reddit I will be able to see the title of the Reddit post (which usually doesn’t match the document title), and a link to comments on the page. If I follow a link from Twitter, I’ll be able to see the Tweet I came from.</p>
<p>This could be interesting paired with link preview (like a tentative forward). Maybe the mobile browser <a href="https://linkbubble.com/">Linkbubbles</a> (now integrated into Brave) has some ideas to offer.</p>
<p>Technically this will use some of the techniques from Personal History Archive, which tracks link sources.</p>
<p>This is based on the train of thought I wrote down <a href="https://news.ycombinator.com/item?id=17702206">in an <span class="caps">HN</span> comment</a> – itself a response to <a href="https://www.reinterpretcast.com/open-hypermedia">Freeing the Web from the Browser</a>.</p>
<p>I want to try this still, and have started a repo <a href="https://github.com/ianb/crossnav"><code>crossnav</code></a> but haven’t put anything there yet. I think even some naive approaches could work, just trying to detect the <em>category</em> of link and the <em>related</em> links (e.g., on Reddit the category is other submissions, and the related links are things like comments).</p>

<p><img alt="mockup of Copy Keeper" src="/media/mockups/copy-keeper-start.gif"/></p>
<p>A notebook/logbook that is filled in every time you copy from a web page. When you copy it records (locally):</p>
<ul>
<li>Text of selection</li>
<li><span class="caps">HTML</span> of selection</li>
<li>Screenshot of the block element around the selection</li>
<li>Text around selection</li>
<li>Page <span class="caps">URL</span> and nearest anchor/id</li>
<li>Page title</li>
<li>Datetime</li>
</ul>
<p>This overloads “copy” to mean “remember”.</p>
<p>Clips would be searchable, and could be moved back to the clipboard in different forms (text, <span class="caps">HTML</span>, image, bibliographical reference, source <span class="caps">URL</span>). Maybe clips would be browsable in a sidebar (maybe the sidebar has to be open for copies to be collected), or clips could be browsed in a normal tab (Library-style).</p>
<p>I created a prototype in <a href="https://github.com/ianb/copy-keeper"><code>copy-keeper</code></a>. I thought it was interesting and usable, though whether it would actually get any use in practice I don’t know. It’s one of those tools that seems handy but requires effort, and as a result doesn’t get used.</p>

<p><img alt="mockup of Change Scout" src="/media/mockups/change-scout-start.gif"/></p>
<p>(Wherein I both steal a <a href="https://www.businessinsider.com/mozilla-scout-voice-controlled-browser-2018-6">name from another team</a>, and turn it into a category…)</p>
<p>Change Scout will monitor a page for you, and notify you when it changes. Did someone edit the document? Was there activity on an issue? Did an article get updated? Put Change Scout to track it and it will tell you what changes and when.</p>
<p>It would monitor the page inside the browser, so it would have access to personalized and authenticated content. A key task would be finding ways to present changes in an interesting and compact way. In another experiment I tried some very simple change detection tools, and mostly end up frustrated (small changes look very large to naive algorithms).</p>
<h2/>
<p><img alt="Tab Switcher mockup" src="/media/mockups/tab-switcher-start.gif"/></p>
<p>We take the exact <span class="caps">UI</span> of the <a href="https://github.com/mozilla/side-view">Side View</a> popup, but make it a tab switcher. “Recent Tabs” are the most recently focused tabs (weighted somewhat by how long you were on the tab), and then there’s the complete scrollable list. Clicking on an item simply focuses that tab. You can close tabs without focusing them.</p>
<p>I made a prototype in <a href="https://github.com/ianb/tab-switchr"><code>tab-switchr</code></a>. In it I also added some controls to close tabs, which was very useful for my periodic tab cleanups. Given that it was a proactive tool, I surprised myself by using it frequently. There’s work in Firefox to improve this, unrelated to anything I’ve done. It reminds me a bit of various <a href="https://addons.mozilla.org/en-US/firefox/addon/tree-style-tab/">Tree-Style Tabs</a>, which I both like because they make it easier to see my tabs, and dislike because I ultimately am settled on normal top-tabs. The popup interface is less radical but still provides many of the benefits.</p>
<p>I should probably clean this up a little and publish it.</p>

<p>Create your own <span class="caps">RSS</span> feed.</p>
<ul>
<li>When you are on a page with some audio source, you can add the audio to your personal feed</li>
<li>When on an article, you can generate an audio version that will be added to the feed</li>
<li>You get an <span class="caps">RSS</span> feed with a random token to make it private (I don’t think podcast apps handle authentication well, but this requires research)</li>
<li>Maybe you can just send/text the link to add it to your preferred podcast app</li>
<li>If apps don’t accept <span class="caps">RSS</span> links very well, maybe something more complicated would be required. An app that just installs an <span class="caps">RSS</span> feed? We want to avoid the feed accidentally ending up in podcast directories.</li>
</ul>

<p>There’s a lot of low-rated bookmark managers in addons.mozilla.org and the Chrome Extension store. Let’s make our own low-rated bookmark manager!</p>
<p>But seriously, this would anticipate updates to the Library and built-in bookmark manager, which are deficient.</p>
<p>Some resources/ideas:
<em> <a href="https://news.ycombinator.com/item?id=17072803">Comment with a few gripes</a>
</em> <a href="https://chrome.google.com/webstore/detail/bookmark-manager/gmlllbghnfkpflemihljekbapjopfjik?hl=en">Google’s bookmark manager</a>
<em> <a href="https://addons.mozilla.org/en-US/firefox/addon/bookmarks-manager-and-viewer/?src=search">Bookmark section on addons.mozilla.org</a>
</em> <a href="https://addons.mozilla.org/en-US/firefox/addon/bookmarks-organizer/?src=search">Bookmark organizers on addons.mozilla.org</a>
* <a href="https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/bookmarks">Relevant WebExtension APIs</a></p>

<p><img alt="mockup of the Extended Library" src="/media/mockups/extended-library-start.gif"/></p>
<p>The “Library” in Firefox is the combination history and bookmark browser you get if you use “Show all bookmarks” or “Show all history”.</p>
<p>In this idea we present the user with a record of their <strong>assets</strong>, wherever they are.</p>
<p>This is like a history view (and would be built from history), but would use heuristics to pick out certain kinds of things: docs you’ve edited, screenshots you’ve taken, tickets you’ve opened, etc. We’d be trying hard to find long-lived documents in your history, instead of transitional navigation, articles, things you’ve gotten to from public indexes, etc.</p>
<p>Automatically determining what should be tagged as a “library item” would be the hard part. But I think having an organic view of these items, regardless of underlying service, would be quite valuable. The browser has access to all your services, and it’s easy to forget what service hosts the thing you are thinking about.</p>

<p><img alt="mockup of Text Mobile Screenshot" src="/media/mockups/text-mobile-screenshot-start.gif"/></p>
<p>This tool will render the tab in a mobile factor (using the devtools <a href="https://developer.mozilla.org/en-US/docs/Tools/Responsive_Design_Mode">responsive design mode</a>), take a full-page screenshot, and text the image and <span class="caps">URL</span> to a given number. Probably it would only support texting to yourself.</p>
<p>I’ve looked into this some, and getting the mobile view of a page is not entirely obvious and requires digging around deep in the browser. Devtools does some complicated stuff to display the mobile view. The rest is basic <span class="caps">UI</span> flows and operational support.</p>

<p>Emails the Reader Mode version of a site to yourself. In our research, people love to store things in Email, so why not?</p>
<p>Though it lacks the simplicity of this concept, <a href="https://github.com/mozilla/email-tabs">Email Tabs</a> contains this basic functionality. <a href="https://www.emailthis.me/">Email This</a> does almost exactly this.</p>

<p>An extension that finds and syncs your history between browsers (particularly between Chrome and Firefox).</p>
<p>This would use the <a href="https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/history">history WebExtension APIs</a>. Maybe we could create a <a href="https://en.wikipedia.org/wiki/Firefox_Sync">Firefox Sync</a> client in Chrome. Maybe it could be a general way to move things between browsers. Actual synchronization is hard, but creating read-only views into the data in another browser profile is much easier.</p>
<p>Obviously there’s lots of work to synchronize this data between Firefox properties, and knowing the work involved this isn’t easy and often involves close work with the underlying platform. Without full access to the platform (like on Chrome) we’ll have to find ways to simplify the problem in order to make it feasible.</p>

<p>Everyone (with an <a href="https://www.mozilla.org/en-US/firefox/accounts/">FxA account</a>) gets there own homepage on the web. It’s like Geocities! Or maybe closer to github.io.</p>
<p>But more seriously, it would be programmatically accessible simple static hosting. Not just for you to write your own homepage, but an open way for applications to publish user content, without those applications themselves turning into hosting platforms. We’d absorb all the annoyances of hosting content (abuse, copyright, quotas, ops, financing) and let open source developers focus on enabling interesting content generation experiences for users on the open web.</p>
<p><a href="http://www.ianbicking.org/blog/2019/01/we-need-open-hosting-platforms.html">Here’s a general argument</a> why I think this would be a useful thing for us to do. <a href="https://blog.lmorchard.com/2014/10/09/separating-publishing-from-hosting-on-the-web/">And another from Les Orchard</a>.</p>

<p>This is a proposal for user research:</p>
<p>Electron apps are being shipped for many services, including services that don’t require any special system integration. E.g., Slack doesn’t require anything that a web browser can’t do. Spotify maybe catches some play/pause keys, but is very close to being a web site. Yet there is perceived value in having an app.</p>
<p>The user research would focus on cases where the Electron app doesn’t have any/many special permissions. What gives the app value over the web page?</p>
<p>The goal would be to understand the motivations and constraints of users, so we could consider ways to make the in-browser experience equally pleasant to the Electron app.</p>

<p>Per my previous item: why do I have an <a href="https://www.irccloud.com/">IRCCloud</a> app? Why do people use Slack apps? Maybe it’s just because they want to be able to switch into and out of those apps quickly.</p>
<p>A proposed product solution: add a shortcut to any specific (pinned?) tab. Might be autocreated. Using the shortcut when the app is already selected will switch you back to your previous-selected tab. Switching to the tab without the shortcut will display a gentle reminder that the shortcut exists (so you can train yourself to start using it).</p>
<p>To make it a little more fancy, I thought we might also be able to do a second related “preview” shortcut. This would let you peek into the window. I’m not sure what “peeking” means. Maybe we just show a popup with a screenshot of that other window.</p>
<p>Maybe this should all just overload ⌘1/2/3 (maybe shift-⌘1/etc for peeking). Note these shortcuts do not currently have memory – you can switch <em>to</em> the first tab with ⌘1, but you can’t switch back.</p>
<p>This is one suggested solution to <em>Whatever Electron does for people</em>.</p>
<p>I started some work in <a href="https://github.com/ianb/quick-switch-extension"><code>quick-switch-extension</code></a>, but keyboard shortcuts were a bit wonky, and I couldn’t figure out useful additional functionality that would make it fun. Firefox (Nightly?) now has Ctrl-Tab functionality that takes you to recent tabs, mitigating this problem (though it is not nearly as predictable as what I propose here).</p>

<p>Just Save saves a page. It’s like a bookmark. Or a remembering. Or an archive. Or all of those all at once.</p>
<p>Just Save is a one-click operation, though a popup does show up (similar in function to Pocket) that would allow some additional annotation of your saved page.</p>
<p>We save:
1. Link
2. Title
3. Standard metadata
4. Screenshot
5. Frozen version of page
6. Scroll position
7. The tab history
8. Remember the other open tabs, so if some of them are saved we offer later relations between them
9. Time the page was saved
10. Query terms that led to the page</p>
<p>It’s like bookmarks, but purely focused on saving, while bookmarks do double-duty as a navigational tool. The tool encourages after-the-fact discovery and organization, not at-the-time-of-save choices.</p>
<p>And of course there’s a way to find and manage your saved pages. This idea needs more exploration of <em>why</em> you would return to a page or piece of information, and thus what we’d want to expose and surface from your history. We’ve <a href="https://blog.mozilla.org/ux/2015/02/save-share-revisit/">done research</a>, but it’s really just a start.</p>

<p>We have several <a href="https://dxr.mozilla.org/mozilla-central/source/browser/components/search/searchplugins">open search providers</a>. How many exist out there? How many could we find in history?</p>
<p>In theory <a href="https://developer.mozilla.org/en-US/docs/Web/OpenSearch">Open Search is an <span class="caps">API</span></a> where a user could do personalized search across many properties, though I’m not sure if any sufficient number of sites has enabled it.</p>

<p>It’s <a href="https://addons.mozilla.org/en-US/firefox/addon/notes-by-firefox/">Notes</a>, but with <a href="https://api.slack.com/slash-commands">slash commands</a>.</p>
<p>I other words it’s a document, but if you complete a line that begins with a <code>/</code> then it will try to execute that command, appending or overwriting from that point.</p>
<p>So for instance <code>/timestamp</code> just replaces itself with a timestamp.</p>
<p>Maybe <code>/page</code> inserts the currently active tab. <code>/search foo</code> puts search results into the document, but as editable (and followable) links. <code>/page save</code> freezes the page as one big data link, and inserts that link into the note.</p>
<p>It’s a little like Slack, but in document form, and with the browser as the context instead of a messaging platform. It’s a little like a notebook programming interface, but less structured and more document-like.</p>
<p>The ability to edit the <em>output</em> of commands is particularly interesting to me, and represents the kind of ad hoc information organizing that we all do regularly.</p>
<p>I experimented some with this in Notes, and got it working a little bit, but working with CKEditor (that Notes is built on) was just awful and I couldn’t get anything to work well. Notes also has a very limited set of supported content (no images or links), which was problematic. Maybe it’s worth doing it from scratch (with <a href="http://prosemirror.net/">ProseMirror</a> or <a href="https://github.com/ianstormtaylor/slate">Slate</a>?)</p>
<p>After I tried to mock this up, I realized that the underlying model is much too unclear in my mind. What’s this for? <em>When</em> is it for? What would a list of commands look like?</p>
<p>Another thing I realized while attempting a mockup is that there should be a rich but normalized way to represent pages and URLs and so forth. Often you’ll be referring to URLs of pages that are already open. You may want to open sets of pages, or see immediately which URLs are already open in a tab. A frozen version of a page should be clearly linked to the source of that page, which of course could be an open tab. There’s a lot of pieces to fit together here, both common nouns and verbs, all of which interact with the browser session itself.</p>

<p>Automation and scripting for your browser: make demonstrations for your browser, give it a name, and you have a repeatable script.</p>
<p>The scripts will happen in the browser itself, not via any backend or scraping tool. In case of failed expectations or changed sites, the script will halt and tell the user.</p>
<p>Scripts could be as simple as “open a new tab pointing to this page every weekday at 9am”, or could involve clipping information, or just doing a navigational pattern before presenting the page to a user.</p>
<p>There’s a huge amount of previous work in this area. I think the challenge here is to create something that <em>doesn’t</em> look like a programming language displayed in a table.</p>

<p>Sidekick is a sidebar interface to anything, or everything, contextually. Some things it might display:</p>
<ul>
<li>Show you the state of your clipboard</li>
<li>Show you how you got to the current tab (similar to <a href="#navigational-breadcrumbs">Navigational Breadcrumbs</a>)</li>
<li>Show you other items from the search query that kicked off the current tab</li>
<li>Give quick navigation to nearby pages, given the referring page (e.g., the next link, or next set of links)</li>
<li>Show you buttons to activate other tabs you are likely to switch to from the current tab</li>
<li>Show shopping recommendations or other content-aware widgets</li>
<li>Let you save little tidbits (text, links, etc), like an extended clipboard or notepad</li>
<li>Show notifications you’ve recently received</li>
<li>Peek into other tabs, or load them inline somewhat like <a href="https://addons.mozilla.org/en-US/firefox/addon/side-view/">Side View</a></li>
<li>Checklists and todos</li>
<li>Copy a bunch of links into the sidebar, then treat them like a todo/queue</li>
</ul>
<p>Possibly it could be treated like an extensible widget holder.</p>
<p>From another perspective: this is like a continuous contextual feature recommender. I.e., it would try to answer the question: what’s the feature you could use right now?</p>

<p>Generally in order to commit something to long-term memory you must revisit information later, ideally long enough that it’s a struggle.</p>
<p>Is anything we see in a browser worth committing to long-term memory? Sometimes it feels like nothing is worth remembering, but that’s a kind of nihilism based on the shitty aspects of typical web browsing behavior.</p>
<p>The interface would require some positive assertion: I want to know this. Probably you’d want to highlight the thing you’d “know”. Then, later, we’d want to come up with some challenge. We don’t need a “real” test that is verified by the browser, instead we simply need to ask some related question, then the user can say if they got it right or not (or remembered it or not).</p>


<p>Reader mode is a bit spartan. Maybe it could be a bit nicer:</p>
<ul>
<li>Pick up some styles or backgrounds from the hosting site</li>
<li>Display images or other media differently or more prominently</li>
<li>Add back some markup or layout that Readability erases</li>
<li>Apply to some other kinds of sites that aren’t articles (e.g., a video site)</li>
<li>A multicolumn version like <a href="https://anoved.github.io/mcreadability/">McReadability</a></li>
</ul>

<p>Inspired by <a href="http://www.fullhn.com/">Full Hacker News</a> (<a href="https://news.ycombinator.com/item?id=18132281">comments</a>): take a bunch of links (typically articles) and concatenate their content into one page.</p>
<p>Implicitly this requires Reader Mode parsing of the pages, though that is relatively cheap for “normal” articles. Acquiring a list of pages is somewhat less clear. Getting a list of pages is a kind of news/<span class="caps">RSS</span> question. Taking a page <em>like</em> Hacker News and figuring out what the “real” links are is another approach that may be interesting. Lists of related links are everywhere, yet hard to formally define.</p>
<p>This would work very nicely with complementary text summarization.</p>
<p>Open question: is this actually an interesting or useful way to consume information?</p>

<p>There’s an underlying concept here worth explaining:</p>
<p>Feature develop receives a lot of skepticism. And it’s reasonable: there’s a lot of conceit in a feature, especially embedded in a large product. Are people going to use a product or not because of some little feature? Or maybe the larger challenge: can some feature actually change behavior? Every person has their own thing going on, people aren’t interested in our theories, and really not that many people are interested in browsers. Familiar functionality – the back button, bookmarks, the <span class="caps">URL</span> bar, etc. – are what they expect, what they came for, and what they will gravitate to. Everything I’ve written so far in this list are things <em>people won’t actually use</em>.</p>
<p>A browser is particularly problematic because it’s so universal. It’s for sites and apps and articles. It’s for the young and the elderly, the experienced and not. It’s used for serious things, it’s used for concentration, and it’s used for dumb things and to avoid concentrating. How can you build a feature for everyone, targeting anything they might do? And if you build something, how can a person trust a new feature is really for <em>them</em>, not some other person? People are right to be skeptical of the new!</p>
<p>But we also know that most people regularly use more than one browser. Some people use Chrome for personal stuff, and Firefox for work. Some people do the exact opposite. Some people do their banking and finance in a specific browser. Some use a specific browser just for watching videos.</p>
<p><em>Which</em> browser a person uses for which task is seemingly random. Maybe they were told to use a specific browser for one task, and then the other browser became the fallback. Maybe they once heard somewhere once that one browser was more secure. Maybe flash seemed broken on one browser when they were watching a video, and now a pattern has been set.</p>
<p>This has long seemed like an opportunity to me. Market a browser that actually claims to be the right browser for some of these purposes! Firefox has <a href="https://www.mozilla.org/en-US/firefox/developer/">Developer Edition</a> and it’s been reasonably successful.</p>
<p>This offers an opportunity for both Mozilla and Firefox users to agree on <strong>purpose</strong>. What is Firefox for? Everything! Is this feature meant for <em>you</em>? Unlikely! In a purpose-built browser both sides can agree what it’s trying to accomplish.</p>
<p>This idea often gets poo-pooed for how much work it is, but I think it’s simpler than it seems. Here’s what a “new browser” means:</p>
<ul>
<li>Something you can find and download from its own page or site</li>
<li>It’s Firefox, but uses its own <a href="https://support.mozilla.org/en-US/kb/profiles-where-firefox-stores-user-data">profile</a>, keeping history/etc separate from other browser instances (including Firefox)</li>
<li>It has its own name and icon, and probably a theme to make it obvious what browser you are in</li>
<li>It comes with some browser extensions and prefs changed, making it more appropriate for the proposed use case</li>
</ul>
<p>The approach is heavy on marketing and build tools, and light on actual browser engineering.</p>
<p>I also have gotten frequent feedback that <a href="https://addons.mozilla.org/en-US/firefox/addon/multi-account-containers/">Multi-Account Containers</a> should solve all these use cases, but that gets everything backwards. People already understand multiple browsers, and having completely new entry points to bring people to Firefox is a feature, not a bug.</p>
<p>Sadly I think the time for this has passed, maybe in the market generally or maybe just for Mozilla. It would have been a very different approach to the browser.</p>
<p>Some of us in the Test Pilot team had some good brainstorming around actual concepts too, which is where I actually get excited about the ideas:</p>

<p>For students, studying.</p>
<ul>
<li>Integrate note-taking tools</li>
<li>Create project and class-based organizational tools, helping to organize tabs, bookmarks, and notes</li>
<li>Tools to document and organize deadlines</li>
<li>Citation generators</li>
</ul>
<p>I don’t know <em>what</em> to do with online lectures and video, but it feels like there’s some meaningful improvements to be done in that space. Video-position-aware notetaking tools?</p>
<p>I think the intentionality of opening a browser to study is a good thing. iPads are somewhat popular in education, and I suspect part of that is having a device that isn’t built around multitasking, and using an iPad means stepping away from regular computing.</p>

<p>To watch videos. This requires very few features, but benefits from just being a separate profile, history, and icon.</p>
<p>There’s a small number of features that might be useful:</p>
<ul>
<li>Cross-service search (like <a href="http://www.canistream.it/">Can I Stream.it</a> or <a href="https://www.justwatch.com/">JustWatch</a>)</li>
<li>Search defaults to video search</li>
<li>Cross-service queue</li>
<li>Quick service-based navigation</li>
</ul>
<p>I realize it’s a lot like Roku in an app.</p>

<p>This is really just about security.</p>
<p>Funny story: people say they value security very highly. But if Mozilla wants to make changes in Firefox that increase security but break some sites – particularly insecure sites – people will then stop using Firefox. They value security highly, but still just below <em>anything at all breaking</em>. This is very frustrating for us.</p>
<p>At the same time, I kind of get it. I’m dorking around on the web and I click through to some dumb site, and I get a big ol’ warning or a blank page or some other weirdness. I didn’t even care about the page or its security, and here my browser is trying to make me care.</p>
<p>That’s true some of the time, but not others. If you are using Firefox for Finance, or Firefox Super Secure, or whatever we might call it, then you really do care.</p>
<p>There’s a second kind of security implied here as well: security from snooping eyes and on shared computers. <a href="https://support.mozilla.org/en-US/kb/use-master-password-protect-stored-logins">Firefox Master Password</a> is a useful feature here. Generally there’s an opportunity for secure data at rest.</p>
<p>This is also a vehicle for education in computer security, with an audience that we know is interested.</p>

<p>Maybe we work with proxy services. Or just do lots of content blocking. In this browser we let content break (and give a control to load the full content), so long as you start out compact.</p>
<ul>
<li>Cache content that isn’t really supposed to be cached</li>
<li>Don’t load some kinds of content</li>
<li>Block fonts and other seemingly-unimportant content</li>
<li>Monitoring tools to see where bandwidth usage is going</li>
</ul>

<p>Sadly making things for kids is hard, because you are obliged to do all sorts of things if you claim to target children, but you don’t have to do anything if kids just happen to use your tool.</p>
<p>There is an industry of tools in this area that I don’t fully understand, and I’d want to research before thinking about a feature list. But it seems like it comes down to three things:</p>
<ul>
<li>Blocking problematic content</li>
<li>Encouraging positive content</li>
<li>Monitoring tools for parents</li>
</ul>
<p>There’s something very uninspiring about that list, it feels like it’s long on negativity and short on positive engagement. Coming up with an answer to that is not a simple task.</p>

<p>Inspired by a bunch of things:</p>

<p>What would a calm Firefox experience look like? Or maybe it would be better to think about a calm presentation of the web. At some point I wrote out some short pitches:</p>
<ul>
<li>Read without distraction: Read articles like they are articles, not interactive (and manipulative) experiences.</li>
<li>Stay focused on one thing at a time: Instead of a giant list of tabs and alerts telling you what we aren’t doing, automatically focus on the one thing you are doing right now.</li>
<li>Control your notifications: Instead of letting any site poke at you for any reason, notifications are kept to a minimum and batched.</li>
<li>Focused writing: When you need to focus on what you are saying, not what people are saying to you, enter focused writing mode.</li>
<li>Get updates without falling down a news hole: Avoid clickbait, don’t reload pages, just see updates from the sites you trust (relates to <a href="#your-front-page">Your Front Page</a>)</li>
<li>Pomodoro: let yourself get distracted… but only a little bit. The Pomodoro technique helps you switch between periods of focused work and letting yourself relax</li>
<li>Don’t even ask: Do you want notifications from the news site you visited once? Do you want videos to autoplay? Of course not, and we’ll stop even asking.</li>
<li>Suggestion-free browsing: Every page you look at isn’t an invitation to tell you what you should look at next. Remove suggested content, and do what <span class="caps">YOU</span> want to do next. (<a href="https://addons.mozilla.org/en-US/firefox/addon/remove-youtube-s-suggestions/">YouTube example</a>)</li>
</ul>

<p>Not just the conclusion of this list, the conclusion of my work in this area…</p>
<p>Some challenges in the design process:</p>
<ol>
<li>Asking someone to <em>do</em> something new is hard, and unlikely to happen. My previous post (<a href="http://www.ianbicking.org/blog/2019/01/overengaged-knowledge-worker.html">The Over-engaged Knowledge Worker</a>) relates to this tension.</li>
<li>… and yet a “problem” isn’t enough to get someone to do something either.</li>
<li>If someone is consciously <em>and specifically</em> doing some task, then there’s an opportunity.</li>
<li>Creating wholistic solutions is unwelcome, unintuitively each thing that adds to the size of a solution <em>diminishes</em> from the breadth of problems the solution can solve.</li>
<li>… and yet, abstract solutions without any clear suggestion of what they solve aren’t great either!</li>
<li>Figuring out how to package functionality is a big deal.</li>
<li>Approaches that increase the density of information or choices are themselves somewhat burdensome.</li>
<li>… and yet context-sensitive approaches are unpredictable and distracting compared to consistent (if dense) functionality.</li>
<li>I still believe there’s a wealth of material in the <em>content</em> of the pages people encounter. But it’s irregular and hard to understand, it takes concerted and long-term effort to do something here.</li>
<li>Lots of the easy stuff, the roads well traveled, are still hard for a lot of people. Maybe this can be fixed by optimizing current <span class="caps">UI</span>… but I think there’s still room for novel improvements to old ideas.</li>
<li>User research is a really great place to start, but it’s not very prescriptive. It’s mostly problem-finding, not solution-finding.</li>
<li>There’s some kinds of user research I wish I had access to, specifically really low level analysis of behavior. What’s in someone’s mind when they open a new tab, or reuse one? In what order do they scan the <span class="caps">UI</span>? What are mental models of a <span class="caps">URL</span>, of pages and how they change, in what order to people compose (mentally and physically) things they want to share… it feels like it can go on forever, and there would be a ton of detail in the results, but given all the other constraints these insights feel important.</li>
<li>There’s so many variables in an experiment, that it’s hard to know what failures really means. Every experiment that offers a novel experience involves several choices, and any one choice can cause the experiment to fail.</li>
</ol>
<p>As Test Pilot comes to an end, I do find myself asking: is there room for qualitative improvements in desktop browser <span class="caps">UI</span>? Desktop computing is waning. User expectations of a browser are calcified. The only time people make a choice is when something breaks, and the only way to win is to not break anything and hope you competitor <em>does</em> break things.</p>
<p>So, is there room for improvement? Of course there is! The millions of hours spent every day in Firefox alone… <em>this is actually important</em>. Yes, a lot of things are at a local maximum, and we can A/B test little tweaks to get some suboptimal parts to <em>their</em> local maximum. But I do not believe in any way that the browsers we know are the optimal container. The web is bigger than browsers, bigger than desktop or mobile or <span class="caps">VR</span>, and a user agent can do unique things beyond any site or app.</p>
<p>And yet…</p>