davidbgk
/
larlet-fr-david-cache

title: Pyxley: Python Powered Dashboards
url: http://multithreaded.stitchfix.com/blog/2015/07/16/pyxley/
hash_url: cbd6db7ad8f2f3c724cc12215f27f754

<p>Web-based dashboards are the most straightforward way to share insights with clients and business partners. For R users, Shiny provides a framework that allows data scientists to create interactive web applications without having to write Javascript, HTML, or CSS. Despite Shiny’s utility and success as a dashboard framework, there is no equivalent in Python. There are packages in development, such as Spyre, but nothing that matches Shiny’s level of customization. We have written a Python package, called <a href="http://github.com/stitchfix/pyxley">Pyxley</a>, to not only help simplify the development of web-applications, but to provide a way to easily incorporate custom Javascript for maximum flexibility. This is enabled through Flask, PyReact, and Pandas.</p>

<h2>Pyxley: React-powered Flask for interactive visualizations and reports</h2>

<p>Most web frameworks follow the model-view-controller (MVC) pattern and lots of people use React.js as the V in MVC. For the M and C, Flask provides a great minimalistic framework for web applications that allows us to get a simple application running with only a little bit of code. Together, Flask and React.js provide a fast and flexible framework with a low barrier to entry. The focus of Pyxley is to provide a set of tools for quickly developing React-powered Flask applications.</p>

<p>The Github repo has a few <a href="https://github.com/stitchfix/pyxley/tree/master/examples">examples</a>, but we are going to walk through the creation of a simple line chart. To get Pyxley running, you need to get a Flask setup going (see github link for details). For this blog, we will focus on the Pyxley components that are needed for a basic dashboard.</p>
<div class="highlight"><pre><code class="language-python" data-lang="python"><span class="kn">from</span> <span class="nn">pyxley</span> <span class="kn">import</span> <span class="n">UILayout</span>
<span class="kn">from</span> <span class="nn">pyxley.filters</span> <span class="kn">import</span> <span class="n">SelectButton</span>
<span class="kn">from</span> <span class="nn">pyxley.charts.mg</span> <span class="kn">import</span> <span class="n">LineChart</span><span class="p">,</span> <span class="n">Figure</span>
<span class="kn">from</span> <span class="nn">pyxley.utils</span> <span class="kn">import</span> <span class="n">FilterFrame</span>
</code></pre></div>
<h3>UI</h3>

<p>Pyxley takes a modular approach where the developer specifies a list of filters and charts. We’ve created a class called <code>UILayout</code> that ties together all of the components for the user interface. This class is a wrapper for a parent React component composed of filters and charts. As such, we will need to provide the name of the parent component (e.g. <code>FilterChart</code>), the location of the source Javascript, and an HTML <code>div</code> name. The specifics of <code>FilterChart</code> can be found in the
<a href="https://github.com/stitchfix/pyxleyJS/blob/master/src/layouts.js">pyxleyJS</a> repository.</p>
<div class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">ui</span> <span class="o">=</span> <span class="n">UILayout</span><span class="p">(</span>
    <span class="s">"FilterChart"</span><span class="p">,</span>
    <span class="s">"./static/bower_components/pyxley/build/pyxley.js"</span><span class="p">,</span>
    <span class="s">"component_id"</span><span class="p">)</span>
</code></pre></div>
<h3>Filters</h3>

<p>Pyxley is built upon the idea that you have some dataframe in memory that you want to filter with some user selected options. For this example, we have some activity data taken from my Fitbit. Assume for the sake of simplicity that the dataframe has been formatted so that it has the following columns: <code>Date</code>, <code>Metric</code>, and <code>Value</code>. The <code>Metric</code> column is a string indicating the type of data to display (e.g. <code>Steps</code>, <code>Calories Burned</code>), while <code>Date</code> and <code>Value</code> provide the actual time-series. This will allow us to select the desired <code>Metric</code> to plot.</p>
<div class="highlight"><pre><code class="language-python" data-lang="python"><span class="c"># items contains the possible selections</span>
<span class="c"># Args: Label, Items, ColumnName, Default</span>
<span class="n">btn</span> <span class="o">=</span> <span class="n">SelectButton</span><span class="p">(</span><span class="s">"Select Metric"</span><span class="p">,</span> <span class="n">items</span><span class="p">,</span> <span class="s">"Metric"</span><span class="p">,</span> <span class="s">"Steps"</span><span class="p">)</span>
<span class="n">ui</span><span class="o">.</span><span class="n">add_filter</span><span class="p">(</span><span class="n">btn</span><span class="p">)</span>
</code></pre></div>
<p><img src="http://multithreaded.stitchfix.com/assets/images/blog/pyxley_dropdown.png" alt="Dropdown"/></p>

<p>We now have a button, labeled <code>Select Metric</code>, containing a drop-down list of items that we will use to filter the dataframe.</p>

<h3>Charts</h3>

<p>MetricsGraphics.js is an awesome library for making beautiful D3 charts. We've written a few wrappers to make it easier to integrate in our app. For those of you familiar with matplotlib, we are going to write code that is somewhat similar to the mpl API. We start with a <code>Figure</code> object and supply two things: a unique path that Flask will use for request routing and a unique element id that will be used by the React component. The figure is customizable, so let’s code up the desired aesthetics via Pyxley’s API.</p>
<div class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">fig</span> <span class="o">=</span> <span class="n">Figure</span><span class="p">(</span><span class="s">"/mgchart/"</span><span class="p">,</span> <span class="s">"mychartid"</span><span class="p">)</span>
<span class="n">fig</span><span class="o">.</span><span class="n">graphics</span><span class="o">.</span><span class="n">transition_on_update</span><span class="p">(</span><span class="bp">True</span><span class="p">)</span>
<span class="n">fig</span><span class="o">.</span><span class="n">graphics</span><span class="o">.</span><span class="n">animate_on_load</span><span class="p">()</span>
<span class="n">fig</span><span class="o">.</span><span class="n">layout</span><span class="o">.</span><span class="n">set_size</span><span class="p">(</span><span class="n">width</span><span class="o">=</span><span class="mi">450</span><span class="p">,</span> <span class="n">height</span><span class="o">=</span><span class="mi">200</span><span class="p">)</span>
<span class="n">fig</span><span class="o">.</span><span class="n">layout</span><span class="o">.</span><span class="n">set_margin</span><span class="p">(</span><span class="n">left</span><span class="o">=</span><span class="mi">40</span><span class="p">,</span> <span class="n">right</span><span class="o">=</span><span class="mi">40</span><span class="p">)</span>
</code></pre></div>
<p>All that remains is to pass our <code>Figure</code> object to the <code>LineChart</code> class and add it to the UI. In addition, we’ll supply the column name of the x-axis, a list of columns for the y-axis, some initial filter parameters, and provide an indicator that we are plotting a time-series.</p>
<div class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">lc</span> <span class="o">=</span> <span class="n">LineChart</span><span class="p">(</span><span class="n">df</span><span class="p">,</span> <span class="n">fig</span><span class="p">,</span> <span class="s">"Date"</span><span class="p">,</span> <span class="p">[</span><span class="s">"Value"</span><span class="p">],</span> <span class="n">init_params</span><span class="o">=</span><span class="p">{</span><span class="s">"Metric"</span><span class="p">:</span> <span class="s">"Steps"</span><span class="p">},</span> <span class="n">timeseries</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="n">ui</span><span class="o">.</span><span class="n">add_chart</span><span class="p">(</span><span class="n">lc</span><span class="p">)</span>
</code></pre></div>
<p><br/></p>

<p><img src="http://multithreaded.stitchfix.com/assets/images/blog/pyxley_chart.png" alt="Chart"/></p>

<h3>Transform to Javascript via PyReact</h3>

<p>The final step is to compile the filters and charts into a single React jsx file, register it with the Flask app, and transform the jsx to Javascript using PyReact.</p>
<div class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">ui</span><span class="o">.</span><span class="n">render_layout</span><span class="p">(</span><span class="n">app</span><span class="p">,</span> <span class="s">"./static/layout.js"</span><span class="p">)</span>
</code></pre></div>
<p>Including only the components mentioned above, we should get something that looks like:</p>

<p><img src="http://multithreaded.stitchfix.com/assets/images/blog/pyxley_final.png" alt=""/></p>

<h1>How Does it Work?</h1>

<h2>Web Applications in Python</h2>

<p>A very simple web application in Python might contain the following:</p>

<ol>
<li>Data source (e.g. SQLite database)</li>
<li>HTML Template</li>
<li>Request dispatching</li>
<li>Javascript visualizations</li>
</ol>

<p>Flask provides a great framework for handling the first three components. It’s very easy to set up an application to serve some static content such as a table or a plot. However, any interactive components are left to the developer to implement in Javascript. Pyxley helps bridge the gap and provides a set of Flask helper functions and wrappers for some basic Javascript widgets and charts.</p>

<h3>React to the Rescue!</h3>

<p>React.js is the perfect front-end complement to Flask. In addition to providing a succinct way of composing the UI elements of an application, React provides a simple structure through which we will integrate with Python. We’ve provided some basic React components in <a href="https://github.com/stitchfix/pyxleyJS">PyxleyJS</a> that will handle most of the interaction between UI elements. The specifics of each component are passed through as React props. Integration with Python only requires a single template which is compiled into Javascript using PyReact.</p>

<h3>PyxleyJS</h3>

<p>We've simplified the process by making the assumption that most dashboards are a combination of filters and charts. Several useful components have been included in PyxleyJS. These components follow a very simple pattern: the user supplies props that determine which component is created. Let's look at the <code>Chart</code> interface.</p>
<div class="highlight"><pre><code class="language-javascript" data-lang="javascript"><span class="kr">const</span> <span class="nx">Chart</span> <span class="o">=</span> <span class="nx">React</span><span class="p">.</span><span class="nx">createClass</span><span class="p">({</span>
    <span class="nx">getDefaultProps</span><span class="o">:</span> <span class="kd">function</span><span class="p">()</span> <span class="p">{</span>
        <span class="k">return</span> <span class="p">{</span>
            <span class="nx">type</span><span class="o">:</span> <span class="nx">React</span><span class="p">.</span><span class="nx">PropTypes</span><span class="p">.</span><span class="nx">string</span><span class="p">.</span><span class="nx">isRequired</span><span class="p">,</span>
            <span class="nx">options</span><span class="o">:</span> <span class="nx">React</span><span class="p">.</span><span class="nx">PropTypes</span><span class="p">.</span><span class="nx">object</span>
        <span class="p">};</span>
    <span class="p">},</span>
    <span class="nx">update</span><span class="o">:</span> <span class="kd">function</span><span class="p">(</span><span class="nx">params</span><span class="p">)</span> <span class="p">{</span>
        <span class="k">this</span><span class="p">.</span><span class="nx">refs</span><span class="p">.</span><span class="nx">chart</span><span class="p">.</span><span class="nx">_update</span><span class="p">(</span><span class="nx">params</span><span class="p">);</span>
    <span class="p">},</span>
    <span class="nx">render</span><span class="o">:</span> <span class="kd">function</span><span class="p">()</span> <span class="p">{</span>
        <span class="kd">var</span> <span class="nx">Z</span> <span class="o">=</span> <span class="nx">ChartFactory</span><span class="p">(</span><span class="k">this</span><span class="p">.</span><span class="nx">props</span><span class="p">.</span><span class="nx">type</span><span class="p">);</span>
        <span class="k">return</span> <span class="p">(</span>
            <span class="o">&lt;</span><span class="nx">Z</span> <span class="nx">ref</span><span class="o">=</span><span class="p">{</span><span class="s2">"chart"</span><span class="p">}</span> <span class="nx">options</span><span class="o">=</span><span class="p">{</span><span class="k">this</span><span class="p">.</span><span class="nx">props</span><span class="p">.</span><span class="nx">options</span><span class="p">}</span> <span class="o">/&gt;</span>
        <span class="p">);</span>
    <span class="p">}</span>
<span class="p">});</span>
</code></pre></div>
<p><code>Chart</code> has two props: <code>type</code> and <code>options</code>. These props are used to determine the specific component we wish to render. In Python, we only need to specify the type and options, reducing the need to write obscure Javascript templates for every charting package. Instead, we simply need to write a single React component that initializes when the component mounts and updates when we call the update method. Following the same pattern we can create the entire UI with just a single React component composed of several charts and filters.</p>

<h3>Flask Integration</h3>

<p>Through PyReact and Jinja2 templating, we now have a way to integrate our React components with Python. The remaining pieces to constructing our web applications are:</p>

<ol>
<li>Setting up RESTful APIs to our data</li>
<li>Providing interfaces to the React components</li>
<li>Collecting and compiling the components</li>
</ol>

<p>We wanted to accomplish the above while hiding as much complexity as possible from the user.
Prioritizing simplicity led to the creation of the following two classes.</p>

<ol>
<li><code>UIComponent</code>
a. Stores component type, options, and request routing functions
b. Chart and Filter are subclasses</li>
<li><code>UILayout</code>
a. Stores charts, filters, and registers routes with the Flask app</li>
</ol>

<p>These classes only require the user to specify a unique identifier for the <code>div</code> component and a unique route for the request. The request route functions are determined by the type of chart or filter being created.</p>

<h2>Pyxley and the Future</h2>

<p><img src="http://multithreaded.stitchfix.com/assets/images/blog/pyxley_tweet.png" alt=""/></p>

<center>@wesmkinn we agree :)</center>

<p>While this initial release has only a few components, it introduces a very simple pattern that is easy to extend with minimal effort. More components will be added over time to both Pyxley and PyxleyJS and we welcome any contributions from the community and hope that this package makes developing dashboards in Python both simple and enjoyable.</p>