A place to cache linked articles (think custom and personal wayback machine)
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283848586878889909192
  1. title: Building Large Flask Apps In The Real World
  2. url: https://etscrivner.github.io/posts/2014/10/building-large-flask-apps-in-the-real-world/
  3. hash_url: 23a57db93ad36a37f40e1ff30b5dea2d
  4. <p>Scaling a <a href="http://flask.pocoo.org/">Flask</a> application is no immediately obvious
  5. matter. At plug.dj we had ~22,000 line Flask application. At my previous
  6. employer our Flask application was significantly larger. Ultimately scaling a
  7. code-base is less about the framework used and more about the software design
  8. experience of the developers working on it. Scaling in terms concurrent users
  9. also has little to do with the web framework and more to do with your
  10. understanding of load-balancing, caching, databases, etc. That being said, what
  11. have I learned about how to organize a Flask application to comfortably grow?</p>
  12. <p>Firstly, <a href="https://github.com/imwilsonxu/fbone">fbone</a> and
  13. <a href="https://github.com/cburmeister/flask-bones">flask-bones</a> are great first
  14. approximations. If you're struggling to figure out how to structure your flask
  15. application have a look at those and consider using either one as a template
  16. that you can evolve to your needs. Also, I have to mention
  17. <a href="https://github.com/audreyr/cookiecutter">cookiecutter</a> as a tool for templating
  18. the structure of python applications in general. In terms of the web application
  19. itself you might also consider using
  20. <a href="https://pythonhosted.org/Flask-Classy/">Flask-Classy</a> to build out your views.</p>
  21. <p>Beyond that I hestitate to dictate anything else. There's never a
  22. one-size-fits-all solutions for complex real-world problems like this. There
  23. will never be a substitute for thinking up-front, and deeply at that, about the
  24. organization of your application. The first few organizational decisions will
  25. have ripple effects throughout the lifetime of the code base. Bad decisions can
  26. trap you into a corner. Good decisions can make previously difficult problems
  27. much easier. So instead here are a few heuristics that I've used to kickstart
  28. this process:</p>
  29. <ol>
  30. <li><strong>Think about deployment</strong>. How is it getting to the server? egg, wheel, rpm?
  31. Will there be <a href="http://jenkins-ci.org/">continuous integration</a>? Are you using
  32. <a href="http://www.saltstack.com/">salt</a> or <a href="http://www.puppetlabs.com">puppet</a>? How
  33. you deploy your application will determine what kind of structure you need
  34. and what kind of supporting utilites you may or may not have to write.</li>
  35. <li><strong>Think about app initialization</strong>. Where is the entry point? How are
  36. components initialized and shared? If my user module needs a database
  37. connection how do I ensure that it always gets an initialized database
  38. connection? Do I use singletons? lazy loading? dependency injection? It
  39. depends, and you should always be willing to revisit this decision. Also
  40. think about how you'd do a deploy to a completely uninitailized environment.
  41. How do you initialize the database(s)? Is the app configured by environment
  42. variables or cfg files? How are those being shared and deployed?</li>
  43. <li><strong>Think about resource lifetimes</strong>. Make sure you understand how your
  44. database connections and other resources should be managed within a Flask
  45. application. Typically you should initialize a resource when a request comes
  46. in and tear it down before the response goes out. SQLAlchemy
  47. <a href="http://docs.sqlalchemy.org/en/rel_0_7/orm/session.html#using-thread-local-scope-with-web-applications">explicitly covers</a>
  48. integration with web frameworks in its documentation.</li>
  49. <li><strong>Organize by principle of least suprise</strong>. Ask yourself, "How would I
  50. organize this so that someone using Notepad with a good grasp on the
  51. programming language would be able to find and edit any arbitrary component?"
  52. This is ultimately how your codebase will seem to every new person who
  53. encounters it. For example, if you are asked to modify the function that
  54. geocodes a location and you have no experience with a code base it's
  55. reasonable that you'd look in <em>app.geolocation.utils</em> as a first
  56. approximation. You'd be suprised if instead it were somewhere like
  57. <em>app.auth.models</em>. The first example follows the principle of least surprise.
  58. Reduce the mental strain on yourself and others by sensibly organizing
  59. components into well named modules.</li>
  60. <li><strong>Think about testing</strong>. A focus on testing can help you avoid sticky designs
  61. because they simply become untestable. Organize your tests along the same
  62. lines as your modules so that the corresponding tests for any chunk of code
  63. can easily be found.</li>
  64. <li><strong>Think about logging</strong>. Bugs are going to happen and you're going to need to
  65. gather the information to solve them. Come up with a logging strategy that
  66. covers your whole application and stick to it. You should be able to log data
  67. from anywhere in any module and the logs should indicate exactly where the
  68. data came from. In Python the best way to do this is to initialize a logger
  69. at the top of each .py file, that way you always have access to a logger from
  70. every module.</li>
  71. <li><strong>Think about infrastructure changes</strong>. One of the best design heuristics you
  72. can use is to imagine how you would build your application so that arbitrary
  73. third-party dependencies (databases, web frameworks, etc.) could be swapped
  74. out with minimal impact. As your application grows your infrastructure will
  75. change. You should be able to switch databases, web frameworks, or deploy
  76. code to mobile devices with minimal code changes. A good place to start in
  77. figuring out how to do this is the
  78. <a href="http://confreaks.com/videos/759-rubymidwest2011-keynote-architecture-the-lost-years">Architecture The Lost Years</a>
  79. talk by Bob Martin.</li>
  80. </ol>
  81. <p>Each of these topics could easily fill a blog post on its own. With Flask in
  82. particular (2), (3), and (6) are crucial. Flask isn't like Ruby on Rails for a
  83. reason. Flask is designed to be easy to get up and running. It also puts you
  84. closer to WSGI. This, however, is a double-edged sword. It can make development
  85. easier in areas where you know what you're doing while also making it easy to
  86. shoot yourself in the foot in the areas where you don't.</p>
  87. <p>In the end you should be aiming to design your application to depend on Flask as
  88. little as possible. The framework shouldn't dictate your application design, and
  89. microframeworks in particular try to avoid doing this as much as possible.
  90. Recently even Flask has felt bulky. <a href="http://falconframework.org/">Falcon</a> seems
  91. like a good step in the direction of something smaller.</p>