Fixing MVC in Web Applications

Note: The Model, View, Controller pattern, or "MVC", actually comes in many flavors, but for the purposes of this article, I'm referring to MVC for the Web. Feel free to comment below, but keep in mind that we have a particular viewpoint here.

The Problem: getting MVC wrong is expensive and costs you a lot of money.

I often start working with a new client as a manager or developer and find a Web app with a common set of mistakes. On one hand, I would like to see software developers learn from past mistakes. On the other hand, this pays my bills. In particular, MVC—which has unfairly been getting a bad rap—is thoroughly misunderstood and much of this can be blamed on rubbish examples and tutorials.

Here's a perfect "bad" example from an MVC explanation using Python and Flask:

@app.route('/')
def main_page():
    """Searches the database for entries, then displays them."""
    db = get_db()
    cur = db.execute('select * from entries order by id desc')
    entries = cur.fetchall()
    return render_template('index.html', entries=entries)

In this example, we have a page which finds "entries" and displays them on a Web page. It's entirely possible that you see no problem with this, but it perfectly demonstrates what's wrong with people's understanding of MVC: you've pushed knowledge of your database, your business logic, and your HTML into the controller.

That's what our examples and tutorials are teaching developers. Because of this, when I work with a new client using "MVC", what I really see is this:

def some_controller_method():
    """begin hundreds of lines of code"""

How do I hate thee? Let me count the ways. (with apologies to Elizabeth Barrett Browning). The following comes from years of experience seeing this over and over again.

  • The database or ORM is exposed at the controller level (nooooo!!!!!)
  • You can't reuse that logic outside of the controller
  • You probably can't test that logic outside of a Web context
  • Tests are much slower because of the web context
  • Test coverage suffers because it's hard to test conditional logic in the controller
  • Many parts of that logic get duplicated in other controller methods
  • Different behaviors become tightly coupled, making it hard to develop
  • Performance suffers because you can't cleanly target a layer
  • Integrating new technologies is harder because the layers are tightly coupled
  • ... and so on

But what does this mean in practice? Let's consider a real example I encountered a few years ago.

One company I worked with had all of the above problems. In particular, they had a 3,000 line "method" for searching for products. It conflated many things, including searching for products and filtering the search results. Thus, even though I was tasked with working on it, every time I touched search, it would break filtering. Every time I touched filtering, it would break search. It was slow and tedious to get anything done. It took me some time to separate search and filtering and when I was done, I had a surprise visit from one of the developers of the mobile side. It seems they wanted a proper search engine for ages but never added it because they couldn't separate the searching (which they needed) from the filtering (which they didn't). They thanked me because, after my work, it took them 15 minutes to implement a search engine.

Not getting this right flushes money down the toilet. Often we hear that a new feature will take a lot of time to implement "because the application wasn't designed for this use case" when what is really meant is "because we didn't understand separation of concerns and thus a one week feature will take months."

Note: you can fix this mess, but it takes time and you have to understand what MVC should look like.

A Proper Controller Method

To contrast this, let's look at a the controller for an extremely complex bit of logic from the free MMORPG, Tau Station, using Perl and the Catalyst MVC framework:

sub station ( $self, $c, $station ) : Local Args(1) {
    $c->character->travel_to_station($station);
}

Without going into detail about what travel_to_station does (trust me, it's complex), all we see is a request for a particularly complex action to be performed. In the controller we can see:

  • No knowledge of how the model is constructed
  • No knowledge of how the data will be used
  • Knowledge of the route to this controller method (the Local attribute)

Having the route information hard-coded is a bit unfortunate, but that's easy to work around. However, this controller method merely connects the model to the view, and that's all. That's what you're looking for in MVC. MVC, when appropriate, takes an application with a UI (user interface), and decouples the responsibilities into reasonable layers.

It's now easy to test the logic (you're testing the model, not the controller layer and all of its web context). It's easy to reuse this code. And, if the model is properly built, you have a great separation of concerns that makes it easy to extend, maintain, and repair the application.

Model
The application.
View
What the consumer (user) sees. Often HTML, JSON, or XML.
Controller
A thin layer connecting the View and the Model.

And that's it! There's nothing fancy, but it often requires explanation. The controller, as shown above, should be as thin as possible. The view should only have display logic and the model is ... what, exactly?

The Model

I'll write more about this another time, but the model is the part where everything breaks down. Almost every MVC tutorial that I see confuses the data model with the Model. This is what we get (pseudocode):

method neighbors():
    model     = this.get_orm();
    neighbors = model.table('Characters')
                     .search( location: this.character.location );
    this.template("people", people=neighbors);

Now that looks reasonable. In fact, it seems so sensible that I did something very similar in the Tau Station MMORPG a long time ago. But it was deeply flawed. It turns out there are many cases where you need to know the other characters in your vicinity and this logic should be encapsulated. But it's just one method call, so factoring it out is silly, right?

Wrong. As it turns out, you didn't want to show any characters who hadn't been active in the past day. Well, unless they were NPCs. Or the other characters have something else which makes them "non-visible". Any time we discover a new case of when characters are or are not visible, we can visit this logic in every controller method, or we can abstract it out into a single method:

So let's try it again:

method neighbors():
    model     = this.get_orm();
    neighbors = model.table('Characters').visible_to(this.character);
    this.template("people", people=neighbors);

Ah, that's much better. Except we have exposed our ORM (and thus, our database). We have many data sources which have data in Redis, or configuration files, or a custom cache, or from all sorts of locations that the controller should never, ever know about. Some of my clients with this anti-pattern have worked around this by trying to implant those other data sources directly in their ORM, which simply shuffles the coupling around. The controller should never care about the source of data. The controller should only be asking for the data. Instead:

method neighbors():
    neighbors = this.model('Characters').visible_to(this.character);
    this.template("people", people=neighbors);

Where do the "visible" characters come from? You don't care. By hiding all of these details from the controller, the developers are free to implement the fetching of this data any way they want, so long as they respect the contract originally specified when this method was created.

The model is your business logic. And I'll go a step further and say a good design choice is to make the model provide services the client needs, with a headless application underneath. By creating a this, you can layer a controller and HTML view on it. If you're careful, you can have your controller target multiple views and build a native mobile application on top of it, only requiring a new view. You could create a native client/server desktop application, too. If the controllers are too inflexible, that's OK. They're tiny and easy to skip. Write thin controllers specifically for your mobile and desktop apps, secure in the knowledge that the application logic is all in your headless application.

There any many benefits to this architecture and I'll cover them in a later article. Further, the headless application needs its own architecture. When is the last time you saw an MVC explanation go into detail about the architecture of the model? That's a rare beast indeed and it's why so many companies have themselves mired in technical debt: they get a partial explanation of a complex concept and start building from there..


If you'd like top-notch consulting or training, email me and let's discuss how I can help you. Read my hire me page to learn more about my background.


Copyright © 2018-2019 by Curtis "Ovid" Poe.