SOFEA, AJAX, Full-JAX and the Death of Half-JAX

SOFEA stands for Service Oriented Front End Architecture. It is an architectural pattern first described in a 2007 paper called Life Above the Service Tier. Its main premise is that no aspect of presentation logic runs on the server. For web applications, this implies a smart client application that makes calls to various back-end services. In this architecture the Model, View and Controller are all client-side components. Server-side MVC frameworks are considered an anti-pattern.

In my opinion, the premise of SOFEA is spot on and SOFEA is now my standard for new web applications. In fact, I would argue that SOFEA is one of the most important architectural patterns to come out in recent years. But the paper hasn't received much attention and most Java shops are still not doing SOFEA. So I thought I would take a stab at explaining SOFEA from the perspective of a Java web developer.

As mentioned earlier, SOFEA is now my preferred architecture for new web applications. Whenever I make a sweeping statement like that, I always get a few rebuttals like: "there is no one-size-fits-all architecture", "every project is different", "pick the right tool for the job", yada, yada, yada. Yeah, I know all that. I agree with that. But none-the-less, it's good to have a starting point. So for new web applications, SOFEA is my starting point.

How I arrived at SOFEA

The idea that no aspect of UI logic runs on the server is pretty radical. It practically obsoletes 90% of the web frameworks in use today. So it's important that you understand the reasoning behind SOFEA. So let me explain the history and thought process that has led me to the same conclusion as the authors of the SOFEA paper.

Green Screen Days

When I first entered the field, most applications were green-screen applications. They executed entirely on the server in a closet somewhere. The user sat at a dumb terminal (think monitor and keyboard with really long wire). The entire screen was generated on the server and sent to the user every time the screen needed refreshing or updating. Each user had his own working memory (session state) on the server. In fact, all session state was stored on the server. Dumb terminals can't store anything. They are stateless. So what you had was a dumb, stateless client and a smart, stateful server. All processing and all memory was on the server. Thus every time you added a new client, server CPU and memory requirements went up.

Client/Server

Then came client/server. The client was no longer dumb. With client/server, some of the app runs on the client and some runs on the server. Where to place that division (fatter client versus thinner client) is the decision of the architect. For example, an app may consist of a thin Visual Basic client with most of the business logic running on the server in the form of Oracle stored procedures. With early client/server applications the database server was also the application server and stored procedures were the services. DCOM and CORBA-based application servers came later, which allowed us to write services independent of the database.

Unlike dumb terminal apps, client/server apps do not need to store transient, user-specific data (session state) on the server. They just use Visual Basic objects for that purpose. Thus, the server-side services can be mostly stateless. Stateless services have many advantages. They can be cached, scaled, clustered and failed-over much better than stateful services. And there is no annoying "session time outs". So one could describe this class of client/server apps of that day as follows: a smart, stateful client calling stateless back-end services.

Early Web Apps

Then came the web and web applications. Web apps were great because they did not have to be installed on the client. Web browsers are essentially software-based dumb terminals. Since web browsers (at first) were not able to store session-state, all session-state had to be stored on the server (like the green screen apps). These apps were not very good (UI-wise) compared to client/server apps of the day.

Applets

Then came Java Applets! Applets carried the promise of rich client/server applications - plus they ran in the browser. The UI is written as a Java Applet. Java Applets (like the old VB half of the old client/server apps) run in the client's CPU and RAM and make calls to back-end data services - either directly thru JDBC (2 tier) or indirectly thru some kind of service layer (http services). Also, Applets (again, like the old VB apps) would store the user's session-state in client memory. So one could write an Applet as a smart, stateful client that makes calls to stateless back-end services. All of the benefits of client/server plus all the benefits of the web. What more could anyone want! I was building applications like this in my "Beginning Java" workshop all the way back in 90's. And although the term had not been coined yet, these were the first SOFEA applications.

But all good things come to an end. Applets (and the SOFEA architecture) did not succeed on the Web. Not because it wasn't a good architecture. It was because the Java Browser Plug-in (which is required to run Applets in the browser) didn't live up to its promise. There were subtle bugs and incompatibilities, slow downloads, slow startup, firewall issues, political issues, etc. Also, Sun left it up to the OS vendor to create the Java Plug-In. Thus, if MS or Apple didn't play along, the whole concept fell apart. So, while the architecture (smart, stateful, browser-based client calling stateless back-end services) made perfect sense in principle, in practice it was difficult to create an Applet that ran reliably across varying browsers, OS's and Java Plugin versions. Applets were (and are) a great idea, but the devil is in the details and it just didn't work.

JavaScript

Another candidate technology for creating rich client/server applications in the browser was JavaScript. But at the time, JavaScript had no way (or no well-known way) of talking to back-end data services. Also, like Applets, it was hard to get JavaScript (and its companion technologies CSS and DOM) to run consistently across browsers. Thus, JavaScript was not the answer - at least not yet.

The Dark ages

Sadly, instead of fixing the Java Plugin, the community gave up on rich browser-based clients all together. Instead they threw their energies into developing mainframe-like, server-side frameworks. Applets and the SOFEA architecture they encouraged would languish for the next 10 years. And when the time came, it would not be Sun that brought the promise of Applets to fruition.

So the Java community decided to settle for crappy, dumb terminal style web applications. That is, web applications where all UI logic executes on the server and screens are send back and forth. The Java community spent the next 10 years trying to make dumb terminal style web applications suck less. How? By creating dozens of server-side UI frameworks (AKA server-side MVC frameworks): JSP, Struts, JSF, Portlets, Tapestry, Web Work, Struts 2, Wicket, Rife, Stripes, Grails, Spring MVC, Seam, etc. These are all server-side MVC frameworks. That is, the M, the V and the C all execute in the server's RAM and CPU.

And so we lived with these server-side UI frameworks for 10 years. We got used to them. They became entrenched. The generation who remembered the promise of Applets (or for that matter, Visual Basic client/server) were now retired or working in the mortgage industry. The Java community spent so much time and energy building more and more elaborate server-side UI frameworks that when advances arrived that could finally bring back the promise of Applets, no one too notice.

Enter AJAX and Flex

So what were these advances? There were two:

1. JavaScript got better
2. Flex arrived

How did JavaScript get better?

1. AJAX: All major browsers added the XmlHttpRequest object giving JavaScript the ability to call back-end services
2. More consistent JavaScript support in all major web browsers
3. Cross-browser JavaScript Libraries that hide any remaining browser incompatibilities
4. Dozens of powerful JavaScript UI widget libraries
5. JavaScript tool support (Firebug, Intellij, etc.)
6. And for those that don't like JavaScript, you have tools like the Google Web Toolkit that will compile Java into JavaScript. Think of JavaScript as the new byte-code.

And what about Flex? Flex is pretty much the same concept as Java Applets. It's just implemented better. Flex applications run on the Flash player (AKA the Flash VM). The Flash Player has solved most of the problems that plagued Applets: it installs fast, starts fast, it's reliable and ubiquitous. The programming language is ActionScript (an implementation of ECMAScript) and XML. And it includes a powerful, well thought-out UI framework.

Half-Jax: The Wrong Turn

Getting back to our story, here is what happened next. Developers were so deeply entrenched in their server-side UI frameworks that everyone missed an extremely important point: Server-side UI frameworks were invented to solve a problem that no longer exists - they are no longer needed. It's not that they are good or bad. They just are not needed anymore. With modern JavaScript (or Flex), we can finally deliver on the original promise of Applets. That is: rich, stateful, client applications that run in the browser and call back-end services. We don't need server-side MVC frameworks anymore.

Most developers prefer evolution over revolution. So rather than give up our cherished server-side MVC frameworks we thought it best to layer AJAX on top of server-side MVC. And that exactly what most server-side MVC frameworks did. On the surface, this seems logical. But in practice, what you end up with is a technology stack that is impossibly complicated. I know this from experience. If someone tells you otherwise, do not trust them. Here are examples of two freakishly complicated stacks that some vendors are still trying to push:

• AJAX + JSF + Portlet + IBM Portal Server
• AJAX + SEAM + JSF + Facelets

Server-side UI (MVC) Frameworks

Now I will explain why we don't need server-side MVC frameworks anymore.

The Post-AJAX World. Users want responsiveness. They want rich user interfaces. They want AJAX. Resistance is futile. Like it or not, large quantities of client-side code (in the form of JavaScript and/or Flex) will be downloaded and executed in the browser. This is the reality. This is a given. You can't fight it. Let's call this the AJAX Doctrine. And once you accept this fact you start to realize that many of our assumptions about web development are no longer valid. Things that made sense for server-side MVC frameworks no longer make sense. Below are some specific assumptions that are no longer valid in a post AJAX world:

Server-side Runtime View Generation. With server-side MVC frameworks, the View (the V in MVC) is typically generated at runtime on the server. This may involve one or more of the following technologies: HTML, CSS, JavaScript, JSP, EL, Java and various server-side tag libraries (i.e. JSTL, Struts, JSF). Some of these things run on the server and some run on the client. Essentially, what you have is this: some code that the runs on the server (JSP, tag libraries, Java, server-side UI framework) whose purpose is to generate more code (HTML, CSS, JavaScript) which ultimately is delivered to and executed on the client. This is very complex and (today) mostly unnecessary. How about this for an idea, instead of this list of view technologies:

• HTML, CSS, JavaScript
• JSP
• Server-side tags (Struts, Spring MVC or JSF)
• Java
• Server-side UI framework

• HTML, CSS, JavaScript

Your view then becomes a few static files (an .html file, a .js file and a .css file). These files are stored on the server. They are downloaded to the browser where they are cached and executed.

Server-side MVC Controllers. With server-side MVC frameworks, you have this concept of the server-side controller (the C in MVC). In Struts, the controller is represented by the Struts servlet (the front-controller) and Struts actions (the page controller). The controller takes input from the user, updates the state of the model and then refreshes the view (to reflect model changes), possibly navigating to a different view altogether. The concept of controller in MVC is very similar to the concept of an event handler. In fact, in Swing they are one and the same. As luck would have it, JavaScript and Flex have event handlers. Thus, in a post AJAX world there is little or no need for server-side controllers. Events are handled on the client. Some of those events are completely processed on the client. Others will do some client-side processing but also make calls to back-end services for data. But server-side controllers are no longer needed. They are just extra moving parts that add no value and cause slowness. Thus, for SOFEA apps, any place you used server-side controllers (aka actions) in the past, you would now use client-side event handlers instead.

Server-side MVC Models (and session-scoped data). In MVC, the View component is a rendering of the Model (the M in MVC). Thus, as the View is rendered, it makes calls to the Model object.

With server-side MVC frameworks, the MVC Model is typically stored on the server in one of the web scopes (request, session or application). By keeping the MVC Model on the server, 200 miles from the user, we introduce unnecessary complexity, extra round-trips, sluggishness. For session-scoped MVC Models we have a whole world of hurt: session-time outs, fail-over issues, clustering head-aches and scalability problems. And we loose the extremely well thought out caching architecture of the web. But here is the good news: in a post-AJAX world, the need to manage the MVC Model (and session-state) on the server goes away. The best place to manage the MVC Model (and session-state in general) is on the client - specifically, in JavaScript (or Flex) object variables.

Today, there is little or no reason to use the server's "session scope" feature. Any place you would have used the server's session-scope in the past, just use a client-side JavaScript (or Flex) object instead.

Server-side UI widgets. Many of the server-side MVC frameworks, like JSF, are actually UI widget frameworks. That is: a technology for creating and using server-side UI widgets. They usually include some built-in widgets: a table widget, a tree widget, a calendar widget, a menu widget, etc. Plus they allow you to create your own widgets. This is in contrast to traditional non-web UI toolkits (like Swing or Visual Basic) which are client-side widget frameworks.

But here's the thing: there are now plenty of very good client-side UI widget libraries for the web: GWT, DOJO, Scriptaculous, etc. Thus, in the post AJAX world, there is little or no need to use server-side widget frameworks (like JSF) any more.

SOFEA MVC Recap

Here is how server-side MVC transitions to client-side MVC:

• View: Server-side UI frameworks (like JSF) and template engines (like JSP) are replaced with static .html .css and ,js
• View: Server-side widgets are replaced with client-side widgets
• Controller: Server-side controller frameworks (and their "action" classes) are replaced by client-side event handlers
• Model: Server-side session-scoped objects become client-side JavaScript (or Flex) object variables

A few other points

• SOFEA is not a fat-client architecture. Placing the MVC on the client gives the architect the choice as to how fat or thin the client should be. Ideally, all business logic should be delegated to back-end services, making the client as thin as possible. But there are valid cases for moving some logic to the client. For example: to reduce chattiness, make the application more responsive, support off-line use, etc.
• Stateful Client - Stateless Server. In general, I feel the best SOFEA variant to be one where session-state is stored on the client as JavaScript (or Flex) object variables and back-end business services are stateless. Stateless services provide better support for caching, fail-over and scalability. In general, these services will be serving mostly data not pages or partial pages. i.e. these services will be serving JSON or XML rather, not HTML. Finally, these services will generally adhere to the principles of REST. Note: there are valid cases for stateful back-end services: multi-user games or on-line collaboration applications, where session-state is shared. But in general, I prefer stateless.
• Caching. In general, caching is used much less than it should be. Many developers simply turn off caching because its easier than figuring out how to cache what should be cached without caching what shouldn't be cached. Server-side MVC almost forces this. But SOFEA, by separating application download from data download, makes it easier to separate the cacheable from the non-cacheable:

  Application Download	HTML files	Cacheable
  	JavaScript files	Cacheable
  	CSS files	Cacheable
  	Image files	Cacheable
  Data Download (http responses of service calls)	Stateful Services	Non-Cacheable
  	Stateless Services (TTL = 0)	Non-Cacheable
  	Stateless Services (TTL > 0)	Cacheable

  The net result is that with SOFEA more stuffed gets cached! User response time and load on your servers both go down.

• Browser and Edge Caching. With SOFEA, not only will more stuff be cached, it will be cached closer to the user, specifically in the browser cache or the edge cache (AKA CDN AKA Akamai). If you don't already understand how these technologies work, then now is the time to learn.
• Session Timeouts. Last week I was entering some data into a web app. Then I got side tracked for about 30 minutes. When I came back my session had timed out and I lost all of my data. Arg! With session-state stored on the client (the most common case with SOFEA) you no longer have this problem.
• Session Clustering/Replication. With most of the session-state being stored on the client, complex session clustering technologies are less relevant. More applications will be able to use commodity app servers (like Tomcat or Jetty) to host their applications.
• Server Load. With the application and much of the data cached upstream, session-state stored in the browser, and more logic running in client RAM and CPU, server loads are dramatically reduced. That million dollars you would have spent on server upgrades can now pay for programmer raises and training :).
• SOFEA - Basic HTML Edition. There is nothing in SOFEA that mandates how the UI is created. You could use Flex. Or for AJAX apps you could create 100% of your UI programmatically - using DOM and JavaScript. These are both common. But it is equally valid for a SOFEA app to create its UI with plain HTML (i.e. html tags like input, select, button and textarea). SOFEA is agnostic on this point. One can imagine an application with a very plain UI. This app would make heavy use of JavaScript for data interchange. But little or no JavaScript for fancy UI stuff (i.e. DHTML popups, pull-down menus, etc). I call this SOFEA - Basic HTML Edition.

  This has implications with regards to reliability, accessibility and collaboration with HTML designers.

  • Reliability. Heavy use of client-side JavaScript can make reliability and cross-browser compatibility very difficult (or at least expensive in testing). However, you can roughly categorize your use of client-side JavaScript into one of two categories:
    1. DHTML JavaScript for fancy UI features (like popups)
    2. Invisible JavaScript for behind the scenes data interchange (AJAX) with the server
    In my experience, it is the DHTML JavaScript (not the invisible JavaScript) that is more likely to cause cross-browser and reliability problems.
  • Accessibility. There is nothing in SOFEA that prevents accessibility. Since it is perfectly valid for a SOFEA application's UI to be based on plain HTML form widgets, SOFEA applications can be compatible with screen readers.
  • HTML Designers. There is nothing in SOFEA, per se, that makes it hard to work with HTML designers. Some developers work with HTML designers who provide their deliverable in the form of an html page. The programmer would rather not throw out the html page and start from scratch. So the programmer takes the html page and then adds the "programming" part. (I believe most developers do not work this way. Either there is no designer. Or the designer shares their design with the programmer in other ways: wireframes or photoshop). But, none the less, there are some shops that have HTML designers that present their design - as HTML. The good news is, there is nothing in SOFEA to prevent this division of labor. Some frameworks may make this difficult, but there is nothing is SOFEA that prevents this or makes it difficult.
• What about Constrained Devices? If you have an app running in a browser that does not support JavaScript or Flex (Flex requires the Flash Player) then I would recommend No-JAX. In this case, server-side MVC (non-SOFEA) is not a bad choice.
• SOFEA is not a framework. It is an architecture. There are many frameworks you can use for the server and the client-side of a SOFEA application.
• SOA and Contracts The SOA aspect of SOFEA will protect and leverage your coding investments. Most of your back-end services will be serving up data in the form of services with well-defined service interfaces (contracts). This allows other application to leverage your services. It also allows one to freely change the server technology without breaking the client (and visa versa). As long as the contract remains the same. (Update: See below section: "Dave's Modified Flavor of SOFEA")
• What about Standards? Most SOFEA applications will be based on the following standards: JavaScript, HTML, CSS, HTTP, XML, JSON, REST, SOA and Web Services. Also, a key benefit of standards is to protect your code investment. With SOFEA, these benefits will come primarily from the SOA aspect of SOFEA mentioned above.
• Isn't SOFEA just AJAX? I call it Full-JAX. You can look at SOFEA as taking AJAX to its final conclusion: Full-JAX. You can look at AJAX enabled server-side UI frameworks (Half-JAX) as a learning step on the way to Full-JAX.
• Isn't SOFEA just Client/Server? Basically yes. But it depends on how you define client/server. Some people use the term client/server to mean "two tier" where the only service is a database.
• Flashy UI. SOFEA is a distinct (and orthogonal) concept from "Rich UI". With JavaScript and DHTML (or Flash/Flex) one can create some really snazzy UI's. Personally, I am annoyed by some of this. I like simple and functional. But regardless, SOFEA (and Full-JAX) are more about describing the interactions and responsibilities between client and server. And not so much about fancy UI widgets. Also, most of the reliability/compatibility problems related to JavaScript/CSS/DHTML are usually related to the fancy UI widget stuff and not so much to the client/server communication stuff.

Dave's Modified Flavor of SOFEA

I think the most important part of SOFEA is that the model, view and controller should be client-side components. Whether the server-side services are true services (in the SOA use of the word) is less critical (IMHO). There are many (maybe most) client-server apps where the client and the server form a single, logical application. For these apps, the service (if you can call it that) has only one client. For these types of apps, rather than XML web services, I prefer less "open" but easier to program technologies like GWT-RPC or Flex's RPC (AMF) or JSON. Of course, when you start using GWT-RPC or Flex-AMF, it's not really service-oriented. This is why I prefer the term Full-JAX.

Where is Full-JAX not a Fit

Dave's Preferred SOFEA Technology Stack

I always hate to recommend tools in a blanket fashion. Every project is different and every team is different. With that said, it's not bad to have a starting point. Here are my current preferences:

• Front-end (one of the following)
  • GWT
  • Flex
  • Plain HTML + JavaScript, jQuery + various JavaScript widget libraries
• Back-end services (one of the following)
  • Plain servlets
  • JAXB (optional, only if a lot of XML is used)
  • GWT RPC (GWT only)
  • BlazeDS (Flex only)
• Back-end persistence: JPA

Summary

Trying to layer AJAX on top of existing server-side MVC frameworks (Half-JAX) is a fundamentally flawed design. It's over complicated and creates too many additional moving parts. Full-JAX applications, where the M, V and C run on the client, will provide a better user experience and be simpler to build. The user user wins. The developer wins.

References