The Gaslight Anthem's The '59 Sound has been on repeat around here for weeks.

Recommended if you like Bruce Springsteen, Bad Religion, Rancid, The Clash, The Kaiser Chiefs.

Thankfully, most of what I'm about to describe will become illegal on July 1, 2010. But until then, you may want to be watchful of the following, especially if you're a current U.S. Bank customer. And even after that point, you'll still need to be careful not to opt-in to a policy that allows banks to do this to you.

All of the below is on an old checking account that should have been closed long ago, but was left open to make sure all automated charges started getting charged to a new bank. My mistake.

Day 1 - Account balance: $21.01. Automated online check card charge for $30 comes in, and the bank pays it (I'd have preferred it to bounce so that the vendor would contact me for updated payment info). New balance: $-21.01.

Day 2 - Initial overdraft charge: $19.00. New balance: $-40.01. Just so we get this right, the bank charged a fee of $19 to cover a deficit of $21. In effect, this is a 90% interest, 1-day loan. This is already unethical in my view, but it gets worse.

Day 3 - U.S. Bank Online Service Fee: $3.95. They charge almost $4 a month to have access to account information via a website. Is this 1995? New balance: -$43.96.

Day 5 - "Continuous Overdraft" Charge: $8.00. They still haven't contacted me to let me know about the overdraft (the notice is apparently in the mail), but they begin charging $8 per day at this point. New balance: -$51.96.

Day 6 - I receive notice of the original -$21.01 via postal mail. I receive no email notice, despite them having my email on file. Again, is this 1995?

Day 7-9 - The bank is closed (Saturday, Sunday, MLK Day). It's not physically possible for me to come into the bank to clear the negative balance, so I wait.

Day 10 - I visit the bank to find that they've charged an additional 4 days of continuous overdraft charges, despite the fact that the bank was closed all of those days. New balance: -$83.96.

In summary: I've been charged almost 200% interest for a $21.00, 10-day loan I never requested. Even those questionable payday loan outlets have better rates than that. Banks like to pretend that this is a "service" that helps me being embarrassed by a vendor getting notice that may payment was refused, but that's a lie. If they would have refused to pay the original $30 charge, the vendor would nicely contacted me for updated payment info. But the bank chose to pay the charge on my behalf because it gave them an opportunity to take $59 from me. This "service" nets banks tens of billions in revenue every year, and the majority of it isn't stolen from abandoned checking accounts like mine, but from primary accounts of people who are barely making ends meet.

I ask the bank manager to reverse the charges. She has to ask the branch manager of the branch where the account was opened to reverse the charges (what a beautiful corporate bureaucracy that is). The other branch manager refused to reverse the charges. Her reason: because I want to close the account. In other words, the bank doesn't want to be fair and make me happy because I might not give them any more money in the future.

Well, turn that "might not give them any more money" into a "definitely won't ever, now that you've stolen from me", and you get the picture. Oh, and I'll blog about it so you might get less money from other customers as well. Thankfully, this is not 1995, and information travels quickly. With any luck, this post will make its rounds to thousands of other U.S. Bank customers who might decide to close their accounts just like I did.

If you're interested in supporting some pending legislation on this topic, a number of bills in Congress are addressing this and similar issues.

Lots of people seem to have been inspired by Tim Bray's Doing it Wrong post just like I was. There's a great response on rc3.org titled "Do you want to be in the software business?" that expresses a lot of the same issues I raised in The Ruby on Rails CMS Dilemma.

The gist of the post is that you shouldn't commission custom development unless you're going to be "in the software business". If your custom software as intellectual property that is a reason for people to invest in your company, then by all means, build it custom. If it's not, then make due with whatever existing product best fits your needs. Just because you can, doesn't mean you should.

Tim Bray's post on how enterprise systems are "Doing it Wrong" just triggered a realization for me. This whole agile vs. whatever debate has been a kind of red herring for a much larger argument about how businesses themselves ought to operate.

I've seen large companies adopt agile methodologies, and they don't magically transform as a result. Something of the earlier culture persists, and the tendency to overplan, overbuild and generally waste tons and tons of money continues.

The reason is that this whole thing isn't about the methodologies. It's not about the day-to-day operating procedures. It's about the culture of control and hubris inherent in the corporate structure. Big companies, especially successful ones, think they know what's right, and they think their success stands as proof of that skill. They underestimate the role of sheer luck in their past successes, and overestimate their ability to predict the future. So software teams in the enterprise make the mistake of believing that:

1. They know (or can know, with enough requirements gathering) exactly the nature of the problem their software should solve, and
2. The nature of that problem won't change over time. Or, it won't change fast enough to matter during the course of development to matter. Or, it will change but they'll be capable of recognizing and adjusting accordingly.

The world is much more complex and unpredictable than this, and these beliefs have become the downfall of many projects. The economist-philosopher F.A. Hayek explained the difficulty of central planning (he's writing about the economics of the entire economy, but I think this reasoning also applies within an individual organization, especially in enterprise software):

If we possess all the relevant information, if we can start out from a given system of preferences, and if we command complete knowledge of available means, the problem which remains is purely one of logic...

This, however, is emphatically not the economic problem which society faces... the "data" from which the economic calculus starts are never for the whole society "given" to a single mind which could work out the implications and can never be so given.

-- from the article The Use of Knowledge in Society

It's impossible for a single mind (or even a small group of minds) to encapsulate all the knowledge of complex enterprise operations. So a single software project should never even hope to be a perfect solution. Because of the fundamental information problem, even the requirements document could never be a perfect design for a single point in time. I believe this is the explanation for Galls Law.

Hayek goes on to explain that "emergent" order is superior (i.e. more efficient) to the order that central planners try to impose from above. We in western capitalist countries seem to know this in our bones in the post-communist era, but Hayek's analysis is particularly conclusive.

Before we turn too sharply into classical economic thought, and before I start re-reading Adam Smith and Karl Marx, let me bring this back on topic.

When you view software development through this Hayekian lens, the benefits of agile methodologies really become apparent. By avoiding upfront design, the software is allowed to emerge from authentic, real-world interactions, the same way economic order emerges in capitalist economies. By shortening the feedback loop between conception, construction, testing and usage, you're forced to start relying on empirical observations rather than predictions. By using dynamic languages and DSLs, you avoid the "information problem" (erg, more economics jargon) in translating human concepts into computing concepts.

So agile programming ideas make sense as potential solution in this whole enterprise software debacle. And Bray speaks of "bridging the gap" between the elephantoid enterprise software world and the grasshopper-like "web world". He closes with this word of advice:

[Successful systems like Twitter] simply cannot be built if you start with large formal specifications and fixed-price contracts and change-control procedures and so on. So if your enterprise wants the sort of outcomes we’re seeing on the Web (and a lot more should), you’re going to have to adopt some of the cultures and technologies that got them built.

He's right in the observation, but I think he's under-prescribing on the solution. It's going to take a lot more than adopting agile for in-house enterprise development at Citibank to start performing on par with Google's programming teams, for example. It's the corporate culture, not the development culture, that's to blame for these problems. It's the hubris of the organization that causes overspecification of documents, not the hubris of the software developer. In practical terms, it's the CEO, not the CTO that's got to lead the type of gap-bridging that Bray is speaking of.

Large companies (or "Enterprises", I guess. I can't seem to use that term without parentheses or quotes... it's like the term itself is an ironic joke) won't start making better use of IT until they realize that great software is a process not a project. And the process has to start outside the IT department if it has any chance of being successful.

Apple's iPhone 3Gs has had a bit of a muted--dare I say disappointing--reception. Unlike with the 3G launch, very few of the new OS 3.0 features depend on the 3Gs model, and for most iPhone users (for me at least), there aren't compelling reasons to upgrade.

None of that matters, though. Apple's going to make far more money for another reason: they didn't discontinue sales of the regular iPhone 3G. Through AT&T subsidies, you can now get an iPhone 3G for $99. That puts the iPhone in the same price category as an iPod Shuffle!!!

While Palm, Microsoft and the phone manufacturers try to figure out how to build a better (more feature-filled, expensive) iPhone, Apple snatched the floor out from under the market. There's very little distinction now between "regular" phones and "smart" phones in terms of price now, and Apple is making the best smart phone at the cheapest price. I wouldn't be surprised to see iPhones completely dominating the entire phone market soon, forget about dominance in the smartphone category.

Update: I've since realized that the iPhone 3Gs might be what Dan Ariely calls a "decoy choice". The expensive, top-of-the-line 3Gs causes the iPhone 3G to appear as a middle-of-the-road, value-conscious choice, by virtue of its straightforward comparison with the more expensive model. Ariely's book Predictably Irrational, describes Williams Sonoma's successful use of this technique to sell the first bread machines.

Ruby on Rails is an excellent framework for building web applications. Perhaps the best. But it's not currently very well suited to what I call web sites.

The difference is simple. In a web site, the unique business value comes from the content creators (authors, bloggers, photographers, etc). In a web application, the business value comes directly from the programmers. Twitter, Google, Basecamp and eBay are web applications. CBSSports.com, KentuckyDerby.com, corporate brand sites and original news sources are all web sites.

For these projects where it's the content that matters, there's no reason to spend time hand-coding models, controllers, views and test code for things called Pages, Articles, BlogPosts, CalendarEntries, and all the other unique types of content. You're just re-inventing the wheel if you do, because there are numerous existing commercial and open source CMS tools that can do the job cheaper and better than you.

Unfortunately, I find that one of the most common scenarios is a hybrid web site that needs an attached web application. A corporate web site that needs an attached scheduling tool. A news site that needs an interactive, custom-searchable database. A sports site which aggregates content from an old NNTP-based data source.

Depending on the specifics, you usually have 3 different approaches to this problem:

1. Two side-by-side applications. - Pair an off-the-shelf CMS with a custom built application. Copy/paste the visual elements, and loosely wire the two together with a subdomain or some kind of web proxy to share the URL structure
2. Extend the CMS - Build a proprietary plugin or extension for the CMS to implement the application features.
3. Soup-to-nuts custom application. Build a complete CMS as part of the project.

If scenario #1 is truly possible, it's the best approach. Unfortunately, it's often the case that the two systems need to share user accounts, group permissions (or worse, business logic) and data. Once you link the code and data of the two applications, you actually have a particularly "smelly" case of scenario #2.

And scenario #2 is usually a bad idea, unless you are really committed to the CMS you've chosen. This approach tightly couples your value-adding intellectual property directly to a CMS that generally has nothing to do with your business. If your CMS vendor is out of business in 5 years, you'll probably need to completely rewrite that custom extension to work in some new system. Unfortunately, this seems to be the most commonly-used approach. Radiant's extensions have this problem, and so does the Block/Portlet system in BrowserCMS.

Finally, scenario #3 is just plain wasteful. If your CMS needs are anything but trivial, you're going to spend a lot of time and money building CMS features that have nothing to do with your business enterprise. Unless you accidentally build an exceptionally good CMS tool that you then sell to others, all that money is wasted.

The reason for this problem is that Ruby on Rails is a framework for applications, not systems. The error in all of the reasoning above is that the described CMS project is not a single application, but a system of two (or more) applications.

I've seen some very attractive developments that will help to address this issue in future Rails applications. First, the inclusion of Rails Engines in 2.3 is going to lower the level of coupling for cases where the CMS can be an "engine" within the application. There's some hope that an even more flexible solution may be forthcoming. These solutions help to flip scenario #2 on its head. Unfortunately, this still leaves your application largely dependent on the CMS vendor. You may have trouble customizing the CMS' features without hacking the engine (or "mounted app") code directly.

Over the past few months, I've been working to find better solutions to the issues I've raised here. I'm in the process of building an experimental engines-based CMS plugin, through which I hope to gain some active insight into what makes this problem so difficult. I'll be posting my analysis as it crystalizes. If the CMS turns out to be effective for my uses, I'll be releasing it to GitHub.

For now, I'll simply leave it at this: I believe the solution lies in finding a common "language" for an application to communicate with its plugins, and for those plugins to communicate with one another. In the case of authentication and authorization, this means a plugin needs to know how to ask about the identity and rights of the active user.

Django and Drupal, have made the framework itself answer that question (both Django and Drupal have admin interfaces and user management features somewhat baked in). I'm not convinced that's the best approach, but it seems worthwhile to consider that approach for Rails.

My CMS plugin is opting instead to look for API functions at the controller level to answer these questions (i.e. the plugin assumes you have a current_user() method in ApplicationController, but allows you to customize the name and behavior of that).

Update: Thank you for all the encouraging comments. There are sure some good people out there in the Rails community. I wanted to draw some attention to ferrisoxide's blog post: "Rails is a CMS". This thought is very much along the lines of the experimental development I'm doing. The goal for my approach is to leverage as much of the flexibility of Rails itself for it's feature set.... in that way I'm planning more of a CMS-enabler plugin than an actual out-of-the box CMS.

Update 2: I wanted to also draw attention to Aaron Bedra's comments about CAS, specifically Castronaut as an implementation pattern for scenario #1. It didn't even cross my mind when I was writing this post... and CAS seems to be one of the most under-hyped technologies out there. More on that later.