AMR rode into the On Demand/Sofware as a Service (SaaS) debate last week with its guns loaded with much-needed frameworks for evaluating SaaS solutions and measuring success of providers in this rapidly emerging enterprise application model.

In one of his first notes on the subject, AMR’s new resident SaaS wrangler, Robert Bois, got beyond the speculators and took aim at the fact that On Demand and SaaS models are inherently different than traditional installed applications. Because of this, the industry requries new metrics for measuring SaaS solutions and provider success.

“[SaaS] has created a new culture within the software community where customer service, user adoption, and easier implementations are the new gold standards,” wrote Bois. “Metrics formerly regulated to the likes of telecommunications companies now pervade the software industry — total contract value, customer retention, and churn rates.” He adds that this is good news for enterprises because it forces solution providers to “put a new premium on customer satisfaction.”

Bois recommends that enterprises adjust their evaluation criteria for SaaS solutions as well. In addition to functionality, Bois suggests that enterprises scrutinize On Demand solutions in the following areas:

• Pricing models (is it user based? is there a startup fee?)
• Service level agreements (is there one and are there explicit rebates?)
• Customer support (is it included in the monthly fee and how comprehensive is it?)

Bois also puts to rest some common misconceptions about SaaS. “While many early SaaS deployments started out as temporary fixes, AMR Research hasn’t found many companies that then dropped their On Demand for on-premise software deployments.”

On a personal side note, I’m glad to see AMR take up this issue. Over the past three months I have challenged all the leading analyst firms to introduce new measures that more accurately reflect new On Demand models. (I even had a personal sit down with AMR President Tony Friscia on this issue.)

My concern? After viewing technology market assessments from both AMR and Forrester, I was concerned that they were measuring and penalizing vendors based on outmoded attributes, such as “percent of revenues from database sales” and “percent of revenues from system integration services.” On Demand and SaaS models eliminate the need for these unnecessary burdens and costs.

Kudos to AMR for embracing the fact that On Demand is shaking the architecture, pricing, and service models of traditional software. (Bonus points to AMR for dedicating a complete practice to SaaS and On Demand.) We’ll keep a hopeful eye out to see if other analysts follow suit.