Fred Beecher did an informal usability test of 2 conversion apps for the iPhone--Convert and Covertbot--and wrote up a thought provoking review regarding the relationship between playfulness and usability. Convertbot could be viewed as the more playful and whimsical of the 2 apps. He writes:

Fun isn’t always the new usable. There are situations in which usability is more important than playfulness and those in which it’s the other way around. The delight that playfulness contributes to an experience depends on the context surrounding that experience.

...

[W]hile playfulness is undoubtedly an important new focus in user experience design, it’s not a panacea. User experience designers need to understand when playfulness is more effective than pure usability in inspiring delight and vice versa.

Fred observed that playfulness wasn't always favored, but that "playfulness is more effective at certain levels of interaction and when there is a clear benefit to the user to learn the mechanics of that playfulness."

With regard to the use case of converting measurements, his small sample favored the less playful interface that required less to learn. Clearly the dimensions of frequency of use, and specificity of the use case are big factors. The quick information need when converting a measurement is nothing like exploring other kinds of information-rich environments. But the exception in his test points out that once she learned how to use the playful app, she preferred it better. What I'm wondering is, how do you determine when playful comes at too large a cost? What is the point where the burden of learning is outweighed by the benefit of use?

He's suggesting that ultimately UX designers need to understand the balance between playfulness and usability, and the goal is to deliver appropriate experiences that achieve sustainability while making users enjoy using the product. Again, I think audience, context, and use case seem to be key factors. Read the full article for more.

http://userexperience.evantageconsulting.com/2009/09/playfulness-usability-context-delightful-user-experience/