Friday, November 30, 2007

JSFUnit Beta 1 is Released !!!

As I write this I look at the title and wonder how many exclamation points I should have used. What I thought would be a tiny little framework has grown into a full-fledged project with a real beta release. We even have live demos.

JBoss JSFUnit is the first open source community dedicated to testing JSF applications. With this release, it is not one, but three different tools:
  • In-container Testing framework This is for testing both client and server side JSF artifacts. Test scope ranges from fine-grained tests of individual classes to full-fledged integration testing of a running JSF application. This is done with no mock objects, so you know you are getting reliable real-world behavior in your tests.
  • Framework for JSF Static Analysis testing This allows you to test your JSF configuration so that you find config problems early.
  • JSFTimer for performance testing of the JSF lifecycle This shows you how long each phase of the JSF lifecycle took for a given request. Of course, it also gives you total JSF lifecycle time. The JSFTimer allows you to write threshold tests that fail whenever a use case doesn't meet your performance standards.
If you want more info, read the other posts in this blog and then take a look at

But now a few more words about the release itself. First, it's called Beta 1. To me, that implies a few things. The first implication is that it's useful. Even before a downloadable release was ready, many developers built the project from subversion and started using it. So I know it meets my main criteria for a beta, which is that it fills a need. It's useful, so by all means, feel free to use it.

The second thing about being a beta release is that it's not done. That is, we know some features are missing. Mainly, JSFUnit needs support for more Ajax components and it needs more deployment tools.

The last thing I want to say about the release is "Thank You". First, thanks goes to Dennis Byrne of ThoughtWorks and Matt Wringe of JBoss. These guys contributed code to the project and continue to be active. We need more help as demand for bug fixes and new features has ballooned even before we ever released. So if you want to help out just send me an email or post on this blog and let me know how you'd like to get involved.

Second, I want to thank Julien Viet, JBoss Portal lead, and Patrick McDonald, my manager at JBoss/Red Hat. These guys have given the high level support that made it possible for me to work on the project in the first place.

Lastly, thanks to all who have been the early, early, early adopters. Your feedback on the JSFUnit forums and through email have been invaluable.

So try it out. Let me know what you think.

So long, and thanks for all the fish,

Stan Silvert

Monday, November 5, 2007

JSFUnit and Grey-box testing

An anonymous viewer posted a question about the nature of JSFUnit. I was going to reply directly to the comment, but I felt that the answer needed its own top-level post since it is so fundamental to the nature of JSFUnit.

Anonymous asks,
Why do you want to implement another testing framework? Why do you not consider WebTest ?
The answer is that frameworks like WebTest, HttpUnit, HTMLUnit, etc. only test the client side. These are pure black-box frameworks for web apps.

With JSFUnit, you get both client side and server side testing in one test. Some call this grey-box testing.

Plus, the JSFUnit API is written specifically for ease of testing everything you would want to know about a JSF application.

For instance, with JSFUnit I can not only submit data on a form, I can also make sure that managed beans were properly updated (using EL expressions).

I can make sure that my component tree on the server side is correct using the actual server-side objects created by JSF. Checking the server-side component tree has three distinct advantages that you don't get with just checking the HTML:
  1. It's much easier as I can make simple JSF API calls instead of checking raw text or the contents of the client-side DOM.
  2. I don't have to rewrite my test when the HTML changes.
  3. I can validate components where rendered=false. In this case, the component is not rendered on the client side, but still has state on the server side.
With access to the FacesContext, I can test navigation from the server side with a simple check of the viewID. If you are only testing the client side, you have to infer the viewID and that inference may become wrong as the application evolves. Again, the test breaks simply because you changed the HTML, not because you changed the application logic.

I could go on and on about server side testing. There is a lot more to be validated such as database state, security state, and even memory usage.

If I want to do the client side (black box) testing, JSFUnit gives full access to the HttpUnit API and the client-side DOM. So that is covered as well. And again, I can even do both black box and white box testing in the same test if I want.

Lastly, JSFUnit includes static analysis tests that do validation of your configuration beyond what JSF itself provides. This lets you know about problems specific to JSF that you can catch long before they show up in some obscure use case at run-time.

Good question Anonymous. So long, and thanks for all the fish,