I've been fiddling around with this problem for some time now and I finally got all the pieces together so I guess I'd better share my newfound knowledge on these obscure topics that I hope I'll never have to encounter again.
It all started with a new fine and dandy testframework called uispec4j, that we wanted to use for DataCleaners GUI. Uispec4j is supposedly "Java GUI testing made simple" and so they caught our attention because the code coverage of DataCleaner GUI was not that impressive (yet, if you read this blog post and a lot of time has passed, it may hopefully be looking better).
So we started of by creating some neat unittests for DataCleaner GUI, using uispec4j. Hurray. They worked fine and dandy on our Windows development machines so we uploaded them to the repository and into the Continous Integration loop. This is where hell broke loose.
First off, our Continous Integration server was headless (ie. no screens, monitors, displays, whatever, just a remote console). Surely this wouldn't do because uispec4j requires a window manager to use for emulating the Java GUI. Fair enough, I installed X with the Xfce window manager:
apt-get install xorg xfce4
Then came the next problem. When starting X a fatal error occurred, telling me that no screens where installed. That seems fairly reasonable, but what the heck should I do about it? I decided to install a VNC server to host a remote screen. This would hopefully rid me of my troubles, since I didn't have the (physical) room for installing a monitor for the damn thing.
apt-get install vncserver
After configuring the vncserver I tried running my tests... Next obstacle: Telling Java which screen to use. This required to set the DISPLAY environment variable in /etc/profile:
export DISPLAY=:1
Now came the time for some mind-bobbling uispec4j errors. I found out that uispec4j only works with Motif on linux so you had to append "-Dawt.toolkit=sun.awt.motif.MToolkit" to your commandline like this:
mvn install -Dawt.toolkit=sun.awt.motif.MToolkit
every time you need to build the damn thing. Sigh, this wasn't something that my Continous Integration system (Hudson) was built for so I started to edit various batchscripts to see if I appended the damn "-Dawt.toolkit=sun.awt.motif.MToolkit" parameter to my containers startup script it would work, but no. Instead I found out that you could set the MAVEN_OPTS environment variable, so I did that in /etc/profile:
export MAVEN_OPTS="-Dawt.toolkit=sun.awt.motif.MToolkit"
But that didn't work either because Hudson doesn't comply with the damn thing :( I tried to set that "awt.toolkit" system property using some static initializers (which I generally think is a poor, poor, poor thing to do in Java in general), but guess what? Uispec4j is filled with static initializers as well, so that brought me no guarantees whether or not I was the first static initializer run. (edit: Apparently I might be wrong in this claim about uispec4j, check out the comments for more details).
Finally I got a new version of Hudson that had a per-project configuration of MAVEN_OPTS and that did the job. The last issue was actually a JVM issue. I had to change the runtime user of my J2EE container to be the same user that hosts the VNC server instance. If you try to access another users desktop, the JVM turns fatal. So don't touch my desktop or you'll get your fingers burnt!
Ah and a last thing about GUI testing: Make sure to set the Locale in your junit setUp methods or else the unittests won't be portable between computers if they have different languages and you assert on the labels of UI elements.
I once heard a very wise colleague and fellow developer say:
"You should test functionality and domain models through unittesting and test UI through UI!"
...
3 comments:
Hello Kasper,
I am one of the authors of UISpec4J - the one who answered your support questions.
I understand that you had a hard time having your UI tests work in your headless environment, but I feel insulted by your post.
You claim - with an emphasis - that "UISpec4J is filled with static initializers". How long did you study our source code before writing that? Can you support this claim?
As I told you, the awt.toolkit property is a Swing thing. Swing components will get hooked to the UI Toolkit that is statically initialized at the time when they are created. In order to be able to intercept windows and pop-ups without requiring any change in the production code, we have found no simpler solution than to provide our own implementation of this tookit. To set up our own toolkit, and make sure that your Swing components will use it, we need to have it installed before the static Swing default toolkit is initialized, and the awt.toolkit property is proposed by the Swing guys for this purpose.
The interception of Swing windows is a tricky topic, especially when you try to properly handle modal dialogs. We have spent lots of time trying to make this easier for our users, and I'm sorry if you had a bad time making it work in your context. However, please do not be too quick in giving our product a bad reputation.
Régis.
Hi regis
Sorry for offending you, I actually think your testing framework is really really good. If I didn't I would have gone through this length for keeping it in the DataCleaner project :) This blog was written at a time when it (the situation, not just your framework) was causing my really really big headaches aswell though.
Regarding the static initializers - if what you say is true then I'm having a hard time understanding the whole story. I tried statically initializing the awt.toolkit property in my test classes. As I see it no swing or awt classes could have been initialized at this point? Perhaps I'm mistaken (and I'll edit in a short comment on this in the blog) and there's some kind of "hidden" initialization of Swing that one can't interfere with because it's done directly in the JVM startup?
Also, sorry for the late reply - I'm going to look for some kind of "comment notification" thing here on blogger cuz' I only noticed your comment by accident.
OK, I just took a quick look at the source for uispec4j and in the UispecTestCase class there's a static initializer calling Uispec4j.init(). This method in turn accesses the java.awt.Toolkit class which has the static initializer which starts up the whole Swing framework. Because of this chain of initializers it's impossible to make the hack "awt.toolkit hack" (ie. setting the environment variable in the code instead of "outside" the project) inside your uispec4j tests.
Post a Comment