Java and then some

It's about the relationship, stupid

2010-12-11T04:23:00.000-08:00

Every year or so, I am overwhelmed by nostalgia for the B BS days and start playing a TradeWars 2002 game (yes, they still exist). I don't stop to think why those ANSI text days were better than the current webiness I enjoy today, but while reading this Un-Marketing book, it hit me - it's about the relationship, stupid.

Back in the "good old days", you dialed into some local guy's computer, tying up your your line and his. There was no anonymous access, so you had to sign up. Usually, this sign up process asked more than your name and phone, but forced you to fill out a quick survey about yourself, or in some cases, write an essay on why you should be allowed access. In several cases, I had ensure my phone number was correct, as the sysop would try to call it, usually within minutes, to verify I existed and have a little chat. The relationship had begun.

From then on, connecting to that BBS was as much a social event as anything. Sure, there were single player games and files to leach or even fidonet-backed "global" discussion groups, but the real fun I found was in the multiplayer games and local discussion groups. Some boards would have real life meetups, where you get to meet that guy who kept podding your CT (TradeWars thing).

The icing on the cake was the sysop chat. At any time, the sysop could decide to be friendly and interrupt whatever you were doing to start chatting. Of course, you could call him as well, resulting in a physical ringing on his end. Not only was it usually the only sort of live chat boards offered, but doing it with the guy who ran the board on which you were dependent gave it extra value. Some times, the chats were simply to tell you he needed to shut down the board to reboot the computer or that he needed the line for something else. Other times, he just wanted to be friendly and find out more information about you. There is nothing like that surge of adrenaline when your screen blanked and then informed you the sysop wanted to chat.

His powers could go even further. Once, after uploading a rather naughty Trojan horse containing code to give our TradeWars characters billions of dollars (ended up crashing the game right after we logged in, actually), the sysop was so pissed that when we tried to log in next, he kept hitting the backspace key when we tried to type in our login name. It goes to show, BBSing was about the relationship between a user and other users, but primarily for me, between the user and the sysop. The sysop was g*d. If pissed that guy off, he'd likely tell the other 5 or so boards in your area, and you'd immediately have no access to anything.

With the Web, it feels like that extra relational dimension just isn't there, or at least it isn't forced on you as it was in the BBS days. I suppose there is social sites like Facebook, Twitter, and Reddit, but it just isn't the same. That said, it is likely better and I'm just being a curmudgeon for not putting myself out there and giving it a proper go. Maybe I should stop muttering about how Twitter has nothing on IRC, and start tweeting every day. Or not. :)

Introducing Maven Trap - the honey pot for Maven malcontents

2009-09-27T02:09:00.000-07:00

For far too many years, Maven 2 has dominated my builds, and I've sat idly by, taking what they've given, meekly replying, "Thank you sir, can I have some more?" I've watched my builds break apart on unmodified code or just hang, I've nearly drown fighting the torrent of jumbled output trying to find the lone test failure, and I've lost my sanity wading through huge XML build files trying to understand why my build is downloading the internet.

Sure, there have been moments of resistance, like when I created a fork of Maven to add features like parallel downloading of artifacts, but the effort in maintaining a fork is too high when, honestly, I'd rather be doing just about anything than hacking build systems. Still, the revolutionary fire never died, despite not having a sustainable outlet.

Like the scrappy fighter that keeps bouncing back after a huge left hook, I'm back, but this time, looking for victory by treating Maven like the unpredictable wild animal it can be. Instead of forking Maven, a fellow Atlassian Sam Le Berrigaud and I have put it in its cage by wrapping the executable with an interceptor framework, allowing modules to control how Maven gets executed and what happens after it gives up the ghost.

Maven Trap is this executable interception framework and three modules:

Console Colorization - Pages of white or green text is hard to read and makes it easy for critical errors to slip by. This module transforms output into ANSI colorized text to make it easier to see errors and other key events.

Always Offline - A build system should not connect the network unless you explicitly give it permission. This module inverts the meaning of the "-o" flag to make the default offline, only to be enabled via the "-o" flag.

YAML POM - While XML is great for programs, it sucks for humans. This module automatically converts your XML pom.xml file into a YAML-formatted pom.yml for either read-only or editable usage.

Since Maven Trap treats Maven as an opaque black box, what it can do is limited, however, it provides low-risk integration points that can be extended fairly easily and should work with most any version of Maven. Each module can be enabled by setting a unique environment variable, giving you full control over which features you want without modifying your Maven installation.

No announcement is complete without a screenshot, so here is all three modules enabled on simple project:

We use this in the Atlassian Plugin SDK, as we include our own customized installation of Maven to try to reduce the Maven hassles that our development community continually faces. If you too are fed up with how Maven is always accessing the internet, produces heaps of unmanageable output, or uses obtuse XML configuration, give Maven Trap a go, or better yet, join GitHub and fork it for full control.

Better server-side JavaScript unit testing

2009-09-24T08:44:00.001-07:00

As part of a project aiming at a server and client-side tag library based Rhino, I thought it might be nice to write my unit tests in JavaScript. A quick Google search later and I find the non-uniquely named JsUnit project, which not only implements *unit in JavaScript, but has Ant and Maven 2 support. However, quite soon, I hit some problems, mainly around very little error reporting and feedback. Few things are as annoying as knowing there is a problem, but having no idea where.

A quick fork later and I now have JsUnit 1.3-db. So far, this fork adds:
* Callstack reporting for errors and failures
* Text test result output, not just XML
* File and line numbers in all error messages
* Upgraded Rhino

Previously, if you hit an error, you'd get this:

[INFO] [jsunit2:jsunit-test {execution: test}]
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE

Then, you'd have to dig into the target/surefire-reports/TEST-MyTestCase.xml to find this:

<testcase name="TestFieldTest.testRender" time="0.006">
   <error message="ReferenceError: "bar" is not defined." type=""/>
 </testcase>

Not helpful. Now, you get this:

[INFO] [jsunit2:jsunit-test {execution: test}]
TestRunner (1 test cases available)
> Starting test suite "TextFieldTest"
- Running test 1: "TextFieldTest.testRender"
ERROR in TextFieldTest.testRender: ReferenceError: "bar" is not defined. (TextField.js:7)

Stack trace: 
1: TextField.js:7
2: TextFieldTest.js:6 (TextFieldTest_testRender)
3: JsUnit.js:827 (TestCase_runTest)
4: JsUnit.js:808 (TestCase_runBare)
5: JsUnit.js:386
6: JsUnit.js:409 (TestResult_runProtected)
7: JsUnit.js:390 (TestResult_run)
8: JsUnit.js:797 (TestCase_run)
9: JsUnit.js:1033 (TestSuite_runTest)
10: JsUnit.js:1014 (TestSuite_run)
11: JsUnit.js:2614 (EmbeddedTextTestRunner_run)
12: Components:4
< Completed test suite "TextFieldTest"
1 error, 0 failures.
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE

For fellow Maven folks, you can find the releases at my repo:

http://twdata-m2-repository.googlecode.com/svn/de/berlios/jsunit/jsunit-maven2-plugin

GWT for JavaScript Junkies

2009-09-05T21:19:00.000-07:00

The most appealing thing about GWT for me isn't the fact that I, as a Java developer, don't have to write JavaScript anymore (hey, I like JavaScript!), but the single development experience. Since everything is in Java, you can have common libraries, coding patterns, frameworks, and editor behavior, drastically simplifying the development experience. My second favorite aspect of GWT is its component library. For far too long, we Java developers have treated HTML as just text to output, which is like treating programming as a series of ones and zeros - surely we can work at a higher level than that.

Therefore, I set out to explore what it would be like to have a GWT-like development experience for JavaScript developers, only instead of Java everywhere, you write JavaScript. Also, I wanted the bulk of the UI to be built on the server-side to minimize XHR calls and improve accessibility. To implement the server-side component library, I ripped of as much of Apache Wicket as I could, while keeping it 100% stateless (no session usage or complicated state sharing schemes). I'm not sure how to package this yet (new framework, Struts 2 plugin, require OSGi, or whatever), but before I go too far down the road of making it a proper framework, I thought I'd get some feedback on the basic design.

This is what the code of a simple application looks like:

/controllers/comment.js
/templates/comment/comment.html

comment.html

The comment.html template is a simple HTML page with "jsid" attribute markers to determine where the components are rendered. Absolutely no logic is allowed in a template:

<html>
  <head>
    // import jquery and json
  </head>
  <body>
    <form jsid="commentForm" action="#">
      <input jsid="title" type="text">
      <input jsid="body" type="text">
      <input jsid="submit" value="Submit" type="submit">
    </form>
    <script type="text/javascript" jsid="proxies"></script>
  <body>
</html>

comment.js

Where it gets interesting is in comments.js. This follows the basic design of Ruby on Rails controllers with some interesting twists:

page({
comment : function(params) {
   this.commentForm = new Form();
   this.commentForm.add(new TextField("title"));

   var body = new TextField("body", {
       value : "Great feature",
       label : "Body",
       description : "The body of the comment"
   });
   this.commentForm.add(body);
   this.commentForm.add(new Submit("submit"));

   this.commentForm.onsubmit = function() {
       alert('calling comment');
       saveComment({
           title : jQuery('#title').val(),
           body  : jQuery('#body').val()
       }, function() {
           alert("Comment saved");
       });
       return false;
   }.toString();

   this.proxies = new Proxies(['saveComment']);
},
saveComment : function(comment) {
   print("comment title: " + comment.title + " body: " + comment.body);
   this.result = "success";
}
});

Ok, starting from the top. You pass into the "page" function a map of methods to be used as controller methods, although there is a mix of HTML and JSON-producing methods: "comment" returns a comment page, while "saveComment" is a JSON-invokable method.

comment()

Since this is the method that will render HTML, we will build up our component tree. Instance variables like 'commentForm' are the first to be matched against comment.html, with subcomponents matching accordingly as the template processor works its way through the HTML. There are four components used in this page:

"Form" - represents an HTML form
"TextField" - represents an HTML text input field
"Submit" - represents a submit button
"Proxies" - generates client-side proxy methods that call the corresponding server-side ones

The main advantage of building your components in server-side code and not in a template is the much greater options you have for presentation logic. In this example, I just use hard-coded strings, but you can see how these could just as easily be initialized using service calls.

Nothing eye-opening so far...until you get down to the onsubmit() method. What we are doing is creating a method that will be executed on the client-side in our server-side code, using the nifty toString() feature on functions to get the source. We could have just as easily wrote it in a big string, but this way we get better editor support and fail-fast parser failure reporting. This also serves to help "blur" the line between server and client-side code ala GWT.

Finally, we create the "Proxies" component, passing the server-side function names that we want to be able call from the client. My implementation uses JQuery and JSON to stringify objects, but those details are irrelevant here. The point is we are able to call "saveComment()" on the client-side as if it was on the server-side, with all the json parsing and construction happening behind the scenes. Since both sides are JavaScript, you don't have the immedience mismatch you do in other languages.

saveComment

The "saveComment()" function is a remotely executable function, meant to receive and return JSON, although you aren't limited to it. JSON in the request body is generally a good idea since it limits your vulnerability to cross-domain request forgery. Of course, this technique also means your form only works with JavaScript turned on, which may not be an option. The "comment" argument is the serialized object passed from the client, and we return a serialized instance of our controller, which is why we set the result by setting the instance variable.

This basically completes the tutorial. One other thing to note is the components are also written in JavaScript, and since that means Rhino, you get all the E4X niceities in constructing HTML. This means even the HTML is syntaxally processed when the script is loaded.

Next steps

My next step is to build a real application using this little framework and see how it scales as the feature demands pile on. Thanks to Rhino, I can keep developing the backend of this application in Java with all the libraries and frameworks I depend on. The consistent use of JavaScript for the presentation layer hopefully means less code, more reuse, and more productivity.

Playing second fiddle - Java backend for Bespin

2009-08-18T17:27:00.001-07:00

So I'm the poor sap who decided to take on building a Java backend for Bespin. Michael Mahemoff wonders if Bespin will ever have multiple backends, and I obviously believe the answer is yes, so I thought I'd go into why I think it is a good idea.

Bespin represents an opportunity to fundamentally change how we develop software. It dramatically lowers the barrier of entry to development, ideal for software modules that are smaller, more focused, and testable. At this point, it isn't suited to all software development, but for extension systems like a good plugin system that allows small bits of code to be plugged into a system at runtime, I think it is a perfect fit.

While Java development, for example, can be quite fast once you get all your frameworks, build systems, and IDE's in place, it also be a house of cards that the slightest network or external server-related wind can bring to a tumbling crash. I'd rather have an application focus on providing a solid platform with accessible API's, then provide a development environment in the app itself to write little plugins. Then, any developer can log in and get to writing real code without all the setup and build issues.

Therefore, building an embeddable Java backend is critical since all the web applications that I work on are Java-based. There are many times I want to write a little plugin, but dealing with Maven 2 and IDEA seems overkill. Being able to log into a dev server and hack away would dramatically reduce the time from idea to code and make it more likely I'll actually do it and not put it on the "I should do that someday" shelf.

They say in order to take advantage of your creative potential, you should carry around a notepad to jot down ideas as you have them. Think of Bespin as a code-able notepad.

Installing OSGi bundles with dependencies

2009-07-25T20:07:00.000-07:00

One of my favorite features of OSGi is you can write jars/bundles that provide code and/or services to other jar/bundles. However, one of my least favorite features of OSGi is how you can't just install a single jar, as most likely, that bundle will require packages and services from other bundles in order to work. Passing a list of bundles with instructions to install them in a certain order sucks.

To help solve this deficiency, there is the OSGi Bundle Repository (OBR), which is a bit like Maven 2's dependency management in that when you install a bundle using it, OBR can look at external repositories to find any required dependencies and install them as well. Unfortunately, a) there is barely any documentation or code samples out there on integrating OBR into your OSGI application and b) having an OSGi app go out to the internet to download other bundles at runtime just plain scares me; take Maven's penchant for downloading the internet and put that "feature" into your production application... *shudder*.

The solution I came up with uses the dependency resolution aspect of OBR but without needing any network access. Instead of installing a bundle directly, you install an ".obr" file, which is a zip file with the following layout:

main-bundle1.jar
obr.xml
dependencies/dep-bundle.jar
dependencies/dep-bundle2.jar

When installed, the installer looks at the obr.xml and determines if the main bundle can be installed as is. If there are any missing dependencies, it takes them from the "dependencies" directory and installs as necessary. Therefore, only the necessary bundles are installed yet OBR doesn't go out to the network, and, bottom line, you can again distribute and install one file.

The obr.xml file is the standard OBR descriptor file that describes the main bundle and all its dependencies. The dependency bundles are packaged in the .obr zip only to be used if needed during installation. For example, if user A installs the .obr file on their osgi system that already contains dep-bundle.jar, then the installation process will only install dep-bundle2.jar. If user B has neither, then the installation process installs both dependency bundles.

There are other options I found (PAR, DeploymentAdmin) for grouping bundles, however, they tend to exist as a way to define multiple bundles as a single, possibly indivisible, unit, but in our case, dependencies are meant to be shared between bundles of different origins. Did I miss anything or is this the best option for achieving my goals?

Greasemonkey for Maven 2

2009-07-17T07:24:00.000-07:00

Few things are more tedious than crawling through hundreds of lines of Maven 2 output to discover which test failed, mojo didn't execute correctly, or error was thrown during integration tests. After using git for my primary version control tool, I'm spoiled with its lovely full-color output, so why can't I have that for Maven 2?

Turns out, there is a ticket open in the Maven JIRA project for this, but so far, there doesn't seem much traction, and fixing it for me now would involve forking Maven again, which I'd rather not. Therefore, I decided to take the Greasemonkey approach and write a wrapper around the Maven output so that I can format it exactly how I want.

Introducing the Console Colorizer, a command line tool for wrapping Maven execution, then feeding its output through a Lexer that adds VT100 color codes where appropriate. This approach is suprior to the usual colorizing the logging output, because I can match any text and highlight it any way I want. Going a bit farther, I could even detect certain bits of output and execute arbitrary code, like sending a Growl or IM notification. All this without modifying any Maven Java code.

Compare this raw Maven 2 output:

With this:

It is so much easier to see that the test failed, and what warnings your Maven plugins are complaining about. This tool works by wrapping the Maven main execution and running the System.out through a JFlex lexer. The lexer is nice because it allows me to define tokens via regular expressions, yet remain very fast and compact. A few VT100 codes allow me to add the coloring.

Of course, this same technique could be applied to any Java program, or any program at all if you really wanted to, but for now, I think it'll do nicely to clean up Maven 2 a bit.

Goodbye JRoller

2009-07-17T07:15:00.000-07:00

After some four or five years, it is time to move off of JRoller. That site goes down more often than a...well, you get the point. Let's see how blogger treats me...