Fedex 8 - Confluence Aspect Expression Language

In the beginning...

For Fedex VII I delivered the future macro: a macro that would defer rendering of its contents until after the Confluence page was loaded by the client. This worked by substituting the content for a place holder that would collect the result via an AJAX call after rendering. The main use-case for this macro was to wrap other macros that may take a while to render. These are typically components that make long running network connections to generate output such as the JIRA issues macro. The problem with these potentially sluggish macros is two-fold:

1. They hold up the delivery of other content on the same page.
2. There's minimal feedback to the user that something is still happening with their request and it hasn't just been lost in the bit bucket.

The Future macro was an initial stab at addressing these issues.

Problem -> Solution -> Problem -> ...

While the Future macro addresses this problem, the application of this solution is where another harsh scenario should be considered: what happens if I already have a Confluence instance that contains hundreds or even thousands of these slow macros. How do I take advantage of something like the future macro (or cache macro) given this situation? Nobody really wants to go through their entire instance and manually edit wiki source to add this feature and while a wiki community might eventually reach the desired results through a slow progression of edits there should be an easier, quicker and more unobtrusive way. What are the traditional options?

1. Write a batch script, either at the RPC or database level with some gnarly regex replacements.
2. Alter the JIRA issues macro itself to deliver a built in option for asynchronous rendering.
3. Add some other static means for a macro author to nominate another macro to be a filter if it's available.

None of these options are particularly exciting; batch scripts are hard to maintain, inflexible beasts; altering the macro themselves could result in much duplicated work, and doesn't cater for any other new and exciting filter macros that might be provided in the future. Option three seems a little hacky and isn't easily adaptable to the needs of different instances and user bases.

What's a nerd to do?

Help! I've got an aspect and everything looks like a cross-cutting concern

What if there were a way for an administrator to write some rules to tell Confluence to apply certain macros in certain situations? What if this could be expanded to allow a full range of dynamic manipulations of macros in the render pipeline? Useful or bat-shit loco? I decided to conduct an experiment for Fedex VIII.

The way I chose to look at the problem was to borrow a few concepts from the land of Aspect Oriented Programming. Here's a brief rundown of key AOP concepts:

• Join point: A join point is any part of an execution flow that might be interesting to intercept programmatically.
• Pointcut: A pointcut is a description specifies a set of particular join points. A pointcut might select a join point via method name matching or stack inspection.
• Advice: Is the name given to code that can be used to provide additional behaviour during the execution of a join point.
• Aspect: An aspect is the application of advice at a join points selected by a pointcut.

If we look at Confluence in terms of these concepts we could come up with some rough mappings:

• Join point: Some part of Confluence execution that we might want to intercept and take action at; macro executions in this case.
• Pointcut: A set of expressions to allow us to target which macro executions we wish to modify. In AOP, a pointcut expression language provides a set of primitives that can be used to select interesting join points. In terms of macros those primitives might include the page title, space key and user name.
• Advice: Given the ability to target certain macro executions based on these primitives, what would be some useful actions to take? The prime motivation at the start of this project was to wrap the target macro in another macro. Other ideas include skipping or replacing the macro execution entirely. Maybe we'd also like to log a message or update some runtime statistics.

I began to form up some details at this point given the ideas above. Now for a blow by blow account of progress over the course of Fedex day.

And they're off

The first step at 2pm was to see just how feasible this would be – there was a chance that the renderer architecture would not abide such a modification to the way it works. I spent the first three hours of Fedex splicing in a crude modification to the DefaultMacroManager that would evaluate a static pointcut specification and provide some advice that would replace a selected macro with the text "I've been advised!". It was at this point that I also had to decide on what the pointcut expression language would look like. I decided to use the JEXL expression language as it allowed me to easily populate a context with some pointcut primitive beans.

Then it was 5pm and home time: the crude, prototypical implementation was working, proving that the concept was at least feasible from an architectural point of view.

I said "do you speak-ah my language?"

The next hurdle would be coming up with an easy to grasp way of specifying Confluence aspects. Using AspectJ's concise syntax as a model I came up with the following draft syntax:

pointcut busyTime: time.between('14:00', '18:00');
pointcut jiraMacro: macro.name('jiraissues');

aspect futureMe: pc.name('jiraMacro') && pc.name('busyTime') && conf.userName('cowen') {
	macro.filter('future:title=JIRA issues')
}

aspect futureAgain: pc.name('jiraMacro') && pc.name('busyTime') {
	macro.replace('note', 'This macro has been disabled')
}

I wrestled at home with ANTLR from about 9pm to midnight, trying to author an ANTLR specification to support this language. I managed to get a lexer up and running in ANTLRworks that was breaking up the tokens correctly but I couldn't get it to actually produce any Java code to do it. After venturing into the parser side of ANTLR a little more I eventually threw in the towel at midnight and went to bed, thoroughly depressed and convinced that the project was going to end in failure.

After about one and a half hours of restlessness and insomnia, with my old compiler design lecturer uttering incomprehensible things in my head over and over, another faint voice emerged, took over and whispered: "XML. It's the answer to all incompetence." The voice was correct of course – I couldn't believe that this hadn't been apparent from the start. Exhausted I finally slept and was haunted by dreams of angle brackets. I awoke in a lather at about 6:30am, terrified that I hadn't properly closed the sleep start element.

He just smiled and gave me...

... an XML sandwich. I was back in the office at 8am and was feverishly trying to get together an XML representation for the aspect configuration and write code that would process it. By about 11am the class for parsing the aspect specification into a useable domain model was complete. The result was something like the following, not dissimilar from Spring XML aspect definitions:

<aspects>
    <pointcut name="busyTime">time.between('11:00', '18:00')</pointcut>
    <pointcut name="testingSpace">conf.spaceKey('test')</pointcut>
    <pointcut name="inDemoSpace">conf.spaceKey('ds')</pointcut>
    <aspect name="overviewAdvice">
        <pointcut>pc.name('inDemoSpace') and conf.matches.pageName('.*Overview.*')</pointcut>
        <advice>"&lt;p&gt;This regex advice matched&lt;/p&gt;"</advice>
    </aspect>
    <aspect name="noJIMForExample">
        <pointcut>conf.spaceKey('example') and macro.name('jiraissues') and pc.name('busyTime') and conf.anonymous</pointcut>
        <advice>macro.skip()</advice>
    </aspect>
</aspects>

This example also demonstrates some of the major pointcut expression primitives.

Global pointcuts

1. busyTime – matches when the server local time is between 11am and 6pm.
2. testingSpace and inDemoSpace – matches on the appropriate space keys.

Aspects

1. overviewAspect – this aspect is applied whenever the global testingSpace pointcut matches and the page name contains the string "Overview". The pointcut expression uses the matches extension, an expression that may be used on any pointcut primitive when a regular expression match is desired rather than an exact, case-insensitive string match. This aspect will replace any matching macro with a simple HTML paragraph.
2. noJIMForExample – this aspect will apply to all jiraissues macros in the example space during the time that the busyTime pointcut matches when viewed by anonymous users. It will abort rendering of the jiraissues macro under these conditions.

My kingdom for a filter

One of the most valuable features of the system, macro filtering, was still missing at this point. The only tricky part of implementing this was handling the case where the wrapping macro does not process its body. This means that at the point we've intercepted the original macro call, we have to reconstruct the wiki mark-up of the current macro so that the new, wrapping macro can process it properly. It took up until lunch time to get a final implementation of this behaviour, handling both macros with and without bodies.

What good is a phone call if you're unable to speak

After a short lunch I quickly implemented a dynamic runtime for the aspects. This involved providing a simple input mechanism for editing the current aspect configuration at runtime and rebuilding it as required.

The woefully inadequate aspect editing screen

With only a couple of hours left before the 4pm presentation deadline, I set about beefing up the range of primitives available for pointcuts, adding the time primitive and the regular expression matching options among others. I stopped adding new features with little less than an hour remaining and made sure everything was rock solid for the demo.

A macro aspect in action

<aspect name="noIssuesForAnonymous">
    <pointcut>conf.spaceKey('aspect') and conf.anonymous and macro.name('jiraissues')</pointcut>
    <advice>macro.replace('note', 'The JIRA issues macro is not available for anonymous viewing')</advice>
</aspect>

Delivered pointcut primitives

Delivered advice primitives

"About to render " + macro.name; macro.proceed(); "Finished rendering " + macro.name;

The resulting rendered text is simply the string representation of the advice result. All of the advice primitives return strings that are inserted into the rendered output as the final result. Consequently you may simply provide a string literal as simple HTML advice.

Results and future

In the end, the main use-case I had in mind for this system, the dynamic wrapping of the JIRA issues macro with the future or cache macro, worked flawlessly. As Fedex day progressed and I added more primitives to the system it became apparent that this could be a very powerful system for savvy administrators, enabling them to customise the behaviour of macros to address their individual needs. What about power users? What if users could customise the way macros render for themselves? Support engineers? Perhaps the biggest barrier to use would be the specification language. If the aspect primitives could be refined to be as intuitive as possible and a simple, dedicated syntax was to be introduced that would go a long way to making the system easier to use. What about join points other than macro execution? The first thing that leaps to mind is a join point on all of the internal Confluence events, being able to hook them simply and perhaps veto their execution. UI events? I think I feel ill.

In terms of runtime performance, the main overhead is evaluating all of the aspect pointcuts at every macro execution (at least until a match is reached). This means the overhead scales with the number of pointcuts that need evaluating, the number of macros processed and the complexity of any matching advice. Optimisations are possible here such as caching pointcut results, reordered evaluation etc.

Whether or not I continue working on this will depend mostly on public opinion of whether it would be useful or not, and whether the great time monster will stop having his fill.