Roberto's API enhancements wish list

Added by Roberto Dominguez, last edited by Roberto Dominguez on Mar 05, 2008  (view change)

Confluence is becoming very popular as an Enterprise knowledge sharing application. Enterprise as in being used in non-technical and more process-driven environments.

As a wiki, Confluence is meant to be a simple, minimalist tool. Challenge is how to fulfill the needs of large and more complex organizations, without compromising the culture/design.

Thing is that we are just talking a segment of Confluence's market. A way of dealing with this challenge is by making it easier to contributors and third party vendors (like myself) to extend Confluence's feature set.

The Approvals Workflow Plugin is a attempt to address some of those needs. However, I am lately finding that significant architectural and API changes are required to continue extending the functionality.

Here is a brain-dump of some ideas/wishes I have came up with during the last few months I've been working on the Approvals Workflow Plugin and the Content Publishing Plugin as well as in other customisation projects.

Attachments as Content Entity Objects

Attachments are content, right? they're versioned, they're indexed, so why aren't they ContentEntityObjects Think of it, we would be able to define permissions and associate properties...

Inter-plugin communication

Understand the security rationale for keeping each plugin with it's own class loader, but there gotta a (clean) way of allowing for plugins talking to each other. Cross-plugin events or services publishing (a la SOAP) are a couple of ways this could be addressed.

Here's an idea on how this could be implemented.

Content Entity Indexing overriding

ContentEntityObject's content are indexed at a very low level of the stack... making it very difficult to play with what we want to index.

Implement content-level permissions throughout the system

The other day I was working on getting the Approvals Workflow Plugin to work on Blog posts. One of the main drivers for that release was to be able to restrict access to un-approved Blog Posts, but after I made all the changes I realized that although ContentPermissionManager handle ContentEntityObjects, the implementation only handles Pages (BTW you might want to vote for CONF-3305).

This a example of how Atlassian can deal with features that might not be required for everybody: Blog Posts permissions might not be a very popular feature and therefore not worth the effort to fully support it, but by implementing the feature at the API/Internal level, it allows other people to provide the required features.

Renderer plugins

Currently there is no (clean) way of defining or overriding existing renderers. Examples: I'd like to override the way headings and tables and attachments are rendered.

Facilitating extending Data model (i.e. Transaction Layers, Backup/Restored adapters)

A lot of plugin's information is stored as Content Properties (through the ContentPropertyManager). Problem is that we are having performance issues because the data is stored as string, and in many cases complex structures have to be XML-serialize/deserialized...

A way of solving this problem is facilitating extending the data model by creating DAOs and having a way of such model be backed-up and restored.

Understand this is not a trivial request (i.e. DB support, product support) but if properly layered and documenting best practices/patterns, the issues/responsibilities could be clearly delimited.

Built-in way of overriding existing XWork actions

There is already a solution that deals with this. It would be nice to get a solution like that built-in into the plugin API.

Custom permissions

Ability to define customer permissions... it's almost there...

Custom notifications (as in email)

Clean way of overriding all the navigation tabs...

Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.

Comments (9)

 

  1. Dec 13, 2007

    David Peterson says:

    Hey Roberto, Some comments for general review :) Renderer plugins Yeah, woul...

    Hey Roberto,

    Some comments for general review

    Renderer plugins

    Yeah, would be handy on occasion, if multiple plugins override the renderer, how do you determine which plugin wins? Perhaps an admin console to allow admins to order them, and/or a way to chain renderers together. If one can't parse something, it gets passed to the next one, and so forth.

    Attachments as Content Entity Objects

    No arguments from me, although I can see why it hasn't been done yet - lots of potential pain upgrading the database/indexes/etc in the conversion process. Lots of gain for the pain though.

    Inter-plugin communication

    I have already implemented a way for plugins to communicate - it's been in use for quite some time now (Reporting/Scaffolding/Linking all use it). A built-in way would be nice, but realistically would require a rewrite of the plugin manager, or using something like the JPF, which allows plugins to specify other plugins they are dependent on and share the class loader. That would be great for other reasons, such as sharing dependent libraries and having the plugin manager automatically install any dependencies for you.

    In any case, if you'd like more info on my existing library, shoot me an email. I need to document it some time anyway...

    Custom Permissions

    Yeah, that would be great, and having them automatically added to the permission management admin pages would be nice too.

    Custom notifications

    Yep.

    Clean way of overriding existing XWork actions

    You've probably already seen my existing workaround. Not exactly clean yet, but I have a few ideas to make it a bit nicer. But essentially, the same problem exists here as for Renderer plugins - how do you determine which plugin wins? That's a design problem, not a technical one...I'm not sure what the best solution is.

    Clean way of overriding all the navigation tabs

    Could be handy I guess. There are a couple of tricks you can use if you want to get your hands a little dirty. Some of them could be put into a library to make things a bit neater and more upgradable.

    1. Dec 14, 2007

      Roberto Dominguez says:

      David, Don't think we have clear solutions yet... just coming up with requireme...

      David,

      Don't think we have clear solutions yet... just coming up with requirements... let's continue the discussion to get a nice proposal for 3.0

      You guys have come up with a couple creative ways on working around some limitations, and that's good... What I'd like to see is Atlassian either implementing built-in solutions, or at least blessing your work-arounds.

      Some specific comments

      On overriding...

      Yeah, that's always a problem, not matter what... I'd like to see a way to either extending (a-lá xwork's extends) or overriding all together... When there are conflicts it'd be nice to have a enable/disable type of console to decide who wins. We can solve lots of conflicts by having Inter-plugin communication.

      Inter-plugin communication

      Don't think a solution would require major changes to the Plugin Manager. My proposal would be coming up with a new type of plugin module, let's call it service for lack of better name (terms are not my forte).

      Service providers would implement the PluginService interface:

      public interface PluginService {
    public Object execute(Map parameters);
}

      Return values would have to be Java standard or Confluence objects.

      in atlassian-plugin.xml:

      ...
    <service key="cheese" class="com.cheese.service.CheeseService"/>

      In the client application:

      (Page) response = (Page)pluginServiceManager.execute("com.foo.cheseplugin:cheese",parameters);

      Hacky? maybe, but minimalist, I am from the Worse is Better school

      1. Dec 14, 2007

        David Peterson says:

        Hi Roberto, Overriding I agree it would be great to have a builtin solution. ...

        Hi Roberto,

        Overriding

        I agree - it would be great to have a built-in solution. For the record, my current workaround basically adds two new elements to the standard XWork XML definition:

        1. <package-override> - This declares that the plugin wants to modify a pre-existing package. If the specified package does not actually exist, an error will be generated. Existing actions and other definitions in the package are not touched unless explicitly overridden with the next element:

        2. <action-override> - This can only occur in a <package-override>, and indicates that the plugin wants to explicitly override an existing action. It can optionally inherit the existing settings and only override specific options, or it can start from scratch with a whole new action definition.

        A couple of enhancements to that I would like to see/implement are:

        1. Allowing definitions of pre- and post-actions for any given action. This would allow you to chain extra functionality before and/or after an existing action. This would let you do stuff like adding a 'force new password' action after logging in, or doing custom work after saving a page.

        2. Some way to handle multiple overrides. This gets tricky though, and as you mentioned, it may come down to having an admin page which lets a real person decide which one should be used. But even then, it will probably be insufficient...

        An alternative would be for Atlassian to come up with an actual custom API for hooking into the page system, which then delegates to XWork (or whatever) for the actual page display. Then, each page could provides some hints about how it could be overridden. Not exactly sure what that would look like, but it could be cleaner than hacking around directly at the XWork layer...

        Inter-Plugin Comms

        Your solution is not bad actually. A couple of potential issues spring to mind:

        1. Error handling. A simple modification, adding a new 'PluginServiceException', thrown by the'execute' methods, would at least allow errors to be passed back up the chain, although I'm not sure what a good solution is if a RuntimeException is thrown - the PluginServiceManager catches it and wraps it in a PluginServiceException maybe?

        However, the main issue is that it would be critical that the original error is at least reported somewhere in full, since it won't make it back to the calling plugin.

        2. What if you don't actually know the name of the plugin you want to communicate with? Take, for example, the Reporting Plugin. It allows you to retrieve information from a variety of sources, including other plugins. Eg:

        {report-info:data:My Scaffold Field}

        The above will look for a Scaffold field called 'My Scaffold Field'. But Reporting doesn't actually explicitly know that the Scaffolding plugin exists. All it does is say 'I'm interested in plugins who support the Supplier interface'. Scaffolding is one of those plugins, so they set up communications. But the above solution would not allow that, since you have to explicitly hard-code the plugin key.

        Granted, you could iterate through all registered plugins and ping them manually, but I don't think that will be a very efficient solution, or even realistic, for that matter.

        Anyway, that's not to say something along the lines of your solution wouldn't be useful. But it won't solve all problems, and in fact doesn't solve the problems I currently use cross-plugin comms for

        1. Dec 14, 2007

          Roberto Dominguez says:

          Overriding Yeah, I like your approach. Since it is Christmas time, let's ask for...

          Overriding

          Yeah, I like your approach. Since it is Christmas time, let's ask for a built in solution

          The pre and post action overriding you're taking about would be really nice. By having a service like this, the need for overriding the actual action would be reduced (not not eliminated).

          For instance, if there was a way to chain the ConfuenceActionSupport's validate we wouldn't need to override the action:

          i.e. in atlassian-plugin.xml:

          <validator action="pagecreateaction" class="com.foo.MyPageValidator"/>

          The Validator interface:

          public interface Validator {
    public void validate(Action action, Map fieldErrors) throws Exception;
}

          in MyPageValidator:

          public void validate(Action action, Map fieldErrors) throws Exception {
    PageCreateAction createAction = (PageCreateAction)action;
    if (somethingIDontLike()) {
       fieldErrors.put("field","I don't like this field");
    } 
}

          Inter-Plugin Comms

          Actually, the "Service Module" would be able to deal with you scenario, see above.

          1. Dec 14, 2007

            David Peterson says:

            Overriding I was thinking more along the lines of defining actual pre and post ...

            Overriding

            I was thinking more along the lines of defining actual pre and post Actions, as opposed to validation (which would also be handy, but would work differently). Any validation which occurs must happen before the original action succeeds in it's action (for example, creating a new page with a title you don't like). Otherwise, the page is already there, breaking the rules, and the user can just close their browser.

            My suggestion was essentially allowing adding extra pages to display before or after the existing action. For example, let's say you wanted to add the following to the login process:

            1. If the user is already logged in, force them to log out before displaying a new login page.
            2. After logging in, check if any messages have been left for the user. If so, display a page with the messages.

            In this scenario, I'd ideally have something like this for my souped-up XWork definition:

            <package-override name="default">
  <action-override name="login" inherit="true">
    <result name="success" type="continueAfter"/>
  </action-override>

  <action-before name="forceLogout" action="login" weight="50" class="com.foo.ForceLogout">
    <result name="input" type="velocity">/path/to/requireLogout.vm</result>
    <result name="success" type="continueBefore"/>
  </action-before>

  <action-after name="displayMessages" action="login" weight="1" class="com.foo.DisplayMessages">
    <result name="input" type="velocity">/path/to/displayMessages.vm</result>
    <result name="success" type="continueAfter"/>
  </action-after>

</package-override>

            If pre and post action order is determined by the weight. If multiple actions have the same weight, the selection is arbitrary. The 'continueBefore' result type will continue to any other 'before' actions, then direct to the original action. After success, the 'after' actions will be processed.

            This does depend on plugins following the rules, but then again, what doesn't. At least it would establish an actual way to achieve it...

  2. Dec 14, 2007

    David Peterson says:

    Hey Roberto, Fun conversation allow me to fire a couple of shots towards your ...

    Hey Roberto,

    Fun conversation - allow me to fire a couple of shots towards your ever-more-armored balloon

    Technically, your solution will work. The main problem with your current inter-plugin comms suggestion is that it is putting the burden on plugin users instead of developers. This has two downsides: a) There are way more users than developers and b) Users don't know why we're asking for a 30-character plugin key every time you do something.

    Here's a more concrete example of how Reporting would have to work for users with the PluginService API. It is a report on children of the current page, retrieving value from a {list-data:User} field, which contains a {user-options} macro allowing the page editor to select a Confluence user.

    Note: the below solution assumes that a separate PluginService class is being created for each type of data that Reporting is dealing with (Pages, Scaffold List Data, Users, etc).

    {report-list}

{local-reporter:net.customware.confluence.plugin.reporting:content:children}

{report-body}
{report-info:net.customware.confluence.plugin.scaffolding:data:My List > net.customware.confluence.plugin.scaffolding:reference:value > net.customware.confluence.plugin.reporting:user:full name}
{report-body}

{report-list}

    And really, that's an extremely simple example of reports. Here's the equivalent with the current system.

    {report-list}

{local-reporter:content:children}

{report-body}
{report-info:data:My List > reference:value > user:full name}
{report-body}

{report-list}

    Requiring plugin keys will add tonnes of unnecessary noise into the system, not to mention requiring hundreds of users to memorise otherwise-uninteresting plugin keys...

    1. Dec 14, 2007

      Roberto Dominguez says:

      Tell you what, let's not fully qualify the services keys, so I can save you 20 c...

      Tell you what, let's not fully qualify the services keys, so I can save you 20 chars

      Yeah, whatever solution has to be backwards compatible... and we're taking about going forward...

      Another idea is creating a services registry type of thing.

      ... but your point actually gave me an idea on how something could be implemented Now.

      It's trick I use in the Approvals Workflow Plugin.

      This is what you'd put in the page:

      {content-reporter:macro=getapprovals|space=DRAFTSDLC|approval=Published}

      (BTW, I am not familiar with the reporting plugin, so feel free to correct my example above)

      this is your ContentReporterMacro code:

      public String execute(Map parameters, String body, RenderContext renderContext) throws MacroException {
   ...
   String macro = (String)parameters.get("macro");
   if (macro != null && macro.length() > 0) {
       macroCall = "{" + macro + ":" + mapToMacroParametersList(parameters) + "}";
       subRenderer.render(macroCall,renderContext,RenderMode.ALL);
       List entities = (List)renderContext.getParam("ENTITIES");
       if (entities != null) {
            doTheMagic();
            ...
       }    
   }
}

      And this is what I'd do on GetApprovalsMacro:

      public String execute(Map parameters, String body, RenderContext renderContext) throws MacroException {
   String space = (String)parameters.get("space");
   String approvalName = (String)parameters.get("approval");
   List approvedPages = approvalsManager.getApprovedPages(space,approvalName);
   List entities = renderContext.getParam("ENTITIES");
   if (entities == null) {
       renderContext.getParams().put("ENTITIES",approvedPages);
   } else {
       entities.addAll(approvedPages);
   }
   return "";
}

      nice uh?

      1. Dec 14, 2007

        David Peterson says:

        That would save in the order of 50 keys, just in my simple example :) But I'm no...

        That would save in the order of 50 keys, just in my simple example But I'm not quite sure how you could not fully qualify the key and still have it work.

        With regards to the other workaround you mentioned, you might be able to get it to work in the context of Reporting, but a) it would be a massive kludge and b) performance would be horrible. It's already slow on some types of reports, particularly over many pages/rows/whatever, and shoving it through the rendering engine every time it evaluates something would be a nightmare...

        Realistically, I'm unlikely to actually switch our existing code to any new solution any time soon - the one we're using is quite reliable, reasonably efficient and works now.

        As I said earlier, your suggestion isn't a bad one, but it won't solve all problems...

        1. Dec 14, 2007

          Roberto Dominguez says:

          Yeah, we don't want to put yet another thing through the rendering engine. That'...

          Yeah, we don't want to put yet another thing through the rendering engine. That's why I am asking for built-in solutions.

          We're getting the ball rolling. Cool thing is that discussion is happening, now we gotta get attention from Atlassian

          Regards,

          Roberto