A trap with aspects

Sometimes you want to write an aspect with behavior that is orthogonal to that in DfSysObject. Nothing wrong with that – we encourage it – but here’s the rub …

When DFC dynamically builds the Java class for an object to which your aspect is attached it dynamically builds an inheritance hierarchy with the TBO and its DFC superclasses as the supermost classes, and with each aspect subclassing each other and ultimately the TBO. DFC literally patches the superclass pointers in the byte code. You are going to subclass DfSysObject and its superclasses whether you want to or not.

Now if you are intending to write an aspect with orthogonal behavior you have to be really careful not to unintentionally override a DfSysObject (or superclass) method, which could have bizarre consequences. A more overt and terminal consequence arises if you unintentionally override a final method – you get the infamous java.lang.VerifyError, although it isn’t related to class loaders this time. An example of this is if you try to use a DfService implementation as an orthogonal aspect implementation (you can’t).

The best tip? Just subclass DfSysObject in your aspect primary class in the first place and the enforced runtime inheritance structure will be reflected at compile time.

Hot deployment in BOF – yes or no?

One of the primary benefits of BOF is hot deployment of updates. But do you get hot deployment for all BOF elements? The answer is no, and you need to understand when you do and when you don’t.

If you look at the BOF class loaders you will see that all module-specific loaders share a common parent – the BOF shared loader. The shared loader is primarily intended to contain module interfaces. There is a second supported case though – non-sandboxed Java libraries. These are used to share implementation jars across modules. This is not a recommended practice, but it does avoid potential duplication of class files in memory (the only reason that it is supported).

It is fatal to update interfaces in a running VM in many cases and ill advised in others, so the expectation is that classes in the shared loader are not dynamic. Anything that is loaded by a module-specific loader is subject to hot deployment, but anything loaded by the shared loader does not enjoy hot deployment. The main reason for this is that invalidating this class loader would mean invalidating all existing module-specific class loaders, possibly resulting in duplicates for some time as modules are garbage collected which could be a significant memory cost. Regardless of the reason, it is a fact.

The take home lesson is that if you use non-sandboxed Java libraries you will not get hot deployment for those libraries. At the cost of some class file duplication you can get all the BOF benefits including hot deployment if you sandbox your Java libraries, and we strongly recommend this. Note that an update to a sandboxed Java library is propagated to all consuming modules via hot deployment.

As an aside, be careful to avoid caching module instances because these instances are immune to hot deployment.

Using aspects with non-sysobjects

The aspect mechanism requires support that is built into dm_sysobject. Persistent objects that are not sysobjects, by default, cannot have aspects attached because they lack this support. You can add this support to a non-sysobject type using the following DQL

alter type my_type allow aspects

You can’t do this to standard Documentum types though.

java.lang.VerifyError and the whole truth

The BOF class loader diagrams I have published before contain a simplification that I had hoped to finesse, but reality seems to have caught up with me.

The whole truth is that the sandboxing BOF class loaders aren’t strictly parent last. Their behavior is:

1. Java extension loader
2. The BOF sandboxed loader
3. The parent loader

The first item is important to avoid the sandboxing of Java runtime classes. This is the behavior that most app server web app class loaders observe.

But not WebSphere. In PARENT_LAST mode, WebSphere does strict sandboxing. Now if your web app includes an extension library (not recommended, but not that uncommon) WebSphere will load it from the web app class loader in preference to the extension class loader. In the absence of BOF, this wouldn’t matter much because it would be loaded once and only once. But along comes a dynamically loaded BOF module which causes it to be loaded by the extension class loader. Now the class is loaded in two loaders in the parent chain. This results in a VerifyError.

DFC accommodates this apparently intractable problem by having a preference – dfc.bof.classloader.enable_extension_loader_first – which is set to true by default. If you are using a PARENT_LAST WebSphere class loader with an extension library in the web app then you will have to set this to false and try your luck. This causes the first step to be skipped by the BOF sandboxing class loader. Better would be to remove the library from your web app altogether and rely on the extension loader.

How did you get an extension library in your web app? Well across Java releases various packages have migrated into the system extensions. Older web apps legitimately packaged the library du jour and only after a Java upgrade does this behavior potentially manifest. Java 5 added a number of XML related packages, for example.

Jar defs and Java libraries

The jars that you use in BOF modules can be delivered in Jar Defs or in Java Libraries. Let’s look at the differences, real and conventional.

Jar Defs (dmc_jar instances in the docbase) are just jar wrappers. They have a nature – implementation or interface – that determines to which class loader they belong. Here’s something that is not obvious – you should not share Jar Defs across modules. We don’t enforce this restriction well enough, but we don’t support this usage. If you want to share jars across modules you should use a Java Library. Read on.

Java Libraries (dmc_java_library instances) are collections of one or more jars. Each Java Library is flagged as being sandboxed or not. All jars in a sandboxed Java Library will belong to the module class loader, just like regular implementation jars. All jars in a non-sandboxed Java Library will belong to the Shared Class Loader. Java Libraries can be shared across modules. So they have two purposes:

• Sharing jars across modules
• Collecting jars into an administrable unit

Some good uses for Java Libraries are:

• third party libraries
• shared logic across a cohesive set of modules

Be very very careful about marking a Java Library non-sandboxed. Because this associates the jars with the Shared Class Loader all BOF modules will have visibility to these classes. The original plan was not to allow this at all, but the Architect broke under pressure. This is back to the kind of jar hell that BOF tries so hard to eliminate.

If you decide to share jars (via Java Libraries) across modules be sure to understand that a sandboxed Java Library will be associated with the module class loaders. If you intend to communicate between your modules then any interfaces that exist in your shared jars will have a separate identity in each class loader, so if you try to use them across modules you will get ClassCastExceptions. If you want to do this then you need to separate out your shared interfaces from the other classes in your shared jars and put them in a distinct non-sandboxed Java Library. This will allow the interfaces to be used by all of your modules. This is a much better idea than just marking your existing Java Library as non-sandboxed which will have the effect of introducing all your implementation classes into the Shared Class Loader too.

Both Jar Defs and Java Libraries can be hot deployed. Because a Jar Def is associated with only one module its hot deployment has a localized effect. If a Java Library is shared, on the other hand, its hot deployment affects all modules that share the Java Library.

Jar Defs should always belong to the containing module so they will appear in the same Composer project as the module itself.

Java Libraries can be shared and so projects containing consuming modules might need to reference another Composer project in order to access the Java Library. That referenced project might be a “primary” module project, or conceivably a project that contains only the Java Library – the choice is yours. The important thing is that the Java Library must exist in only one Composer project.

The things to consider when making choices are hot deployment issues, class visibility issues, and avoiding redundant jars wherever possible.

Calling DFS from within a BOF module

Typically you call DFS from within your web app and everything works as expected. Then you diligently package your DFS classes within a BOF module and try to call the same DFS service from within the module and you get javax.xml.bind.JAXBException. What happened?

Let’s look at the “calling from web app” scenario first. DFS uses JAXB to convert between Java objects and XML. In order to do this JAXB needs to dynamically load classes. Now the JAXB implementation is in the extension class loader (only one level above the bootstrap loader) and it can’t see upwards in the class loader hierarchy so it can never see the application classes that it needs to load. So how does it get to see the classes? It can’t use its own class loader for the dynamic load operation because it has no visibility. It can’t use the web app class loader directly, because it can’t find it and doesn’t even know that it’s the right class loader anyway – how would it? So how does it work?

Each thread has a thread local reference to a class loader called the thread context class loader (TCCL). This can be set and recovered programmatically. In a web app scenario this will have been set by the app server to the web app class loader (WEB-INF/lib). So JAXB (and most other Java runtime factories) use the TCCL for dynamic class load operations in order to find the application provided classes.

In the web app scenario this works just fine because the TCCL is set to the class loader that has visibility to the application classes. Within a BOF module, however, it doesn’t work. Why? Because the TCCL is still set to the web app class loader, but the classes are only visible to the module class loader which is not visible to the web app class loader (see BOF class loaders). So you get javax.xml.bind.JAXBException.

How do you fix it? Fortunately this is simple – you just set the TCCL to your module class loader around your DFS calls. Be sure to bracket this in try/finally logic to ensure the TCCL is appropriately reset.


ClassLoader tccl = Thread.currentThread().getContextClassLoader();
try {
    ClassLoader cl = getClass().getClassLoader();
    Thread.currentThread().setContextClassLoader(cl);
    // .. do DFS stuff
} finally {
    Thread.currentThread().setContextClassLoader(tccl);
}


That’s it.

Configuring a deployed BOF module

Sometimes you might want a BOF module to be configured externally post deployment. You might want to use an XML file or a properties file for this purpose.

The naive way would be to checkout the implementation jar, unzip it, edit the config file, rezip it, and check in the updated jar. That’s too horrible to contemplate, for too many reasons to list.

There is another way. BOF anticipated the need for external, non-jar files to be accessible in the module classpath. Such external files are called downloadable attachments. Attachments in general are any non-jar files that the module author wants to be deployed with the module – say a javadoc zip, or license files. Typically you want these in the docbase, but you don’t want them in the module classpath. If you flag an attachment as downloadable though, it goes all the way to the class loader.

So, if you want a config file to be externally accessible and yet visible in the classpath you make it a downloadable attachment. In Composer you would go to your module’s “Deployment” tab, add the file as an attachment and check the downloadable option.

Downloadable Attachment in Composer

Then, at any time post deployment, you just need to check out, edit, and check back in the config file. The updated version will be hot deployed to the BOF module. You have to control the security of the attachment, but the rest of your module preserves its integrity.

Remember that hot deployment is only revealed to subsequent module instantiations, so don’t hold a cached reference to your module or you won’t see any updates. Prefer newModule or newService on demand – they’re quick.

How to index a qualifiable aspect attribute

1. Find the internal name of the “aspect table”
   select i_attr_def from dmc_aspect_type where object_name = ‘your aspect name’
2. Make the index
   execute make_index with type_name='the returned i_attr_def value', attribute='your attribute name'

   Note that the attribute name in this context is unqualified – you don’t precede it with the aspect name and a period.

Hat tip: Venkat

Finding a BOF module’s interface jar

If you are planning on using a BOF module you can find the jar containing its interface in the docbase as the contents of a dmc_jar object that is a member of the module’s folder (dmc_module and dmc_aspect_type are dm_folder subtypes) . If the object_name of the dmc_jar object doesn’t make it clear look for the dmc_jar with jar_type=1.

Implementing Java server methods as BOF modules

There are a couple of reasons you might want to deploy Java server methods as BOF modules.

• You want to deploy the implementation in the same dar/docapp as the dm_method definition
• You want to avoid jar hell by avoiding the java_methods directory
• You want hot deployment for your method implementation

In order to deploy your method as a module you have to do the following:

• Define a module by subclassing DfSingleDocbaseModule or have your module implement the marker interface IDfModule directly
• Have your module implement IDfMethod – not IDmMethod (no interface jar)
• Put the module name – not the class name – in the command_verb of the dm_method object
• Make your module self-contained. You have no visibility to classes in the java_methods directory so you must be able to find anything you need in one or more BOF modules.

That’s it.