Moving on

In a few weeks I will leave EMC to take on a new position. As a consequence this blog will become inactive but I will keep it available.

Toward the end of my time I was working on the xCP initiative, and I truly believe this will be a successful and exciting product that will change the way EMC does business in the content world. I offer my best wishes to the team. EMC has some very strong people working on your behalf to improve your experience with its offerings.

I hope that those of you who visit the site continue to find some value in these posts.

Don

A trap with aspects

Sometimes you want to write an aspect with behavior that is orthogonal to that in DfSysObject. Nothing wrong with that – we encourage it – but here’s the rub …

When DFC dynamically builds the Java class for an object to which your aspect is attached it dynamically builds an inheritance hierarchy with the TBO and its DFC superclasses as the supermost classes, and with each aspect subclassing each other and ultimately the TBO. DFC literally patches the superclass pointers in the byte code. You are going to subclass DfSysObject and its superclasses whether you want to or not.

Now if you are intending to write an aspect with orthogonal behavior you have to be really careful not to unintentionally override a DfSysObject (or superclass) method, which could have bizarre consequences. A more overt and terminal consequence arises if you unintentionally override a final method – you get the infamous java.lang.VerifyError, although it isn’t related to class loaders this time. An example of this is if you try to use a DfService implementation as an orthogonal aspect implementation (you can’t).

The best tip? Just subclass DfSysObject in your aspect primary class in the first place and the enforced runtime inheritance structure will be reflected at compile time.

Writing Java server methods that invoke DFS services

Sometimes you want to invoke DFS services from within a Java Method Server method implementation.

First you should understand how to implement a method as a BOF module.

What remains is to package your module correctly for DFS service invocation. In order to do this we suggest the following:

1. Create a sandboxed Java library containing the minimum classpath for your chosen DFS client, as described in the DFS SDK doc. This library can be included in all method modules that invoke DFS services.
2. Create a sandboxed Java library containing your generated client classes for each service that you consume. Each library can be included in all method modules that invoke the corresponding DFS service. If you only have one or two methods then these classes could be folded into your implementation jar instead.
3. Create an implementation jar for your method implementation.

Be sure that your method implementation observes the correct thread context class loader handling and you should be good to go. Here’s a little base class that takes care of that for you (note that this class must be in either your implementation jar or a sandboxed java library or it just won’t work).


package com.emc.examples;
import com.documentum.fc.client.DfSingleDocbaseModule;
import java.util.Map;
import java.io.PrintWriter;
public abstract class AbstractMethod extends DfSingleDocbaseModule implements IDfMethod
{
    public final int execute(Map parameters, PrintWriter writer) throws Exception {
        ClassLoader tccl = Thread.currentThread().getContextClassLoader();
        try {
            ClassLoader moduleLoader = AbstractMethod.class.getClassLoader();
            Thread.currentThread().setContextClassLoader(moduleLoader);
            return doMethod(parameters, writer);
        }
        finally {
            Thread.currentThread().setContextClassLoader(tccl);
        }
    }
    public abstract int doMethod(Map parameters, PrintWriter writer) throws Exception;
}


Psst. Wanna buy a Content Server?

You don’t have to – it’s free for developers.

Take your time. Try this and that. Build apps.

Let a thousand flowers bloom.

Hot deployment in BOF – yes or no?

One of the primary benefits of BOF is hot deployment of updates. But do you get hot deployment for all BOF elements? The answer is no, and you need to understand when you do and when you don’t.

If you look at the BOF class loaders you will see that all module-specific loaders share a common parent – the BOF shared loader. The shared loader is primarily intended to contain module interfaces. There is a second supported case though – non-sandboxed Java libraries. These are used to share implementation jars across modules. This is not a recommended practice, but it does avoid potential duplication of class files in memory (the only reason that it is supported).

It is fatal to update interfaces in a running VM in many cases and ill advised in others, so the expectation is that classes in the shared loader are not dynamic. Anything that is loaded by a module-specific loader is subject to hot deployment, but anything loaded by the shared loader does not enjoy hot deployment. The main reason for this is that invalidating this class loader would mean invalidating all existing module-specific class loaders, possibly resulting in duplicates for some time as modules are garbage collected which could be a significant memory cost. Regardless of the reason, it is a fact.

The take home lesson is that if you use non-sandboxed Java libraries you will not get hot deployment for those libraries. At the cost of some class file duplication you can get all the BOF benefits including hot deployment if you sandbox your Java libraries, and we strongly recommend this. Note that an update to a sandboxed Java library is propagated to all consuming modules via hot deployment.

As an aside, be careful to avoid caching module instances because these instances are immune to hot deployment.

Using aspects with non-sysobjects

The aspect mechanism requires support that is built into dm_sysobject. Persistent objects that are not sysobjects, by default, cannot have aspects attached because they lack this support. You can add this support to a non-sysobject type using the following DQL

alter type my_type allow aspects

You can’t do this to standard Documentum types though.

java.lang.VerifyError and the whole truth

The BOF class loader diagrams I have published before contain a simplification that I had hoped to finesse, but reality seems to have caught up with me.

The whole truth is that the sandboxing BOF class loaders aren’t strictly parent last. Their behavior is:

1. Java extension loader
2. The BOF sandboxed loader
3. The parent loader

The first item is important to avoid the sandboxing of Java runtime classes. This is the behavior that most app server web app class loaders observe.

But not WebSphere. In PARENT_LAST mode, WebSphere does strict sandboxing. Now if your web app includes an extension library (not recommended, but not that uncommon) WebSphere will load it from the web app class loader in preference to the extension class loader. In the absence of BOF, this wouldn’t matter much because it would be loaded once and only once. But along comes a dynamically loaded BOF module which causes it to be loaded by the extension class loader. Now the class is loaded in two loaders in the parent chain. This results in a VerifyError.

DFC accommodates this apparently intractable problem by having a preference – dfc.bof.classloader.enable_extension_loader_first – which is set to true by default. If you are using a PARENT_LAST WebSphere class loader with an extension library in the web app then you will have to set this to false and try your luck. This causes the first step to be skipped by the BOF sandboxing class loader. Better would be to remove the library from your web app altogether and rely on the extension loader.

How did you get an extension library in your web app? Well across Java releases various packages have migrated into the system extensions. Older web apps legitimately packaged the library du jour and only after a Java upgrade does this behavior potentially manifest. Java 5 added a number of XML related packages, for example.

Jar defs and Java libraries

The jars that you use in BOF modules can be delivered in Jar Defs or in Java Libraries. Let’s look at the differences, real and conventional.

Jar Defs (dmc_jar instances in the docbase) are just jar wrappers. They have a nature – implementation or interface – that determines to which class loader they belong. Here’s something that is not obvious – you should not share Jar Defs across modules. We don’t enforce this restriction well enough, but we don’t support this usage. If you want to share jars across modules you should use a Java Library. Read on.

Java Libraries (dmc_java_library instances) are collections of one or more jars. Each Java Library is flagged as being sandboxed or not. All jars in a sandboxed Java Library will belong to the module class loader, just like regular implementation jars. All jars in a non-sandboxed Java Library will belong to the Shared Class Loader. Java Libraries can be shared across modules. So they have two purposes:

• Sharing jars across modules
• Collecting jars into an administrable unit

Some good uses for Java Libraries are:

• third party libraries
• shared logic across a cohesive set of modules

Be very very careful about marking a Java Library non-sandboxed. Because this associates the jars with the Shared Class Loader all BOF modules will have visibility to these classes. The original plan was not to allow this at all, but the Architect broke under pressure. This is back to the kind of jar hell that BOF tries so hard to eliminate.

If you decide to share jars (via Java Libraries) across modules be sure to understand that a sandboxed Java Library will be associated with the module class loaders. If you intend to communicate between your modules then any interfaces that exist in your shared jars will have a separate identity in each class loader, so if you try to use them across modules you will get ClassCastExceptions. If you want to do this then you need to separate out your shared interfaces from the other classes in your shared jars and put them in a distinct non-sandboxed Java Library. This will allow the interfaces to be used by all of your modules. This is a much better idea than just marking your existing Java Library as non-sandboxed which will have the effect of introducing all your implementation classes into the Shared Class Loader too.

Both Jar Defs and Java Libraries can be hot deployed. Because a Jar Def is associated with only one module its hot deployment has a localized effect. If a Java Library is shared, on the other hand, its hot deployment affects all modules that share the Java Library.

Jar Defs should always belong to the containing module so they will appear in the same Composer project as the module itself.

Java Libraries can be shared and so projects containing consuming modules might need to reference another Composer project in order to access the Java Library. That referenced project might be a “primary” module project, or conceivably a project that contains only the Java Library – the choice is yours. The important thing is that the Java Library must exist in only one Composer project.

The things to consider when making choices are hot deployment issues, class visibility issues, and avoiding redundant jars wherever possible.

Smart Container demo available

Our intrepid Product Manager, David Louie, has posted a demo of Smart Container functionality. Enjoy.

Calling DFS from within a BOF module

Typically you call DFS from within your web app and everything works as expected. Then you diligently package your DFS classes within a BOF module and try to call the same DFS service from within the module and you get javax.xml.bind.JAXBException. What happened?

Let’s look at the “calling from web app” scenario first. DFS uses JAXB to convert between Java objects and XML. In order to do this JAXB needs to dynamically load classes. Now the JAXB implementation is in the extension class loader (only one level above the bootstrap loader) and it can’t see upwards in the class loader hierarchy so it can never see the application classes that it needs to load. So how does it get to see the classes? It can’t use its own class loader for the dynamic load operation because it has no visibility. It can’t use the web app class loader directly, because it can’t find it and doesn’t even know that it’s the right class loader anyway – how would it? So how does it work?

Each thread has a thread local reference to a class loader called the thread context class loader (TCCL). This can be set and recovered programmatically. In a web app scenario this will have been set by the app server to the web app class loader (WEB-INF/lib). So JAXB (and most other Java runtime factories) use the TCCL for dynamic class load operations in order to find the application provided classes.

In the web app scenario this works just fine because the TCCL is set to the class loader that has visibility to the application classes. Within a BOF module, however, it doesn’t work. Why? Because the TCCL is still set to the web app class loader, but the classes are only visible to the module class loader which is not visible to the web app class loader (see BOF class loaders). So you get javax.xml.bind.JAXBException.

How do you fix it? Fortunately this is simple – you just set the TCCL to your module class loader around your DFS calls. Be sure to bracket this in try/finally logic to ensure the TCCL is appropriately reset.


ClassLoader tccl = Thread.currentThread().getContextClassLoader();
try {
    ClassLoader cl = getClass().getClassLoader();
    Thread.currentThread().setContextClassLoader(cl);
    // .. do DFS stuff
} finally {
    Thread.currentThread().setContextClassLoader(tccl);
}


That’s it.