Sunday, August 23, 2015

Overriding Java classes

Overriding Java classes is a technique that I use a lot, however after talking to other people it seems to be not so widely known. So what do I mean by overriding java classes? What I'm referring to is how the JVM deals with 2 classes with the same class and package name. What happens is the JVM will only use the class appearing first on the classpath.
So to put this to use, one could apply patches without touching the original package, or us it during patch development for quick compiling and testing.

Using this for Hadoop development or maintenance,  one method that I've used a lot is to add a patch dir to hadoop e.g. share/hadoop/patches and make sure that $HADOOP_PREFIX/share/hadoop/patches/* is added first to your classpath. Doing this would allow you to add jars with modified classes that overrides the original ones just by putting them in the patch dir.

To put this to use, lets say we are working on a patch for the Resource manager of yarn, and every time we want to test our changes or dump a new log, instead of spending 10-20 minutes rebuilding the entire Hadoop package, generating a new jar only containing the modified classes and then adding it first on the classpath would allow for sub second compiles meaning a greatly decreased cost for testing changes.

No comments:

Post a Comment