Sunday, November 6, 2011

Booting up Ant project when testing with Maven and surefire plugin


Have you ever had a huge non-maven project that you needed to boot up in order to use it when testing your maven project ? The first time I had to deal with that, I wrote an Ant script that has installed all those hundreds of jars as dummy dependencies with no metadata to maven repository. It was called every time the software finished building.
   The second time I needed to do something similar, I was already aware of the fact that installing dummy artifacts with no metadata has no sense. Just because you need them on classpath, it doesn't mean you have to do it the maven way at all costs.

So I decided to use Maven Surefire Plugin for that. It is a really complex and configurable plugin. But it can drive people crazy because MS Windows has become so popular in past 2 decades. The fact that it has problems with long classpaths made surefire plugin use Manifest-Only JAR :
A jar that declares"Class-Path" attribute  in its manifest where the true classpath is specified 
or an Isolated classloader :
Launching an app by booter that uses new classloader that is passed the classpath
This is cool I get it, plain simple, but take a look at the maven surefire plugin documentation :
Surefire provides a mechanism for using multiple strategies. The main parameter that determines this is called useSystemClassLoader. If useSystemClassLoader is true, then we use a manifest-only JAR; otherwise, we use an isolated classloader. If you want to use a basic plain old Java classpath, you can set useManifestOnlyJar=false which only has an effect when useSystemClassLoader=true.
olol !!! What the heck is that ? Moreover they suddenly mention that it somehow changed and in the end it is the other way around or what. I have to debug that plugin to actually see what is this about. I mean, how many time one sets up surefire plugin a year ? Many times... And every time a programmer hits this crap, unless he spends hours by investigating deeper, he has to spend time thinking about it and dealing with classpath and threading issues.

To make the story short, if you need to do something like this :

   
  <additionalClasspathElements>
     <additionalClasspathElement>
        /path/to/libs/*.jar
     </additionalClasspathElement>
     <additionalClasspathElement>
        /path/to/libs2/*.jar
     </additionalClasspathElement>
     <additionalClasspathElement>
        /path/to/libs3/*.jar
     </additionalClasspathElement>
  </additionalClasspathElements>

and you use a sane operating system and surefire version 2.10, then you need to set these properties to false :

  <useManifestOnlyJar>false</useManifestOnlyJar>
  <useSystemClassLoader>false</useSystemClassLoader>

The explanation is hidden in ForkConfiguration.java :

 if ( useManifestOnlyJar )
  {
      File jarFile;
      try
      {
          jarFile = createJar( classPath );
      }
      catch ( IOException e )
      {
       throw new SurefireBooterForkException( "Error creating archive file", e );
      }
       cli.createArg().setValue( "-jar" );
       cli.createArg().setValue( jarFile.getAbsolutePath() );
  }
  else
  {
      cli.addEnvironment( "CLASSPATH", StringUtils.join( classPath.iterator(), File.pathSeparator ) );
      final String forkedBooter = ForkedBooter.class.getName();
      cli.createArg().setValue( shadefire ? new Relocator().relocate( forkedBooter ) : forkedBooter );
  }
  cli.setWorkingDirectory( workingDirectory.getAbsolutePath() );
  return cli;
 }

 public File createJar( List classPath ) throws IOException
  {
     ..................
     String cp = "";
     for ( Iterator it = classPath.iterator(); it.hasNext(); )
     {
         String el = (String) it.next();
         // NOTE: if File points to a directory, this entry MUST end in '/'.
         cp += UrlUtils.getURL( new File( el ) ).toExternalForm() + " ";
     }
     man.getMainAttributes().putValue( "Manifest-Version", "1.0" );
     man.getMainAttributes().putValue( "Class-Path", cp.trim() );
     man.getMainAttributes().putValue( "Main-Class", ForkedBooter.class.getName() );
     man.write( jos );
     jos.close();

     return file;
 }

This explains that you cannot useManifestOnlyJar if you need to append stuff with wildcards to your overall classpath, see how Class-Path attribute is made in createJar method.

However, if useSystemClassLoader is false, then the context classloader that you get from currentThread is the Isolated classloader that contains only surefire and plexus booting related classes.

   
    Thread currentThread = Thread.currentThread();
    ClassLoader contextClassLoader = currentThread.getContextClassLoader();

On the other hand, if useSystemClassLoader is true, then test(s) is not run at all :

   
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running cz.instance.liferay.test.SampleTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.218 sec

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0


   Which is in fact some sort of hard to kill TestNG issue. If you use Junit instead, guess what classloader you get from currentThread.getContextClassLoader(); Yeah, you get the isolated classloader that contains just the booting classes...

    I came to conclusion, that thanks to Surefire-plugin's toleration to shitty OS like MS Windows, they practically discouraged most people from using this extensively, because you hardly can boot up an app like Liferay via its Isolated classloader. Liferay extensive classloading will simply doesn't work with it. In fact it is even using  ClassLoader.getSystemClassLoader();  Plus consider classloading issues with loggers...

   Unless you're the kinda guy that wants to spend hours playing with it, you better go with a shell script that installs the dummy artifacts and produces a pom artifact that serves only as dependencies aggregator, that you just include as dependency to your project.

No comments: