How to use Liferay third-party libraries as Maven dependencies ?

Almost all day worth of work but finally I finished DependencyInstallerMojo. Simply put, you provide it with destination of Liferay's lib/versions.xml and inclusion regexp patterns that determine which libraries are to be installed into either local maven repository or a custom location and it installs them as Maven artifacts and generates a pom definition that contains corresponding dependencies, so that you can use itself as dependency and thus transitively include all those libraries on your classpath.

  
 <plugin>
    <groupId>com.liferay.maven.plugins</groupId>
    <artifactId>liferay-maven-plugin</artifactId>
    <version>6.1.0-SNAPSHOT</version>
    <configuration>
      <generatedPomLocation>${project.basedir}/target/generated-pom.xml</generatedPomLocation>
      <generatedPomVersion>${project.version}</generatedPomVersion>
      <generatedPomName>Liferay dependencies</generatedPomName>
      <localRepositoryId>liferay-third-party-deps</localRepositoryId>
      <projArtifactId>${project.artifactId}</projArtifactId>
      <projGroupId>${project.groupId}</projGroupId>
      <dependencyScope>test</dependencyScope>
      <libDirPath>/opt/liferay/portal/lib</libDirPath>
      <localRepositoryPath>/home/old-lisak/fake-repo</localRepositoryPath>
      <include>
        <development>jsf-.*,derby,catalina,ant-.*,jalopy</development>
        <global>portlet.jar</global>
        <portal>chemistry-.*,commons-.*,jackrabbit-.*,spring-.*</portal>
      </include>
    </configuration>
 </plugin>

Most of those properties are set by default, you actually need to set at least :
 libDirPath - to the source xml file (versions.xml in case of Liferay)
 localRepositoryPath - to a custom local maven repository, unless you want to use the default one
 include - this element is a map of sub-directories of libDirPath that contains jar packages

Names of the sub-directories become part of groupId (com.example.development) and regular expressions adhere to java.util.regex split up by colons.


Why Liferay developers might need this ?


Because if you wanted your code to be covered by infrastructure tests, you would have to mock Liferay services quite frequently, unless you have Liferay third party libraries on classpath and you can boot up the spring based infrastructure and  hundreds of its services.
   Mocking might seem to be a suitable alternative until you are refactoring or upgrading Liferay version and all those tests turn into a maintenance hell. I personally use rather infrastructure testing than tons of "functional" unit tests, so that I'm trying to deal with this situation.

A year ago I manually listed maven dependencies corresponding to those listed in lib/versions.xml, just those that I needed for using particular Liferay services, but it is a really hideous thing to do because of all those version conflicts and transitive dependencies.

I wrote this Maven Mojo after I got really repelled by Maven Surefire Plugin's way of dealing with additional classpath resources, classloading hell.

A last possible option how to do this is systemPath property of Maven Dependency of system scope, but it is deprecated and it will be probably not supported in next releases.

If you want to try that out, follow these instructions :

$git clone git@github.com:l15k4/liferay-maven-support.git
$cd liferay-maven-support
$mvn install

Then go into an existing maven project and add this plugin into its build section.

  
 <plugin>
   <groupId>com.liferay.maven.plugins</groupId>
   <artifactId>liferay-maven-plugin</artifactId>
   <version>6.1.0-SNAPSHOT</version>
   <configuration>
     <libDirPath>/path/to/portal/lib</libDirPath>
     <localRepositoryPath>/home/user/test-maven-repository</localRepositoryPath>
     <include>
       <development>jsf-.*,derby,catalina,ant-.*,jalopy</development>
       <global>portlet.jar</global>
       <portal>chemistry-.*,commons-.*,jackrabbit-.*,spring-.*</portal>
     </include>
   </configuration>
 </plugin>

$mkdir /home/user/test-maven-repository
$mvn com.liferay.maven.plugins:liferay-maven-plugin:6.1.0-SNAPSHOT:install-dependencies


If everything goes well, the repository will contain the artifacts and there will be target/generated-pom.xml file in your project.

If you found this useful and you wanted to see this as part of liferay-maven-plugin, you could vote up here: LSP-22947

Does Maven honor principles it is based on in its own architecture ?

  I like Maven a lot, makes Java programming more fun and spares a lot of time. I also understand principles of preserving backward compatibility, but it is supposed to be a comprehension tool and despite of that it still uses Plexus or Javadoc tags that are quite hard to comprehend and imho should be deprecated in version 2 and disappear in version 3. Unfortunately I don't see almost any sings of that.

People know and like Guice, not Plexus that is deprecated anyway... What about promised Plexus substitution with Google Guice in Maven?
   First off, I got quite angry because I couldn't find any leads that would reveal this mysterious Dependency Injection Framework swap. After the hype almost a year ago that maven is going to migrate to Google Guice from Plexus I grep through the maven trunk source base and it is full of org.codehaus.plexus references and the only place where Guice is used is maven/app-engine. 

Then I hit MNG-4749 and I realized that the compatibility shim is a work of sonatype dev and I found these two posts : From Plexus to Guice (#2) and From Plexus to Guice (#3) that were quite hidden to me and that revealed Sisu project. Basically, they created a bean injection layer on top of the raw custom injection API provided by Guice. These images tell a little more. The left one expresses how the consequences of preserving backward compatibility or avoiding radical refactoring might look like :-)

You can find the source code in sonatype sisu repository. Btw seeing software moving to git is always nice. Do you remember all those sayings how Maven is lightweight a couple years back :-) ? Although I appreciate the effort of sonatype dev. It must have been a tough work and I certainly try it out and change this article accordingly.

Another point of interest is migration to Java Annotations. As Maven comes from JDK1.4 era, it uses Javadoc tags. They are not represented at the byte code level. It seems to be quite a drawback for Plugin and Mojo API. And tough to understand - missing Annotation's convention. Developers know Annotations, but most of them have never worked with Javadoc tags extensively. In case you need to inherit a Mojo class of different artifact and the maven property doesn't get initialized, what then ? F*** around on Jira and study how QDox and all that work :-) ? Or duplicating all code you would inherited otherwise ? Extending Mojos is so hard now.
    There are some rumors it is going to be changed to Annotations, and one can see some activity around it : Java 5 Annotations for plugins and MPLUGIN-189, but I'm afraid that it might end up with some sort of compatibility layer as in case of the sisu project.
It is so hard
It is a comprehension tool for a user, but these things make it tough to work with for plugin developers. It is sad, but it is true.

I believe that much more developers would create Maven Mojo or plugin instead of a Shell script even for simple things, if these 2 critical issues were solved.

Booting up Ant project when testing with Maven and surefire plugin

Have you ever had a huge non-maven project that you needed to boot up in order to use it when testing your maven project ? The first time I had to deal with that, I wrote an Ant script that has installed all those hundreds of jars as dummy dependencies with no metadata to maven repository. It was called every time the software finished building.
   The second time I needed to do something similar, I was already aware of the fact that installing dummy artifacts with no metadata has no sense. Just because you need them on classpath, it doesn't mean you have to do it the maven way at all costs.

So I decided to use Maven Surefire Plugin for that. It is a really complex and configurable plugin. But it can drive people crazy because MS Windows has become so popular in past 2 decades. The fact that it has problems with long classpaths made surefire plugin use Manifest-Only JAR :

A jar that declares"Class-Path" attribute  in its manifest where the true classpath is specified 

or an Isolated classloader :

Launching an app by booter that uses new classloader that is passed the classpath

This is cool I get it, plain simple, but take a look at the maven surefire plugin documentation :

Surefire provides a mechanism for using multiple strategies. The main parameter that determines this is called useSystemClassLoader. If useSystemClassLoader is true, then we use a manifest-only JAR; otherwise, we use an isolated classloader. If you want to use a basic plain old Java classpath, you can set useManifestOnlyJar=false which only has an effect when useSystemClassLoader=true.

olol !!! What the heck is that ? Moreover they suddenly mention that it somehow changed and in the end it is the other way around or what. I have to debug that plugin to actually see what is this about. I mean, how many time one sets up surefire plugin a year ? Many times... And every time a programmer hits this crap, unless he spends hours by investigating deeper, he has to spend time thinking about it and dealing with classpath and threading issues.

To make the story short, if you need to do something like this :

   
  <additionalClasspathElements>
     <additionalClasspathElement>
        /path/to/libs/*.jar
     </additionalClasspathElement>
     <additionalClasspathElement>
        /path/to/libs2/*.jar
     </additionalClasspathElement>
     <additionalClasspathElement>
        /path/to/libs3/*.jar
     </additionalClasspathElement>
  </additionalClasspathElements>

and you use a sane operating system and surefire version 2.10, then you need to set these properties to false :

  <useManifestOnlyJar>false</useManifestOnlyJar>
  <useSystemClassLoader>false</useSystemClassLoader>

The explanation is hidden in ForkConfiguration.java :

 if ( useManifestOnlyJar )
  {
      File jarFile;
      try
      {
          jarFile = createJar( classPath );
      }
      catch ( IOException e )
      {
       throw new SurefireBooterForkException( "Error creating archive file", e );
      }
       cli.createArg().setValue( "-jar" );
       cli.createArg().setValue( jarFile.getAbsolutePath() );
  }
  else
  {
      cli.addEnvironment( "CLASSPATH", StringUtils.join( classPath.iterator(), File.pathSeparator ) );
      final String forkedBooter = ForkedBooter.class.getName();
      cli.createArg().setValue( shadefire ? new Relocator().relocate( forkedBooter ) : forkedBooter );
  }
  cli.setWorkingDirectory( workingDirectory.getAbsolutePath() );
  return cli;
 }

 public File createJar( List classPath ) throws IOException
  {
     ..................
     String cp = "";
     for ( Iterator it = classPath.iterator(); it.hasNext(); )
     {
         String el = (String) it.next();
         // NOTE: if File points to a directory, this entry MUST end in '/'.
         cp += UrlUtils.getURL( new File( el ) ).toExternalForm() + " ";
     }
     man.getMainAttributes().putValue( "Manifest-Version", "1.0" );
     man.getMainAttributes().putValue( "Class-Path", cp.trim() );
     man.getMainAttributes().putValue( "Main-Class", ForkedBooter.class.getName() );
     man.write( jos );
     jos.close();

     return file;
 }

This explains that you cannot useManifestOnlyJar if you need to append stuff with wildcards to your overall classpath, see how Class-Path attribute is made in createJar method.

However, if useSystemClassLoader is false, then the context classloader that you get from currentThread is the Isolated classloader that contains only surefire and plexus booting related classes.

   
    Thread currentThread = Thread.currentThread();
    ClassLoader contextClassLoader = currentThread.getContextClassLoader();

On the other hand, if useSystemClassLoader is true, then test(s) is not run at all :

   
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running cz.instance.liferay.test.SampleTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.218 sec

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

   Which is in fact some sort of hard to kill TestNG issue. If you use Junit instead, guess what classloader you get from currentThread.getContextClassLoader(); Yeah, you get the isolated classloader that contains just the booting classes...

    I came to conclusion, that thanks to Surefire-plugin's toleration to shitty OS like MS Windows, they practically discouraged most people from using this extensively, because you hardly can boot up an app like Liferay via its Isolated classloader. Liferay extensive classloading will simply doesn't work with it. In fact it is even using  ClassLoader.getSystemClassLoader();  Plus consider classloading issues with loggers...

   Unless you're the kinda guy that wants to spend hours playing with it, you better go with a shell script that installs the dummy artifacts and produces a pom artifact that serves only as dependencies aggregator, that you just include as dependency to your project.

Broadleaf Commerce Vs. Apache Ofbiz

At first, around 2-3 years ago, I gave up on using Apache Ofbiz, because it seemed that I wouldn't be able to be "productive" with it for weeks, because of its complexity. But I noticed of Broadleaf Commerce a few months ago and I find it very interesting. The technology stack suits me a lot ( spring, spring-mvc frontend, GWT backend, hibernate on jpa spec, maven, etc.), if it utilized git and github.com for SCM, I wouldn't have anything to criticize. On the other hand they got rid of flex from backend which makes me happy :-) The principle of building customized shopping cart on this platform is very straightforward, but it expects a developer with some experience too.

First off, I'd like to mention that e-commerce out-of-the-box solutions (this applies especially for non-US businesses) are simply not able to fulfill specific needs. If you look around, you see tons of PHP shopping carts (install & go) that allows you to run a cart, but you have to live with all the disadvantages that result from the "universality" of the solution. Payment, taxes, accountancy and shipment is locale specific. They have mostly terrible architecture, missing modularity, horrible UX and tons of other idiosyncrasies...  Prestashop is not that bad from what I've seen and been told.

Then there are Java e-commerce solutions like partially opensource Konakart or opensource shopizer and jadasite. Jadasite looks quite alright, but it is out-of-the-box, it doesn't even have a shell or ant script, no maven...imagine you would need to change payment or shipment module... no way. I just refuse to work with applications that depend on eclipse WTP anyway...

And finally, there are solutions like Apache Ofbiz and Broadleaf Commerce. They are not out-of-the-box solutions (well broadleaf coffee shop demo is almost production ready for US), but rather platforms. A  developer utilizes them to put a custom solution together, not develop/program/code but rather "put it together", which means : infrastructure setup, beans/services/entities adding, overriding, merging, extending or changing an existing module to adjust it to a concrete region, etc. Complexity of Apache Ofbiz makes you waste a lot of time, in my opinion. Unless you are being paid for that familiarization, it's a big problem. As to Broadleaf commerce, I find it much easier. The documentation could be more actual / current though. But I think that for a developer who is new to these two platforms, choosing Apache Ofbiz means much more investigating and learning the internal working than the actual results and it is very demotivating. But in regards to Broadleaf Commerce, I kinda see everything after 2 hours of reading through the codebase.

I tried to investigate Ofbiz and I came to conclusion, that for my needs (shopping cart) it is not worth it. Its entity and service engines, workflows etc. and the way you're setting up the platform seemed very J2EE non-standard and I felt lost! Then I realized that it is probably inevitable, because how else should e-commerce platform of this caliber look like? It just can't be done as it is in Broadleaf commerce. You simply have to spend  weeks learning how to become productive with it... Whole weeks ? Yeah I'd have to deliver the entire e-commerce cart myself. It's advantage is that it is 10 years old, it is roofed by Apache foundation and it has relatively big community around. The disadvantage is, imho, that its main contributors are employees of companies that doesn't allow radical changes to it which causes defensive programming and not much of a radical refactoring that would be beneficial for the platform (at least I got the impression). This is not a problem for experienced Ofbiz developers, but it's quite a roadblock for newcomers.

I spent a couple hours looking at source base of Broadleaf Commerce and I see very clearly what would need to be done to setup a maven project (webapp) that represents e-commerce cart and that just deploys to a container with broadleaf platform libraries as dependencies and works ! You can see that a few minutes after svn checkout ! There is a demo project that you are supposed to copy/use as your new site/eshop. You  just override/add/merge what you need to (controllers, services, entities, jsp, css, entire modules) simply via spring configuration or filesystem changes and this way you build up your custom e-commerce cart. There is one thing that I would appreciate though : in addition to the demo site there should also be a maven archetype with default/classic template, infrastructure setup and default workflows and modules, that a developer just deploys and starts to modify. Or a theme or two would be nice. Another important point to mention is, that a good knowledge of Google Web Toolkit is great advantage, because you need it for developing backend / administration. Also Broadleaf Commerce seems to be easily deployable to Cloud Foundry for eventual scaling or other cloud-like needs. And finally, I think that this is the only and ideal e-commerce solution that would be relatively easy to integrate with Portals like Liferay or WebSphere and I'm not talking about just embedding it, but about building an e-commerce cart portlet with Broadleaf framwork. It would be tough integration job, but there is definitely no better or suitable candidate for that.

I would like to see and I wish Broadleaf Commerce a growing community because this software deserves attention.