04
Aug

Using Grunt JS with Apache Sling

This weekend I’ve given a try to see how difficult would it be to extract from Maven everything that has to do with the compilation of client-side files ( CofeeScript, JS, CSS, etc ), while keeping the rest of the build in Maven.

Goal

  • Live-reload client-side source files in Apache Sling w/o Maven, as soon as source files change.
  • Compile CoffeeScript files with GruntJS, b/c it’s faster than compiling with Maven, and then deploy the compiled files into JCR.

Solution

  1. First Step: configure a Grunt JS script that watches for CoffeeScript files, compiling them in the same target folder with Maven.  In my case that was ${project.build.directory}/coffee/js.
  2. Once compiled, deploy JS files in Apache Sling, in the right JCR path.
  3. Use grunt-contrib-watch plugin to update the page in Sling automatically.
  4. Integrate Grunt JS back into the Maven build

 Implementation

Source code from Gist.

 

02
Oct

Cobertura Reports for Integration Tests and Unit Tests in Apache Sling

Problem

The past weekend I’ve been struggling to finalize a Maven build that would execute unit tests and integration tests and then aggregate the code coverage reports from these 2 tests.

The task seemed pretty easy at the beginning, especially after checking out the new integration testing tools built in Sling. The sample project is quite straight forward, easy to understand and easy to get started with integration tests.

However, there were a few differences in the way I wanted to run the tests. In Apache Sling integration tests are written in separate projects. In my case, I wanted to create the infrastructure to write and execute the integration tests as part of each OSGI bundle; in this way I would be able to generate a more complete code coverage report, aggregating the results from unit tests with the results from the integration tests.

Solution

  1. Create a special maven profile to execute the unit test : “integration-tests”
  2. Get the code from the integration-tests Sling project sample and integrate it into the profile.
  3. Configure the additional sling bundles that need to be deployed during the integration tests.
  4. Configure the bundle to be tested so that it gets deployed on Sling, along with the other bundles.
  5. Separate executing of unit-tests from integration-tests
  6. Add Cobertura build step in order to compute the code coverage
  7. Aggregate the unit-test reports with the integration-tests reports into a unified code coverage report.

Step by step details

Step 1. Add the new maven profile

       <profile>
            <id>integration-tests</id>
        </profile>

Step 2. Get the Apache Sling testing projects

 svn co http://svn.apache.org/repos/asf/sling/trunk/testing/

Step 3. Copy the build directives and the dependencies from the integration-tests project, into the “integration-tests” profile you’ve created at step 1.
Then update the configuration for maven-failsafe-plugin to include the bundles you need for the integration tests. I.e.

<sling.additional.bundle.1>org.apache.sling.junit.core</sling.additional.bundle.1>
<sling.additional.bundle.2>org.apache.sling.testing.tools</sling.additional.bundle.2>
<sling.additional.bundle.3>org.apache.sling.junit.remote</sling.additional.bundle.3>
... others

Step 4. Add the bundle to be tested into the list of additional bundles b/c the integration tests are for it. Simply update the configuration for build-helper-maven-plugin.

    <plugin>
        <!-- Find free ports to run our server -->
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>build-helper-maven-plugin</artifactId>
        <version>1.7</version>
        <executions>
            <execution>
                <id>reserve-server-port</id>
                <goals>
                    <goal>reserve-network-port</goal>
                </goals>
                <phase>process-resources</phase>
                <configuration>
                    <portNames>
                        <portName>http.port</portName>
                    </portNames>
                </configuration>
            </execution>
 
            <!-- include integration tests folder when compiling tests -->
            <execution>
                <phase>generate-test-sources</phase>
                <goals>
                    <goal>add-test-source</goal>
                </goals>
                <configuration>
                    <sources>
                        <source>src/integration-test/java</source>
                    </sources>
                </configuration>
            </execution>
 
        </executions>
    </plugin>

Step 5. Separate executing of unit-tests from integration-tests. There are several ways to achieve this, what I’m describing here is just what I’ve chosen.
Update maven folder structure:

-src
  - integration-test
  - main
  - test

Use JUnit4 Test @Category(IntegrationTest.class) annotation to differentiate the integration tests from the unit tests. Even though they dwell in different directories, it made it easier for me to configure Maven build in this way.

Update the configuration for maven-surefire-plugin:

    <!-- ignore integration tests groups -->
    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.12.4</version>
        <dependencies>
            <dependency>
                <groupId>org.apache.maven.surefire</groupId>
                <artifactId>surefire-junit47</artifactId>
                <version>2.12.4</version>
            </dependency>
        </dependencies>
        <configuration>
            <excludedGroups>org.mediacenter.testing.IntegrationTest</excludedGroups>
        </configuration>
    </plugin>

Update the configuration for maven-failsafe-plugin:

    <!-- execute only groups marked with IntegrationTest -->
    <groups>org.mediacenter.testing.IntegrationTest</groups>

Step 6. Add Cobertura build step in order to compute the code coverage. This step was a bit tricky, b/c I had to compile the Cobertura JAR for OSGI; but thanks to the good examples from Apache Felix, it was easy to get it done. The Cobertura dependency needs to be added to the list of additional bundles to be installed during integration tests.

    <!-- Cobertura OSGI bundle used to compute code coverage during integration tests -->
    <dependency>
        <groupId>net.sourceforge.cobertura</groupId>
        <artifactId>cobertura</artifactId>
        <version>1.9.4.1</version>
        <classifier>osgi</classifier>
    </dependency>
    ...
   <sling.additional.bundle.8>cobertura</sling.additional.bundle.8>

Then add the cobertura-it-maven-plugin

    <!-- cobertura hook to calculate code coverage -->
    <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>cobertura-it-maven-plugin</artifactId>
        <version>2.5</version>
        <configuration>
            <formats>
                <format>xml</format>
            </formats>
            <check>
                <haltOnFailure>false</haltOnFailure>
            </check>
        </configuration>
        <executions>
            <execution>
                <id>cobertura-clean</id>
                <phase>clean</phase>
                <goals>
                    <goal>clean</goal>
                </goals>
            </execution>
            <execution>
                <id>cobertura-instrument</id>
                <phase>process-classes</phase>
                <goals>
                    <goal>instrument</goal>
                </goals>
            </execution>
            <execution>
                <id>cobertura-check-only</id>
                <phase>verify</phase>
                <goals>
                    <goal>check-only</goal>
                </goals>
            </execution>
        </executions>
    </plugin>

Step 7. Aggregate the unit-test reports with the integration-tests reports into a unified code coverage report.
For unit tests, there is a small trick to unified the reports. Make sure to dump reports into the same place and they get merged. To do this, update maven-failsafe-plugin configuration:

    <!-- show IT results together with the unit tests results -->
    <reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>

For integration tests, I had to use an ant task from Cobertura to generate the reports:

    <!-- generate the code coverage report after integration tests through the ant task -->
    <plugin>
        <artifactId>maven-antrun-plugin</artifactId>
        <dependencies>
            <dependency>
                <groupId>net.sourceforge.cobertura</groupId>
                <artifactId>cobertura</artifactId>
                <version>1.9.4.1</version>
            </dependency>
            <dependency>
                <groupId>log4j</groupId>
                <artifactId>log4j</artifactId>
                <version>1.2.9</version>
            </dependency>
        </dependencies>
        <executions>
            <execution>
                <phase>post-integration-test</phase>
                <id>cobertura-report</id>
                <configuration>
                    <tasks>
                        <taskdef classpathref="maven.runtime.classpath"
                                 resource="tasks.properties"/>
                        <mkdir dir="${project.build.directory}/site/cobertura"/>
                        <cobertura-report format="xml"
                                          datafile="${project.build.directory}/cobertura/cobertura.ser"
                                          destdir="${project.build.directory}/site/cobertura">
                        </cobertura-report>
                        <cobertura-report format="html"
                                          datafile="${project.build.directory}/cobertura/cobertura.ser"
                                          destdir="${project.build.directory}/site/cobertura">
                        </cobertura-report>
                    </tasks>
                </configuration>
                <goals>
                    <goal>run</goal>
                </goals>
            </execution>
        </executions>
    </plugin>

That’s it. I tried to describe briefly about 14 hours of work.

Now, after a successful build in Jenkins ( mvn clean verify site -Pintegration-tests ) I was able to get the aggregated reports for the unit test and integration tests.

27
Jun

Release an OSGI Bundle into an OBR with Maven

Comments Off on Release an OSGI Bundle into an OBR with Maven

I’ve been investigating lately what to do with a bundle, once it’s released:

  1. Should I automatically deploy it into a Staging or Production server ?
  2. Should I deploy it into another repository and then use Felix Web Console to deploy it manually ?

Automation is always good, but in conditions where before reaching Production there is a Staging / Pre-production environment, the #1 option doesn’t seem to fit. Therefore I’ve been looking on solutions for #2.

The idea that attracted me the most was to use an OBR ( OSGI Bundle Repository ). Felix Web Console has already an OSGI Repository section, which can connect to a remote OBR and let admin users install OSGI bundles right from the administration page.

I’m going to describe what I’ve done in order to achieve this.

  1. Create an empty OBR.

    For this I configured a new subdomain : obr.mydomain.com, in one of my development machines. I’ve linked the subdomain to

    /var/www/html/obr

    In that folder I’ve created an empty repository.xml file

    vi repository.xml
    <repository name='MediaCenter Bundle Repository'>
    </repository>

    I’ve then checked that I can open the XML file from the browser:

    http://obr.mydomain.com/repository.xml
  2. Configure Maven to deploy the bundle into the OBR, during a Maven Release.
    mvn -Dresume=false release:prepare release:perform \
       -Darguments="-DremoteOBR=my-obr \ 
                -DobrDeploymentRepository=my-obr::default::file:///var/www/html/obr/ \
                -DprefixUrl=http://nexus.mydomain.com/nexus/content/repositories/releases"

    If you’ve performed a maven release before, release:prepare and release:perform should be familiar to you. I’ll explain the other options.

    • -DremoteOBR – it’s an ID I gave to the OBR repository.
    • -DobrDeploymentRepository – defines the OBR repository: OBR_ID::layout::url .
      I’ve used the “default” layout and the URL points to a local file, b/c Jenkins who executes the maven command runs on the same machine with the OBR.
    • -DprefixUrl – it’s the prefix used in the URL to the JAR bundle. Without this prefix the URL would look like “file:/com/x/y/myBundle.jar” and people won’t the able to download the actual bundle.
      I’ve then pointed my prefix to the Nexus Maven repository. During the maven release the bundle is deployed in that Nexus repository anyway, so I’v added it as a prefix for my bundle. With the prefix the URL looks like “http://nexus.mydomain.com/nexus/content/repositories/releases/com/x/y/myBundle.jar”.
  3. After a maven release, check if the OBR contains the update. Open the URL
    http://obr.mydomain.com/repository.xml

    again and the information should be there. Here is a sample repository.xml:

     
    <repository name="MediaCenter Bundle Repository" lastmodified="20120627090431.258">
        <resource id="org.mediacenter.authentication/1.18.0" symbolicname="org.mediacenter.authentication" presentationname="Media Center Authentication" uri="http://nexus.crestin.tv/nexus/content/repositories/releases/org/mediacenter/authentication/1.18/authentication-1.18.jar" version="1.18.0">
            <description>This bundle is a fragment attached to the Sling FormAuth bundle and provides        a simple customized login form.           </description>
            <size>6362</size>
            <category id="mediacenter"/>
            <capability name="bundle">
                <p n="symbolicname" v="org.mediacenter.authentication"/>
                <p n="presentationname" v="Media Center Authentication"/>
                <p n="version" t="version" v="1.18.0"/>
                <p n="manifestversion" v="2"/>
            </capability>
            <capability name="fragment">
                <p n="host" v="org.apache.sling.auth.selector"/>
                <p n="version" t="version" v="0.0.0"/>
            </capability>
            <require name="bundle" filter="(&amp;(symbolicname=org.apache.sling.auth.selector))" extend="true" multiple="false" optional="false">Required Host org.apache.sling.auth.selector</require>
        </resource>
    </repository>
  4. Go into the Felix Web Console, into the OSGI repository and add
    http://obr.mydomain.com/repository.xml

    as a new repository.

    Apache Felix OSGI Repository

Where to go from here:

  1. Apache Felix Maven Bundle Plugin
  2. Maven Release Plugin
  3. Using the Apache Felix Web Management Console