ESUP-Portail 20 sept. 2017 @Paris ESUP-Days #24

Travaux de déploiement
de Nuxeo, tant DevOps que Dev

  • Henri Jacob
  • Léa Raya DÉCORNOD, Céline PERVÈS
  • Université de Rennes1
  • Université de Strasbourg
  • Henri Jacob
  • Université de Rennes1

Mettre à jour Nuxeo
sans Nuxeo Online Services

Nuxeo as a Platform 
   

DEV OPS

  Léa Raya Décornod Céline Pervès

Contexte

À l'Université de Strasbourg, nous avons intégré la solution open-source
de la société Nuxeo comme support à nos besoins sur deux aspects :
  • en matière d'implémentation de processus métiers orientés documents, communément appelé GED,
  • et comme socle de notre solution de gestion du recrutement des enseignants/chercheurs (DéREC).
Nous avons optés pour un contrat de type “gold” qui nous ouvre
l'accès (payant) à Nuxeo Online Services (NOS) :
  • Nuxeo Studio (et son dépôt maven authentifié)
  • Accès authentifié aux dépôts maven hotfix-releases

Intégration au sein
de nos procédures & méthodes

  • Gestion des changements
    maitriser les changements
    afin de limiter leur impact
    • Changement
    • Gestion des changements
  • Reproductibilité
    re-créer est plus simple que réparer
  • Industrialisation des déploiements
    • gestion des environnements
      et des configurations
    • standardisation

La distribution nuxeo


est construite en assemblant des artéfacts maven

Artefacts maven

Les artéfacts maven peuvent provenir de :

  • de dépôts tiers (publics ou authentifiés) :
    • publics : maven central, https://maven-eu.nuxeo.org/nexus/content/repositories/public-releases/
    • authentifiés :
      https://maven.nuxeo.org/nexus/content/repositories/hotfix-releases
      https://connect.nuxeo.com/nuxeo/site/studio/maven
  • de résultats produits :
    mvn package
    …/target/….jar
  • du dépôt local :
    mvn install

    ${home}/.m2/repository/org/nuxeo/runtime/nuxeo-runtime/7.10-HF24/nuxeo-runtime-7.10-HF24.jar
  • d'un autre dépôt :
    mvn -DaltDeploymentRepository="mon-repo::default::file:///srv/repository/" deploy 

assemblage d'une distribution

yakafokon


« 
— Pouvez-vous ?
— Je le peux.
— Vous pouvez vraiment le faire ?
— Oui…
— Iel peut le faire ! Alors on l’applaudit très fort !
— Pour ce qui nous concerne, n’applaudissez pas trop vite… Attendez d’avoir lu le rapport.  »

PackWar

Documenté chez Nuxeo sous “Deploying as a Standard Static WAR”
nuxeoctl pack /tmp/nuxeo-war.zip
org.nuxeo.runtime.deployment.preprocessor.PackWar (github nuxeo)
PackWar.java
…
package org.nuxeo.runtime.deployment.preprocessor;
…
/**
 * Packs a Nuxeo Tomcat instance into a WAR file inside a ZIP.
 */
public class PackWar {
…
          

« Keep in mind that when you do this, the following dynamic features will not work (we are inside a static WAR):

  • Nuxeo Studio hot reload
  • Nuxeo Marketplace integration (hotfixes and packages installation) »

Packaging

jDeb https://github.com/tcurdt/jdeb

Ant task and a Maven plugin to create Debian packages from Java builds

pom.xml
       
        org.vafer
        jdeb
        1.5
        
          
            package
            jdeb
            
              true
              
                
                
                  ${project.build.directory}/….zip
                  archive
                  webapps/nuxeo/**
                  
                    perm
                    2
                    
                        /var/lib/tomcat7/webapps/nuxeo
                    root
                    root
                    644
                    755
                  
                
              
            
          
        
      
          

Intégration Continue

Nous utilisons Gitlab comme forge logicielle locale.
  • historique : git
  • suivi des itérations : issues, milestones
  • documentations (équipes de dev) : wiki
  • intégration continue : build, test
    • apt-get install gitlab-ci-multi-runner docker.io
      , .gitlab-ci.yml
  • déploiement continu : deploy, environment
    parallel-ssh -l root -h servers.list -it0 -- DEBIAN_FRONTEND=noninteractive apt-get -y "/…/….deb"

Outillage jUnit

Nuxeo utilise, fournit et documente
tout un outillage jUnit.

https://doc.nuxeo.com/corg/unit-testing/

Outillage jUnit


@RunWith(FeaturesRunner.class)
@Features(PlatformWithStudioBundleFeature.class)
@Deploy({"org.nuxeo.ecm.core.event", …})
public class FseanceConseilEHTest {
    @Inject
    protected CoreSession session;

    @Before
    public void setUpRepository() throws Exception {
        // empty repository
        session.removeChildren(session.getRootDocument().getRef());
        session.save();
        // create top folder for tests
        DocumentModel testDom = createDocument(session,
                "/", "test", "Domain",
                "dc:title", "Test Domain");
        grantACE(testDom, "testUser", SecurityConstants.READ);
        …
        session.saveDocuments(new DocumentModel[] { testDom });
        session.save();
    }
    protected Calendar getDUA(Calendar date) {
      return new DateWrapper(date).years(5).getCalendar();
    }

    @Test
    public void testCreateSeanceConseil() throws Exception {
      log.trace("------- testCreateSeanceConseil ------");
      DocumentModel testFile = createDocument(userSession,
            currentDocument.getPath(),
            "test-seance", "FseanceConseil",
            "dgu:DateDoc", testDate);
      assertEquals("Séance du 2001-02-03",
            testFile.getPropertyValue("dc:title"));
      assertEquals(currentDocument.getName()+ "-2001-02-03",
            testFile.getName());
      assertEquals(currentDocument.getPropertyValue("dc:nature"),
            testFile.getPropertyValue("dc:nature"));
      assertTrue("need facet UITypesLocalConfiguration",
            testFile.hasFacet("UITypesLocalConfiguration"));
      assertEquals("Dconseils",
            testFile.getPropertyValue("uitypesconf:defaultType"));
      assertArrayEquals(new String[] { "Dconseils" },
            (String[])testFile.getPropertyValue("uitypes…:allow…"));
      assertEquals(getDUA(testDate),
            testFile.getPropertyValue("dgu:DUA"));
    }
}
          

Outillage jUnit




  org.nuxeo.ecm.directory.sql.storage
  
    
  

          

@RunWith(FeaturesRunner.class)
@Features(RuntimeFeature.class)
@Deploy({
  "org.nuxeo.ecm.core:OSGI-INF/CoreExtensions.xml", "org.nuxeo.ecm.core.schema",
  "org.nuxeo.ecm.directory.api", "org.nuxeo.ecm.directory", "org.nuxeo.ecm.directory.sql",
})
@LocalDeploy({
  "fr.unistra.di.metier.dip.ged.nuxeo.nuxeo-ldap-uds:OSGI-INF/ldap-uds-schema-contrib.xml",
  "org.nuxeo.ecm.directory.sql.storage:default-sql-directories-bundle.xml",
})
public class LDAPSchemaTest {
  @Inject private DirectoryService directorySvc;
  @Test public void disabledDefaultSQLUserDirectoryTest() {
    assertTrue(
      "at least one userDirectory of type SQLDirectory exists",
      directorySvc.getDirectories().stream()
        .noneMatch(
            directory -> directory instanceof SQLDirectory
                      && directory.getName().equals("userDirectory")
      )
    );
  }
}
          

Déploiement Continu

org.nuxeo.launcher.config.ConfigurationGenerator dispose d'une main
→ on peut l'appeler depuis un postinst et regénérer les configurations depuis les templates et le nuxeo.conf

text-template.js
var ConfigurationGenerator = Java.type("org.nuxeo.launcher.config.ConfigurationGenerator")
java.lang.System.setProperty(ConfigurationGenerator.NUXEO_CONF, "/etc/nuxeo.conf")
var confGen = new ConfigurationGenerator()
var TextTemplate = Java.type("org.nuxeo.common.utils.TextTemplate")
var templateParser = new TextTemplate(confGen.userConfig)
templateParser.processText(java.lang.System.in, java.lang.System.out)
          
postinst
__update_tomcat_server_xml() {
	echo "rebuild $tomcat's server.xml and context"
	# rebuild tomcat's configuration
	$package_dir/bin/text-template.js \
		"$package_dir/templates/common-base/conf/server.xml.nxftl" \
		"/etc/$tomcat/server.nuxeo.xml" \
		"$package_dir/templates/common-base/conf/Catalina/localhost/ROOT.xml" \
		"/etc/$tomcat/Catalina/localhost/ROOT.xml" \
		"$package_dir/templates/common-base/conf/Catalina/localhost/nuxeo.xml.nxftl" \
		"/etc/$tomcat/Catalina/localhost/nuxeo.xml"
    displace_link /etc/$tomcat/server .xml
}
          
postinst
## check tomcat uid/gid is coherent across NFS (cluster)
__assert_cluster_grade_tomcat_uid_gid(){
  repository_clustering="$($package_dir/bin/text-template.js <<< '${repository.clustering.enabled}')"
  repository_binary_store="$($package_dir/bin/text-template.js <<< '${repository.binary.store}')"
  if [ "true" = "${repository_clustering}" ]
  then
    echo 'check binary store'
    ## check repository_binary_store is on NFS if repository_clustering=true
    if [ ! -e "$repository_binary_store" \
       -o "xnfs" != "x$(df -T "$repository_binary_store" | awk 'NR==2 {print $2}')" ]
    then
      db_input critical [[artifactId]]/cluster_not_nfs || true
      db_go
      exit 1
    fi
    if [ -n "$repository_binary_store" -a -d "$repository_binary_store" \
       -a -e "$repository_binary_store/config.xml" ]
    then
      ## check repository_binary_store's owner uid/gid
      BS_IDs="$(stat -c '%u/%g' "$repository_binary_store/config.xml")"
      TC_IDs="$(id -u $tomcat)/$(id -g $tomcat)"
      if [ "$TC_IDs" != "$BS_IDs" ]
      then
        db_input critical [[artifactId]]/cluster_ids_mismatch || true
          

Nuxeo as a Framework 
   

DEV

  Léa Raya Décornod Céline Pervès

Bundles OSGi dans Nuxeo

digraph NXPlatform {
  bgcolor=transparent
  id="\G"
  fontsize=11
  node [ id="\G-\N", shape=rect, style=filled, fillcolor=white, fontsize=10 ]
  edge [ id="\G-\E", fontsize=10 ]

  subgraph cluster_bundle {
    style=filled; fillcolor="#dcf0ff";
    label="fr.univ.groupId : artefactId : version:.jar (pom.xml)";
    labelloc=b;
  manifest [ label="META-INF/MANIFEST.MF\lBundle-SymbolicName: fr.univ.mon.bundle" ];
  
  { rank=same
  contrib1 [ label="OSGI-INF/une-contrib.xml\l<component name=“fr.univ.contrib”>" ];
  contribFRWK [ label="OSGI-INF/un-framework.xml\l<component name=“my.target”>" ];
  }

  manifest -> contrib1 [sametail=C, taillabel="Nuxeo-Component:"];
  manifest -> contribFRWK [sametail=C];

  { rank=same
  extension1 [ label="<extension\l  target=“my.target”\l  point=“xtpoint”>"];
  xtpointFRWK [ label="<extension-point\l   name=“xtpoint”>\l" ];
  }
  extension2 [ label="<extension\l  target=“…”\l  point=“…”>"];

  serviceSRV [ label="<service><provide\l    interface=“fr.univ.my.ServiceAPI” />\l"];
  implFROB [ label="<implementation\l    class=“fr.univ.my.ServiceImpl” />\l"];
  objectFROB [ label="<object class=“fr.univ.my.Descriptor”>" ];

  { rank=same
  DescriptorIMPL [ label="@XObject(“truc”)\lclass fr.univ.my.Descriptor\l{\l  @XNode(“@name”)\l  private String id;\l}\l" ]
  ServiceIMPL [ label="class fr.univ.my.ServiceImpl\l  implements fr.univ.my.ServiceAPI {\l    public void registerContribution(…)\l    { … }\l}\l" ];
  XPcontrib [ label="<truc name=“bob”>\l  <elem>val</elem>\l</truc>\l" ]
  }

  { edge [arrowtail=diamond, dir=back];
  contrib1 -> extension1 -> XPcontrib;
  contrib1 -> extension2 [minlen=2];
  contribFRWK -> serviceSRV;
  contribFRWK -> implFROB [minlen=2];
  contribFRWK -> xtpointFRWK -> objectFROB;
  }

  objectFROB -> DescriptorIMPL [dir=none];
  implFROB   -> ServiceIMPL [dir=none];
  serviceSRV-> ServiceIMPL [dir=back,arrowtail=onormal,style=dashed];

  deploymentDPL [ label="OSGI-INF/deployment-fragment.xml"];
  jsfimgDPL [ label="web/nuxeo.war/incl/….xhtml\lweb/nuxeo.war/img/icon.png\l" ] 
  msgDPL [ label="OSGI-INF/\l  l10n/\l    messages_fr_FR.properties\l" ] 

  deploymentDPL -> jsfimgDPL;
  deploymentDPL -> msgDPL [minlen=2];

  propSEAM [label="seam.properties"];
  beanSEAM [ label="@Name(“monBean”)\lpublic class MonBean {\l  @In protected transient\l    CoreSession docMgr;\l}\l" ];

  msgDPL -> propSEAM -> beanSEAM [style=invis];
  jsfimgDPL -> beanSEAM [color=gray];
  }

  {
    node [ style=none, label="", width=0, height=0]
    edge [ arrowhead=odot, constraint=false]
    publicXPFRWK [shape=none];
    publicSRV [shape=none];
    xtpointFRWK -> publicXPFRWK
    serviceSRV -> publicSRV;
  }
  newrank=true;
  { rank=same xtpointFRWK publicXPFRWK }
  { rank=same serviceSRV publicSRV }

  { edge [arrowhead=vee, style=dashed, constraint=false];
  contrib1 -> contribFRWK [color=red,label="<require>my.target</require>"];
  extension1 -> publicXPFRWK    [color=green];
  publicXPFRWK   -> implFROB    [color=blue];
  XPcontrib  -> DescriptorIMPL  [color=green];
  DescriptorIMPL -> ServiceIMPL [color=blue];
  }

}
          

configurations/paramétrages

META-INF/MANIFEST.MF
Manifest-Version: 1.0
Bundle-Vendor: UdS
Bundle-Name: nuxeo-theme-uds
Bundle-SymbolicName: fr.unistra.ged.theme
Bundle-Description: Redefine footer links
Bundle-Version: 3.0.0-SNAPSHOT
Bundle-ManifestVersion: 2
Nuxeo-Component: OSGI-INF/footer-actions-contrib.xml
            
OSGI-INF/footer-actions-contrib.xml
<?xml version="1.0"?>
<component name="fr.unistra.ged.footer-actions">
  <require>org.nuxeo.ecm.platform.actions</require>
  <extension
 target="org.nuxeo.ecm.platform.actions.ActionService"
      point="actions">
    <!-- Redefine FOOTER actions list -->
    <action id="footer_contact_us" enabled="false" />
    <action id="footer_blogs" enabled="false" />
    <action id="footer_community" enabled="false" />
    <action id="footer_answers" enabled="false" />
    <action id="footer_documentation"
      link="https://www.unistra.fr/index.php?id=7548"
      label="label.footer.documentation"
      order="50" type="bare_link">
      <category>FOOTER</category>
      <properties>
        <property name="target">_blank</property>
      </properties>
    </action>
    <action id="footer_twitter" enabled="false" />
    <action id="footer_linkedin" enabled="false" />
  </extension>
</component>
            

configurations/paramétrages

META-INF/MANIFEST.MF
Manifest-Version: 1.0
Bundle-Vendor: UdS
Bundle-Name: nuxeo-theme-uds
Bundle-SymbolicName: fr.unistra.ged.theme
Bundle-Description: Redefine footer links
Bundle-Version: 3.0.0-SNAPSHOT
Bundle-ManifestVersion: 2
Nuxeo-Component: OSGI-INF/footer-actions-contrib.xml,
  OSGI-INF/rights-forall-contrib.xml
            

OSGI-INF/footer-actions-contrib.xml
<?xml version="1.0"?>
<component name="fr.unistra.ged.footer-actions">
  …
            

OSGI-INF/rights-forall-contrib.xml
<?xml version="1.0"?>
<component name="fr.unistra.ged.rightsForAll">
  <require>org.nuxeo.ecm.platform.actions</require>
  <extension
 target="org.nuxeo.ecm.platform.actions.ActionService"
      point="filters">
    
    
      
        WriteSecurity
      
    
  </extension>
</component>
            

code JAVA périphérique

META-INF/MANIFEST.MF
Manifest-Version: 1.0
Bundle-Vendor: UdS
Bundle-Name: nuxeo-ooo-killer
Bundle-SymbolicName: fr.unistra.ged.oooKiller
Bundle-Description: Kill stucked OOo
Bundle-Version: 3.0.0-SNAPSHOT
Bundle-ManifestVersion: 2
Nuxeo-Component: OSGI-INF/ooo-killer-framework.xml
            

OSGI-INF/ooo-killer-framework.xml
<?xml version="1.0"?>

  <require>org.nuxeo.ecm.platform.convert.ooomanager.…
  <implementation
          class="fr.unistra.ged.OOoKiller" />

            
fr/unistra/ged/OOoKiller.java
package fr.unistra.ged;
public class OOoKiller
             extends DefaultComponent {
  public int getApplicationStartedOrder()
  { return 1010; }
  public void applicationStarted(
            ComponentContext context) {
    OOoManagerComponent oooManager =
      (OOoManagerComponent)
        Framework.getService(OOoManagerService.class);
    if (! oooManager.isOOoManagerStarted()) {
      // Kill conflicting processes
      new ProcessBuilder(new String[] {
         "pkill", "-f", "soffice.bin.*jodconverter"
      }).start();
      // retry starting
      oooManager.applicationStarted(context);
} } }
            

code JAVA intégré

META-INF/MANIFEST.MF
Manifest-Version: 1.0
Bundle-Vendor: UdS
Bundle-Name: derec
Bundle-SymbolicName: fr.unistra.derec
Bundle-Description: Démat Recrut EC
Bundle-Version: 2.0.0-SNAPSHOT
Bundle-ManifestVersion: 2
Nuxeo-Component: OSGI-INF/listeners-contrib.xml
            
OSGI-INF/listeners-contrib.xml
<?xml version="1.0"?>

  <extension
    target="org.nuxeo.ecm.core.event.EventServiceComponent"
    point="listener">
    <listener name="applicationremovedlistener" async="false" postCommit="true"
              class="fr.unistra.derec.event.ApplicationRemovedListener" order="100">
      aboutToRemove
    </listener>
  </extension>

            

fr/unistra/derec/event/ApplicationRemovedListener.java
package fr.unistra.derec.event;
public class ApplicationRemovedListener implements EventListener {
  public void handleEvent(Event event) {
    DocumentModel doc = ((DocumentEventContext) event.getContext()).getSourceDocument();
    if (! "Application".equals(doc.getType())) return;
    String candidateId = (String) doc.getPropertyValue("application:candidate_id");
    String jobId =       (String) doc.getPropertyValue("application:job_id");
    UserManager userMgr = Framework.getLocalService(UserManager.class);
    userMgr.deleteUser( "c" + candidateId + "-" + jobId );
  }
}
            

composant complet

gestion avancée des vocabulaires