Friday, 23 November 2012

Deploy Soa Suite 11g composite applications with Ant scripts

With Soa Suite 11g you can deploy your composite applications from JDeveloper or with ANT. In this blog I will do this with the SOA 11g ANT scripts. These ant scripts can only deploy one project so I made an ANT script around the SOA ANT scripts which can deploy one or more composites applications to different SOA environments. So now you can use it to automate your deployment or use it in your build tool.
In my ant script I will deploy shared artifacts to the MDS, compile, build and package the composite applications and deploy this to the SOA Server. After this I use an ANT script to start the unit tests and generate a JUnit result XML and at last I can optional disable the composite.
This JUnit XML can be used in your continuous build system. You can easily extend this build script so you use it to manage the composite applications.
For more info over ANT deployment see the official deployment documentation .
The official ANT scripts are located in the jdeveloper\bin folder. Here is a summary of the scripts and what they can do
  • ant-sca-test.xml, This script can start the test suites of the composite and generates a juinit report and not Attaches, extracts, generates, and validates configuration plans for a SOA composite application, The official documentation description is not correct.
  • ant-sca-compile.xml, Compiles a SOA composite application ,this script is also called in the package scrip, so we don't need to call this directly.
  • ant-sca-package.xml, Packages a SOA composite application into a composite SAR file and also validates and build the composite application.
  • ant-sca-deploy.xml, Deploys a SOA composite application.
  • ant-sca-mgmt.xml, Manages a SOA composite application, including starting, stopping, activating, retiring, assigning a default revision version, and listing deployed SOA composite applications.

Here is the main build.properties where you have to define the jdeveloper and your application home, which composite applications you want to deploy and what is the environment dev or acc.

# demo = true , then no soa scripts will be called.
demo.mode=false
# global
wn.bea.home=C:/oracle/MiddlewareJdev11gR1PS3
java.passed.home=${wn.bea.home}/jdk1.6.0_23
# PS4
#wn.bea.home=D:/Oracle/MiddlewareJDevPS4
#java.passed.home=${wn.bea.home}/jdk160_24
oracle.home=${wn.bea.home}/jdeveloper
wl_home=${wn.bea.home}/wlserver_10.3
# temp
tmp.output.dir=c:/temp
junit.output.dir=../../
# my settings
applications.home=../../applications
applications=HelloWorld
# my settings
mds.repository=C:/oracle/MiddlewareJdev11gR1PS3/jdeveloper/integration/seed/apps/
mds.applications=Woningnet-Test
#demo applications
#applications.home=workspaces
#applications=wrkspc1,wrkspc2
#demo mds locations
#mds.repository=mds/seed/apps/
#mds.applications=company,common
mds.enabled=true
mds.undeploy=true
deployment.plan.environment=dev
# dev deployment server weblogic
dev.serverURL=http://laptopedwin:7001
dev.overwrite=true
dev.user=weblogic
dev.password=weblogic1
dev.forceDefault=true
dev.server=laptopedwin
dev.port=7001
# acceptance deployment server weblogic
acc.serverURL=http://laptopedwin:7001
acc.overwrite=true
acc.user=weblogic
acc.password=weblogic1
acc.forceDefault=true
acc.server=laptopedwin
acc.port=8001

Every application can have one or more SOA projects so the main ant script will load the application properties file which contains all the project with its revision number.
Here is a example of SoaEjbReference.properties file
  1. projects=Helloworld  
  2. Helloworld.revision=1.0  
  3. Helloworld.enabled=true  
  4. Helloworld.partition=default  

Because in my example I have two soa environments so I need to create two configuration plans. With this plan ( which look the wls plan ) can change the url of endpoints so it matches with the environment.
Select the composite application xml and generate a configuration plan.
Add the dev or acc extension to the file name.
Here you see how the plan looks like.



And here is the main ANT build script which can do it all and calls the Oracle ANT scripts.
<?xml version="1.0" encoding="iso-8859-1"?>
<project name="soaDeployAll" default="deployAll">
    <property environment="env"/>
    <property file="${env.CURRENT_FOLDER}/build.properties"/>
    <!-- Antcontrib path -->
    <path id="antcontrib.path">
      <pathelement path="lib/ant-contrib-1.0b3.jar" />
    </path>
    <taskdef classpathref="antcontrib.path"
             resource="net/sf/antcontrib/antlib.xml"/>
    <target name="deployAll">
       <!-- Build time -->
       <tstamp>
          <format property="build.date" pattern="yyyy-MM-dd HH:mm:ss" />
       </tstamp>
       <!-- Build number -->
       <condition property="build.number" value="${env.BUILD_NUMBER}">
          <isset property="env.BUILD_NUMBER" />
       </condition>
       <buildnumber file="build.num" />
<echo message="date = ${build.date}"
level="info"/>
<echo message="build = ${build.number}"
level="info"/>
       <echo file="logs/instance-${build.number}.log" append="true"
             message="deployAll${line.separator}"
             level="info"/>
       <echo file="logs/instance-${build.number}.log" append="true"
             message="basedir ${basedir}${line.separator}"
             level="debug"/>
       <echo file="logs/instance-${build.number}.log" append="true"
             message="current folder ${env.CURRENT_FOLDER}${line.separator}"
             level="debug"/>
<echo file="logs/instance-${build.number}.log" append="true"
message="date = ${build.date}${line.separator}"
level="info"/>
<echo file="logs/instance-${build.number}.log" append="true"
message="build = ${build.number}${line.separator}"
level="info"/>
<echo file="logs/instance-${build.number}.log" append="true"
message="environment = ${deployment.plan.environment}${line.separator}"
level="info"/>
       <mkdir dir="builds/${build.number}"/>
     <if>
          <equals arg1="${mds.enabled}" arg2="true"/>
          <then>
             <antcall target="deployMDS" inheritall="true"/>
          </then>
      </if>
      <foreach list="${applications}"
       param="application"
       target="deployApplication"
       inheritall="true"
       inheritrefs="false"/>
    </target>
    <target name="deployMDS">
        <echo message="undeploy and deploy MDS"
              level="info"/>
        <echo file="logs/instance-${build.number}.log" append="true"
              message="->deployMDS undeploy and deploy MDS${line.separator}"
              level="info"/>
        <if>
          <equals arg1="${mds.undeploy}" arg2="true"/>
          <then>
            <foreach list="${mds.applications}"
             param="mds.application"
             target="undeployMDSApplication"
             inheritall="true"
             inheritrefs="false"/>
          </then>
        </if>
        <foreach list="${mds.applications}"
         param="mds.application"
         target="deployMDSApplication"
         inheritall="true"
         inheritrefs="false"/>
    </target>
    <target name="deployMDSApplication">
        <echo message="deploy MDS application ${mds.application}"
              level="info"/>
        <echo file="logs/instance-${build.number}.log" append="true"
              message="${line.separator}-->deployMDSApplication deploy MDS application ${mds.application}${line.separator}"
              level="info"/>
        <echo message="remove and create local MDS temp"
              level="debug"/>
        <property name="mds.deploy.dir" value="${tmp.output.dir}/${mds.application}"/>
        <delete dir="${mds.deploy.dir}"/>
        <mkdir dir="${mds.deploy.dir}"/>
        <echo message="create zip from file MDS store"
              level="debug"/>
<zip destfile="${mds.deploy.dir}/${mds.application}_mds.jar" compress="false">
     <fileset dir="${mds.repository}" includes="${mds.application}/**"/>
     </zip>
        <echo message="create zip with MDS jar"
              level="debug"/>
<zip destfile="${mds.deploy.dir}/${mds.application}_mds.zip" compress="false">
     <fileset dir="${mds.deploy.dir}" includes="*.jar"/>
     </zip>
        <propertycopy name="deploy.serverURL" from="${deployment.plan.environment}.serverURL"/>
        <propertycopy name="deploy.overwrite" from="${deployment.plan.environment}.overwrite"/>
        <propertycopy name="deploy.user" from="${deployment.plan.environment}.user"/>
        <propertycopy name="deploy.password" from="${deployment.plan.environment}.password"/>
        <propertycopy name="deploy.forceDefault" from="${deployment.plan.environment}.forceDefault"/>
        <echo message="deploy on ${deploy.serverURL} with user ${deploy.user}"
              level="info"/>
        <echo message="deploy sarFile ${mds.deploy.dir}/${mds.application}_mds.zip"
              level="info"/>
        <echo file="logs/instance-${build.number}.log" append="true"
              message="---->deploy on ${deploy.serverURL} with user ${deploy.user}${line.separator}"
              level="info"/>
        <echo file="logs/instance-${build.number}.log" append="true"
              message="---->deploy sarFile ${mds.deploy.dir}/${mds.application}_mds.zip${line.separator}"
              level="info"/>
        <copy todir="builds/${build.number}"
              file="${mds.deploy.dir}/${mds.application}_mds.zip"/>
     <if>
          <equals arg1="${demo.mode}" arg2="false"/>
          <then>
            <ant antfile="${oracle.home}/bin/ant-sca-deploy.xml" inheritAll="false" target="deploy">
             <property name="wl_home" value="${wl_home}"/>
             <property name="oracle.home" value="${oracle.home}"/>
             <property name="serverURL" value="${deploy.serverURL}"/>
             <property name="user" value="${deploy.user}"/>
             <property name="password" value="${deploy.password}"/>
             <property name="overwrite" value="${deploy.overwrite}"/>
             <property name="forceDefault" value="${deploy.forceDefault}"/>
             <property name="sarLocation" value="${mds.deploy.dir}/${mds.application}_mds.zip"/>
            </ant>
          </then>
        </if>
    </target>
    <target name="undeployMDSApplication">
        <echo message="undeploy MDS application ${mds.application}"
              level="info"/>
        <echo file="logs/instance-${build.number}.log" append="true"
              message="${line.separator}-->undeployMDSApplication undeploy MDS application ${mds.application}${line.separator}"
              level="info"/>
        <propertycopy name="deploy.serverURL" from="${deployment.plan.environment}.serverURL"/>
        <propertycopy name="deploy.overwrite" from="${deployment.plan.environment}.overwrite"/>
        <propertycopy name="deploy.user" from="${deployment.plan.environment}.user"/>
        <propertycopy name="deploy.password" from="${deployment.plan.environment}.password"/>
        <propertycopy name="deploy.forceDefault" from="${deployment.plan.environment}.forceDefault"/>
        <echo message="undeploy MDS app folder apps/${mds.application}"
              level="info"/>
        <echo file="logs/instance-${build.number}.log" append="true"
              message="---->undeploy MDS app folder apps/${mds.application}${line.separator}"
              level="info"/>
     <if>
          <equals arg1="${demo.mode}" arg2="false"/>
          <then>
            <ant antfile="${oracle.home}/bin/ant-sca-deploy.xml" inheritAll="false" target="removeSharedData">
          <property name="wl_home" value="${wl_home}"/>
              <property name="oracle.home" value="${oracle.home}"/>
              <property name="serverURL" value="${deploy.serverURL}"/>
              <property name="user" value="${deploy.user}"/>
              <property name="password" value="${deploy.password}"/>
              <property name="folderName" value="${mds.application}"/>
            </ant>
          </then>
        </if>
    </target>
    <target name="deployApplication">
        <echo message="deploy application ${application}"
              level="info"/>
        <echo file="logs/instance-${build.number}.log" append="true"
              message="${line.separator}-->deployApplication deploy application ${application}${line.separator}"
              level="info"/>
        <property file="${env.CURRENT_FOLDER}/${applications.home}/${application}/build.properties"/>
        <foreach list="${projects}" param="project" target="deployProject" inheritall="true" inheritrefs="false"/>
    </target>
    <target name="deployProject">
        <echo message="deploy project ${project} for environment ${deployment.plan.environment}"
              level="info"/>
        <echo file="logs/instance-${build.number}.log" append="true"
              message="${line.separator}---->deployProject deploy project ${project} for environment ${deployment.plan.environment}${line.separator}"
              level="info"/>
        <property name="proj.compositeName" value="${project}"/>
        <property name="proj.compositeDir" value="${env.CURRENT_FOLDER}/${applications.home}/${application}"/>
        <propertycopy name="proj.revision" from="${project}.revision"/>
        <propertycopy name="proj.enabled" from="${project}.enabled"/>
        <propertycopy name="proj.partition" from="${project}.partition"/>
        <echo message="partition ${proj.partition} compositeName ${proj.compositeName} compositeDir ${proj.compositeDir}"
              level="info"/>
        <echo file="logs/instance-${build.number}.log" append="true"
              message="------>partition ${proj.partition}${line.separator}"
              level="info"/>
        <echo file="logs/instance-${build.number}.log" append="true"
              message="------>compositeName ${proj.compositeName}${line.separator}"
              level="info"/>
        <echo file="logs/instance-${build.number}.log" append="true"
              message="------>revision ${proj.revision}${line.separator}"
              level="info"/>
        <echo file="logs/instance-${build.number}.log" append="true"
              message="------>compositeDir ${proj.compositeDir}${line.separator}"
              level="info"/>
        <echo message="build sar package"
              level="info"/>
        <echo file="logs/instance-${build.number}.log" append="true"
              message="------>build sar package${line.separator}"
              level="info"/>
     <if>
          <equals arg1="${demo.mode}" arg2="false"/>
          <then>
            <ant antfile="${oracle.home}/bin/ant-sca-package.xml" inheritAll="false" target="package">
             <property name="compositeDir" value="${proj.compositeDir}/${project}"/>
             <property name="compositeName" value="${proj.compositeName}"/>
             <property name="revision" value="${proj.revision}"/>
             <property name="oracle.home" value="${oracle.home}"/>
             <property name="java.passed.home" value="${java.passed.home}"/>
             <property name="wl_home" value="${wl_home}"/>
             <property name="sca.application.home" value="${proj.compositeDir}"/>
             <property name="scac.application.home" value="${proj.compositeDir}"/>
             <property name="scac.input" value="${proj.compositeDir}/${proj.compositeName}/composite.xml"/>
             <property name="scac.output" value="${tmp.output.dir}/${proj.compositeName}.xml"/>
             <property name="scac.error" value="${tmp.output.dir}/${proj.compositeName}.err"/>
             <property name="scac.displayLevel" value="3"/>
            </ant>
            <copy todir="builds/${build.number}"
                  file="${proj.compositeDir}/${proj.compositeName}/deploy/sca_${proj.compositeName}_rev${proj.revision}.jar"/>
          </then>
        </if>
        
        <property name="deploy.sarLocation"
         value="${proj.compositeDir}/${proj.compositeName}/deploy/sca_${proj.compositeName}_rev${proj.revision}.jar"/>
        <property name="deploy.configplan"
         value="${proj.compositeDir}/${proj.compositeName}/${proj.compositeName}_cfgplan_${deployment.plan.environment}.xml"/>
        <propertycopy name="deploy.serverURL" from="${deployment.plan.environment}.serverURL"/>
        <propertycopy name="deploy.overwrite" from="${deployment.plan.environment}.overwrite"/>
        <propertycopy name="deploy.user" from="${deployment.plan.environment}.user"/>
        <propertycopy name="deploy.password" from="${deployment.plan.environment}.password"/>
        <propertycopy name="deploy.forceDefault" from="${deployment.plan.environment}.forceDefault"/>
        <propertycopy name="deploy.server" from="${deployment.plan.environment}.server"/>
        <propertycopy name="deploy.port" from="${deployment.plan.environment}.port"/>
        <echo message="deploy on ${deploy.serverURL} with user ${deploy.user}"
              level="info"/>
        <echo file="logs/instance-${build.number}.log" append="true"
              message="------>deploy on ${deploy.serverURL} with user ${deploy.user}${line.separator}"
              level="info"/>
        <echo message="deploy sarFile ${deploy.sarLocation}"
              level="info"/>
        <echo file="logs/instance-${build.number}.log" append="true"
              message="------>deploy sarFile ${deploy.sarLocation}${line.separator}"
              level="info"/>
        <echo file="logs/instance-${build.number}.log" append="true"
              message="------>deployment plan used ${deploy.configplan}${line.separator}"
              level="info"/>
     <if>
          <equals arg1="${demo.mode}" arg2="false"/>
          <then>
<ant antfile="${oracle.home}/bin/ant-sca-deploy.xml" inheritAll="false" target="deploy">
<property name="wl_home" value="${wl_home}"/>
<property name="oracle.home" value="${oracle.home}"/>
<property name="serverURL" value="${deploy.serverURL}"/>
<property name="user" value="${deploy.user}"/>
<property name="password" value="${deploy.password}"/>
<property name="overwrite" value="${deploy.overwrite}"/>
<property name="forceDefault" value="${deploy.forceDefault}"/>
<property name="sarLocation" value="${deploy.sarLocation}"/>
<property name="configplan" value="${deploy.configplan}"/>
<property name="partition" value="${proj.partition}"/>
</ant>
          </then>
        </if>
        
        <if>
          <equals arg1="${proj.enabled}" arg2="false"/>
          <then>
<echo message="stop ${proj.compositeName}"
     level="info"/>
         <echo file="logs/instance-${build.number}.log" append="true"
             message="------>stop ${proj.compositeName}${line.separator}"
               level="info"/>
<if>
<equals arg1="${demo.mode}" arg2="false"/>
<then>
<ant antfile="${oracle.home}/bin/ant-sca-mgmt.xml" inheritAll="false" target="stopComposite">
<property name="host" value="${deploy.server}"/>
<property name="port" value="${deploy.port}"/>
<property name="user" value="${deploy.user}"/>
<property name="password" value="${deploy.password}"/>
<property name="compositeName" value="${proj.compositeName}"/>
<property name="revision" value="${proj.revision}"/>
<property name="partition" value="${proj.partition}"/>
</ant>
</then>
</if>
          </then>
        </if>
        <if>
          <equals arg1="${proj.enabled}" arg2="true"/>
          <then>
<echo message="stop activate ${proj.compositeName}"
     level="info"/>
         <echo file="logs/instance-${build.number}.log" append="true"
             message="------>activate ${proj.compositeName}${line.separator}"
               level="info"/>
<if>
<equals arg1="${demo.mode}" arg2="false"/>
<then>
<ant antfile="${oracle.home}/bin/ant-sca-mgmt.xml" inheritAll="false"
target="activateComposite">
<property name="host" value="${deploy.server}"/>
<property name="port" value="${deploy.port}"/>
<property name="user" value="${deploy.user}"/>
<property name="password" value="${deploy.password}"/>
<property name="compositeName" value="${proj.compositeName}"/>
<property name="revision" value="${proj.revision}"/>
<property name="partition" value="${proj.partition}"/>
</ant>
</then>
</if>
<echo message="unit test ${proj.compositeName}"
     level="info"/>
         <echo file="logs/instance-${build.number}.log" append="true"
             message="------>unit test ${proj.compositeName}${line.separator}"
               level="info"/>
<if>
<equals arg1="${demo.mode}" arg2="false"/>
<then>
<ant antfile="${oracle.home}/bin/ant-sca-test.xml" inheritAll="false"
target="test">
<property name="scatest.input" value="${project}"/>
<property name="scatest.partition" value="${proj.partition}"/>
<property name="scatest.format" value="junit"/>
<property name="scatest.result" value="${env.CURRENT_FOLDER}/${junit.output.dir}"/>
<property name="jndi.properties.input" value="${deployment.plan.environment}.jndi.properties"/>
</ant>
</then>
</if>
          </then>
        </if>
        <echo message="finish"
  level="info"/>
      <echo file="logs/instance-${build.number}.log" append="true"
          message="------>finish${line.separator}"
           level="info"/>
    </target>
</project>
view raw build.xml This Gist brought to you by GitHub.

For development testing environment I need to have dev.jndi.properties
java.naming.factory.initial=weblogic.jndi.WLInitialContextFactory
java.naming.provider.url=t3://localhost:8001/soa-infra
java.naming.security.principal=weblogic
java.naming.security.credentials=weblogic1
dedicated.connection=true
dedicated.rmicontext=true


And finally the CMD script to run this ANT script. To make this work we need the ant-contrib library and put this in the classpath of ANT or put it in the ANT lib folder.
  1. set ORACLE_HOME=C:\oracle\MiddlewareJdev11gR1PS3  
  2. set ANT_HOME=%ORACLE_HOME%\jdeveloper\ant  
  3. set PATH=%ANT_HOME%\bin;%PATH%  
  4. set JAVA_HOME=%ORACLE_HOME%\jdk1.6.0_23  
  5.   
  6. set CURRENT_FOLDER=%CD%  
  7.   
  8. ant -f build.xml deployAll  


See my github project for the source code https://github.com/biemond/soa_tools
Latest changes.

  • PS3 / PS4 support
  • Activation of the composite
  • partition support
  • Build number generator
  • Build logging
  • SAR and MDS z ipsfiles are bundled under build number.
  • Demo mode, in which you can test the ANT configuration without deploying
The new file structure with a logs and build folder.
 The logging of a run.

Ant scripts for Oracle SOA Suite 11g code deployment (Windows)

This blog post provides Ant scripts that simplify the deployment, undeployment, and management of code to Oracle SOA Suite 11g. These scripts are designed for Windows (will release the Linux instructions soon). For those familiar with it, I do not use the configuration plan (long story).

Who these scripts are designed for:

  • Developers who want an automated approach to deploy/undeploy/manage code
  • App Admins who want to manage code from a Windows-based operating system

What you can do with the scripts:
  • Deploy multiple composites using a single Ant command
  • Undeploy multiple composites using a single Ant command
  • Start multiple composites using a single Ant command
  • Stop multiple composites using a single Ant command
  • List deployed composites
  • Tokenize and detokenize code
  • Import artifacts (XSDs, DVMs, WSDLs, fault policies) to the MDS
  • Export the entire MDS (for backup or browsing purposes)

To see how these scripts work, check out:


One-time Setup and Configuration

1. Download ant_soa11g_windows.zip from here

2. Unzip ant_soa11g_windows.zip to your local c:\ drive (it should exist as c:\ant)

3. Edit setEnvironmentVars.bat and modify the following:
CODE_FOLDER  <-- top-level directory where your code exists (e.g., c:\ant\code)
ORACLE_HOME  <-- directory of JDev 11g (e.g., c:\jdev11g)
JAVA_HOME    <-- ensure that the JDK path is correct
4. Edit soa-environment.properties and modify only the following variables:
wn.bea.home=C:/jdev11g              <-- directory of JDev 11g (similar to ORACLE_HOME above)
tmp.output.dir=c:/ant/temp          <-- temp directory for MDS generation, default is c:\ant\temp
local.mds.repository=c:/ant/apps    <-- location of local MDS copy (for MDS imports)
local.mds.export=c:/ant/apps.backup <-- location of local MDS backup folder (for MDS exports)
5. Double-click on winInstall.bat so that it copies the necessary JARs to the appropriate directories

6. For every SOA Suite 11g environment, create the following 3 files.
For example, if you have a DEV, TEST, and PROD environments, then create the following 9 files:
soa-build-dev.properties
soa-build-test.properties
soa-build-prod.properties
soa-cfgplan-dev.xml
soa-cfgplan-test.xml
soa-cfgplan-prod.xml
soa-token-dev.properties
soa-token-test.properties
soa-token-prod.properties
The ant_soa11g_windows.zip file includes samples for a DEV environment, so just make copies of those files for your other environments.
7. Edit all soa-build-*.properties and soa-token-*.properties files and modify the following:
USERNAME    <-- weblogic username (e.g., 'weblogic')
PASSWORD    <-- weblogic password
SOAHOST     <-- hostname or IP address of SOA Suite 11g server
SOAPORT     <-- port for the soa_server1 managed server (e.g., 8001)
DBUSERNAME  <-- MDS database username (e.g., 'dev_mds')
DBPASSWORD  <-- MDS database password
DBHOST      <-- database host
DBPORT      <-- database port
DBSID       <-- database name

Deploy, Undeploy, Start, Stop, and Detokenize Composites

1. Define the CustomProcessList.txt (see Define the Custom Process List below)

2. Open a command prompt window and type the following
cd c:\ant
call setEnvironmentVars.bat
call ant -f build.soa.xml <target> -Dtargetenv=<env>
Where <target> is deployComposites, undeployComposites, startComposites, stopComposites, or detokenizeComposites. This will loop through and perform the action against all the composites listed in the CustomProcessList.txt file.
Where <env> is your dev, test, or whatever environments you configured earlier.

List Composites

1. Open a command prompt window and type the following
cd c:\ant
call setEnvironmentVars.bat
call ant -f build.soa.xml listComposites -Dtargetenv=<env>
Where <env> is your dev, test, or whatever environments you configured earlier.

Export MDS

1. Open a command prompt window and type the following
cd c:\ant
call setEnvironmentVars.bat
call ant -f build.soa.xml exportMDS -Dtargetenv=<env>
Where <env> is your dev, test, or whatever environments you configured earlier.
2. The export file will be saved to c:\ant\apps.backup in the format of MDSbackup.<env>.YYYY-MM-DD-HHMI.jar (e.g., MDSbackup.dev.2011-01-20_1140.jar)


Import Artifacts to MDS

1. Define the CustomMDSList.txt (see Define the Custom MDS List below)

2. Add all the schemas, WSDLs, DVMs, fault policies, to the c:\ant\apps\<application> directory.
For example:
c:\ant\apps\App1MetaData\dvm\Currency.dvm
c:\ant\apps\App1MetaData\dvm\Country.dvm
c:\ant\apps\App1MetaData\config\fault-bindings.xml
c:\ant\apps\App1MetaData\config\fault-policies.xml
c:\ant\apps\App1MetaData\schemas\shared\Types.xsd
c:\ant\apps\App1MetaData\schemas\hr\Account.xsd
3. Open a command prompt window and type the following
cd c:\ant
call setEnvironmentVars.bat
call ant -f build.soa.xml importMDS -Dtargetenv=<env>
Where <env> is your dev, test, or whatever environments you configured earlier.

Define the Custom Process List

Before using any of these Ant scripts that are related to code, you must create a CustomProcessList.txt file.

1. Create the c:\ant\CustomProcessList.txt file, using the following format:
<composite_name>,<partition>,<revision>,<default_composite>,<relative_directory_of_code>

For example, the file may look like this:
HelloWorld1,default,1.0,true,hrprojects\employees\HelloWorld1
HelloWorld2,default,1.0,true,hrprojects\employees\HelloWorld2
HelloWorld3,hello99,2.0,false,hrprojects\test\synchronize\HelloWorld3
This says that there are 3 composites to loop through; HelloWorld1, HelloWorld2, and HelloWorld3. If deployed, the first 2 will be deployed to the 'default' partition and a revision of '1.0' while the last one will be deployed to the 'hello99' partition with a revision of '2.0'. The last composite will not be set as the default composite, designated by the value of 'false'.
  
The directory path shown is relative to the %CODE_FOLDER% environment variable set earlier. For example, if %CODE_FOLDER% is set to c:\ant\code, then the three projects above should exist in the following local directories:
c:\ant\code\hrprojects\employees\HelloWorld1
c:\ant\code\hrprojects\employees\HelloWorld2
c:\ant\code\hrprojects\test\synchronize\HelloWorld3

Define the Custom MDS List

Before using any of the Ant targets that are related to the MDS, you must create a CustomMDSList.txt file.

1. Create the c:\ant\CustomMDSList.txt file, using the following format:
<app_name>,false

For example, the file may look like this:
App1MetaData,false
App2MetaData,false
If you run the MDS import script, it will import the following local directory to its relative MDS directory as follows:
c:\ant\apps\App1MetaData  -->  oramds:/apps/App1MetaData
c:\ant\apps\App2MetaData  -->  oramds:/apps/App2MetaData
For example, you may have a combination of XSDs, DVMs, and fault policies as shown: 
c:\ant\apps\App1MetaData\dvm\Currency.dvm
c:\ant\apps\App1MetaData\dvm\Country.dvm
c:\ant\apps\App1MetaData\config\fault-bindings.xml
c:\ant\apps\App1MetaData\config\fault-policies.xml
c:\ant\apps\App1MetaData\schemas\shared\Types.xsd
c:\ant\apps\App1MetaData\schemas\hr\Account.xsd
When you run the MDS import script, because the CustomMDSList.txt file is referencing the local c:\ant\apps\App1MetaData directory, all files under that local directory will be imported to the MDS.

Adding Additional Tokens

The detokenizeComposites target loops through all code in your CustomProcessList.txt file, and detokenizes your code. For example, if your code has the following string @SOAServer@, the command below will replace it with whatever value is specified in the soa-token-*.properties file. This allows you to not hardcode values in your code.

If you want to add additional token variables, say @SomeEndpoint@, you must do the following:

1. Add the new token to the soa-token-*.properties file. For example:
SomeEndpoint = http://dev.ipnweb.com:7777
2. Edit build.soa.xml and add the token in two separate targets (detokenizeComposites and tokenizeComposites):
<replacefilter token="@SomeEndpoint@" value="${SomeEndpoint}"/> 
<replacefilter token="${SomeEndpoint}" value="@SomeEndpoint@"/>

What is Dehydration store in soa 11g? What are the Tables for that?

BPEL dehydration store :
1.Oracle BPEL PM utilizes a database to store metadata and instance data during runtime.
  The process of updating process state in the database is called Dehydration.
  This data is stored in what is known as the Dehydration store, which is simply a database schema
  The Dehydration Store database is used to store process status data, especially for asynchronous      BPEL processes,   like BPEL’s metadata and instance data. This exists in x_SOAINFRA schema created by running RCU.
2. The Dehydration Store database is used to store process status data, especially for asynchronous BPEL processes.
3.A very important to remember if a BPEL process fails without reaching a
dehydration point then the instance will not show up on the BPEL console.
This instance never gets stored to the database.
Below are the main Dehydration tables for BPEL:
1.CUBE_INSTANCE
2.CUBE_SCOPE
3.AUDIT_TRAIL
4.AUDIT_DETAILS
5.DLV_MESSAGE
6.DLV_MESSAGE_BIN
7.INVOKE_MESSAGE
8.INVOKE_MESSAGE_BIN
9.DLV_SUBSCRIPTION
10.TASK
11.WORK_ITEM
Following are processes state codes and their meaning
State                          Code
Closed and Aborted 8
Closed and Cancelled 7
Closed and Completed 5
Closed and Faulted 6
Closed and (Pending or Cancel) 4
Closed and Stale 9
Initiated 0
Open and Faulted 3
Open and Running 1
Open and Suspended 2

Transient vs. durable BPEL processes

As a general practice, it is better to design your BPEL processes as transient instead of durable if performance is a concern. Note that this may not always be possible due to the nature of your process, but keep the following points in mind.

The dehydration store is uses to maintain long-running asynchronous BPEL instances storing state information as they wait for asynchronous callbacks. This ensures the reliability of these processes in the event of server or network loss.
Oracle BPEL Process Manager supports two types of processes; transient and durable.
Transient Processes
Transient processes do not incur dehydration during their process execution. If an executing process experiences an unhandled fault or the server crashes, instances of a transient process do not leave a trace in the system. Thus, these instances cannot be saved in-flight regardless if they complete normally or abnormally. Transient processes are typically short-lived, request-response style processes. Synchronous processes are examples of transient processes.
Durable Processes
Durable processes incur one or more dehydration points in the database during execution. Dehydration is triggered by one of the following activities:
  • Receive activity
  • OnMessage branch in a pick activity
  • OnAlarm branch in a pick activity
  • Wait activity
  • Reply activity
  • checkPoint() within a <bpelx:exec> activity
Durable processes are typically long-living and initiated through a one-way invocation. Because of out-of-memory and system downtime issues, durable processes cannot be memory-optimized.

What should you do?
  • If the design of the process allows it, design your BPEL processes as short-lived, synchronous transactions.
  • If the design of the process allows it, avoid the activities listed above.
Any time your process is dehydrated to the dehydration store, this naturally impacts the performance of the process, and becomes a concern particularly in high volume environments.

===========================================================


Transient and Durable Process

Most of us who are working in Oracle SOA has came across a common question many times,
what is transient and durable process in SOA.

As we know oracle soa uses dehydration store to store all the details of a process which has been executed, may it be a successful execution or may it be a failure each and every details are stored in the the soa dehydration store.
A process will be called a Transient process whose state  is not stored in soa dehydration store or in tern we can say it is an in-memory execution. Any process which are not stored in the dehydration store we can call it a Transient process.

A process will be called a durable process which has its state stored in the soa dehydration store. By default all the BPEL process are durable process and its instances are stored in the dehydration tables.

Oracle SOA gives us flexibility to make any process transient forcefully by setting few properties.
These properties are bpel.config.inMemoryOptimization and bpel.config.completionPersistPolicy.
By setting these properties in tandem we can forcefully make a durable process transient.


Though there is some catch in this. As we know oracle SOA BPEL engine executes a process depending upon its message exchange pattern, if the process is synchronous then dehydration happens at the end of the execution or else if the process is asynchronous then the dehydration happens wherever the process encounters a dehydration point/activity ( wait , receive , pick , onAlarm , dehydrate).
We can make a synchronous process transient by setting above properties in the composite.xml provided the synchronous process does not have any dehydration points within its flow or else these properties won't work.

below is a snap of how to set these properties in the composite xml
<component name="BPELProcess" version="1.1">
   <property name="bpel.config.inMemoryOptimization" type="xs:string" many="false">true</property>
   <property name="bpel.config.completionPersistPolicy" type="xs:string" many="false">off</property>
</component>



Once these properties are set as shown above we will not be able to see the audit trail of this process from the em as the process is a transient process.


=======================================================================




What is diff b/w Transient and Durable?


Transient process:
1.Synchronous process is the example of transient process- hence no dehydration activity.
2.No mid process breakpoint activities (Receive, onMessage, onAlarm, Wait)
3.No non-idempotent invoke And No non-blocking invoke
Durable process:
1.Long Running Process Has mid process breakpoint activities (Receive, onMessage, onAlarm, Wait)
2.Has non-idempotent invoke or Has non-blocking invoke


=========================================================================

Tuesday, 20 November 2012

Using Jenkins for SOA Deployment Automation

Last month, my client wanted me  to have a framework for automating the builds.Already the build scripts were in place. Because of the amount of changes/code fixes that different teams were checking in, the situation called for daily/frequent builds to be initiated.To make my customers job easy I was on the lookout for automating the deployments using a GUI automation tool. In my previous project we had used Cruise Control. After comparing different automation tools, I decided on Jenkins (Hudson) a more light weight easy to use tool and having strong support base.
To get started download Jenkins.war file from http://jenkins-ci.org/

There are 2 ways in which you can use Jenkins
  1. Run Jenkins in Winstone servlet container
The easiest way to execute Jenkins is through the built in Winstone servlet container. You can execute Jenkins like this:
Set JAVA_HOME
Then run java -jar jenkins.war
Accessing Jenkins
To see Jenkins, simply bring up a web browser and go to URL http://myServer:8080 where myServer is the name of the system running Jenkins.
  1. Deploy  Jenkins into Weblogic server
The Jenkins.war cannot be deployed to Weblogic server without some changes. These are necessary because of Weblogic's proprietary class loaders which behave differently compared to Tomcat, JBoss, et. al.

 
Once the web app is up and running.
In Dashboard --> Create New Job 


Add steps to Execute shell scripts and build files



Once the build is initiated the log entries and progress of build can be monitored in the Console output.

If you ask me which automation tool to pick for your project my answer will be Jenkins.

How to remove unused files/artifacts from MDS?

In development phase of project, we will be adding different artifacts to the MDS so that can be accessed by different processes. In course of development /design changes, applying best practices, there is bound to be lot of unused folders /artifacts lying around in the MDS store. So it is always advised to remove the unwanted artifacts, test the processes before we export them to other environments. I always used a workaround to remove the files, even though it was not the best approach.

Workaround
 There is a table called MDS_PATHS where there would be entries corresponding to each artifact. I used to delete the entries from the table and the artifacts never used to show up, even though there will be entries in other related tables.

BestApproach
The best approach/solution is to use WLST command for cleaning up the MDS store. There is a function called deleteMetadata in WLST which will do the job for you.
This is how Oracle documentation describes the command. It deletes the selected documents from the application repository. When this command is run against repositories that support versioning, that is a database-based repository, delete is logical and marks the tip version (the latest version) of the selected documents as "deleted" in the MDS repository partition.

For more attributes and options available with the command refer this doc

The WLST script is located at:

(UNIX) MIDDLEWARE_HOME/ORACLE_SOA1/common/bin/wlst.sh
(Windows) MIDDLEWARE_HOME\Oracle_SOA1\common\bin\wlst.cmd

Once the scripting tool is initialized, Connect to the server

offline>connect(‘username,’pwd’, ‘hostname:7001’)

For running deleteMetada function you need a minimum of 3 inputs
-          application  - since we are deleting from shared artifacts of soa-infra, the value should be soa-infra
-          server -  value should be ‘soa_server1’ or the server u use for SOA other than admin server.
-          docs – the folder path or artifact which you want to delete.

wls:/GEO_domain/serverConfig> deleteMetadata(application='soa-infra',server='soa
_server1',docs='/apps/dvm/oracle/dvm/*')

Executing operation: deleteMetadata.

Operation "deleteMetadata" completed. Summary of "deleteMetadata" operation is:

List of documents successfully deleted:
/apps/dvm/oracle/dvm/GeoXRef.dvm

1 documents successfully deleted.

Undeploy multiple SOA composites with WLST or ANT

As part of our current project the Build Management team asked for a solution to undeploy multiple composites at one time. Of course you have the “Undeploy All From This Partition” menu option in Enterprise Manager but since we have a lot of deployments every day the guys wanted to have a script solution. It is even more important for the nightly deployments on our continuous integration environment – strange, we couldn’t find anybody who wants to do the undeployment via Enterprise Manager manually every night ;-)

However with WLST or ANT the SOA Suite comes with two options to undeploy composites via script. In this article I’d like to explain you both ways.


Undeployment with WLST

You can test the steps below on Oracle's Pre-built Virtual Machine for SOA Suite and BPM Suite 11g.

1) Change to the WLST directory under MIDDLEWARE_HOME/Oracle_SOA1/common/bin.

cd /oracle/fmwhome/Oracle_SOA1/common/bin/

2) Open WLST

./wlst.sh

3)  Connect to the SOA server

wls:/offline> connect('weblogic','welcome1','t3://soabpm-vm:7001')
Connecting to t3://soabpm-vm:7001 with userid weblogic ...
Successfully connected to Admin Server 'AdminServer' that belongs to domain 'dev_bpm'.
wls:/dev_bpm/serverConfig>

4) Run the delete command for the appropriate partition

wls:/dev_bpm/serverConfig> sca_deletePartition('test')
partitionName = test
Partition was successfully deleted.
wls:/dev_bpm/serverConfig>

Please take into account that the command deletes all composites as well as the partition itself. If you need the partition for future deployments just recreate it with the sca_createPartition WLST-command. Also check the Oracle Fusion Middleware WebLogic Scripting Tool Command Reference for a complete list of SOA Suite Custom WLST Commands.

Undeployment with ANT

Another option is to use ANT for the undeployment of multiple composites. The key here is to reference the file ant-sca-mgmt.xml within your custom ANT-target. The file comes with SOA Suite as well as JDeveloper. See my quick example below:

<target name="sca_deletePartition">
  <echo>Undeploy Composites</echo>      
  <ant antfile="/oracle/fmwhome/Oracle_SOA1/bin/ant-sca-mgmt.xml"
     inheritall="false" target="deletePartition">         
      <property name="host" value="soabpm-vm"/>         
      <property name="port" value="7001"/>         
      <property name="user" value="weblogic"/>         
      <property name="password" value="welcome1"/>         
      <property name="partition" value="testPartition"/>      
  </ant>   
</target>

Again this command deletes all composites of the given partition as well as the partition itself. See the Developer's Guide - Managing SOA Composite Applications with Script for more details.

xslt padding with characters call template for left pad and right pad

  Could a call-template be written that took two parameters ?   a string, and a   number) return the string with empty spaces appended t...