WPS - Event sequencing for MQ activation spec in WPS7

From WPS7, IBM retired old listener port concept and replaced it with new MQ Activation Spec.

However, if the MQ activationSpec is not set up correctly in Cluster environment, the Event sequencing may not work.

In cluster environment, you have to create MQ Activation spec on each server level in the cluster, and assure only one MQ Activation Spec running.

Before going through the following steps, an ifix IZ86196 must be applied to make event sequencing work. please ask IBM for this ifix matched to your WAS version.

here is the examples how to set up MQ Activation Spec for event sequencing

(1) assume you have one cluster with two application servers (server1 and server2)

(2) go to administrative console, and then go to Resources->JMS->Activation Specifications

(3) create the MQ Activation Spec on eache server scope.

(4) after creating the MQ Activation Spec, go to created Activation Spec Custom properties, add the following property:

WAS_EndpointInitalState

and set it as true on server1 scope and set it as false on server2 scope.

(5) change another property maxPoolSize as 1 on both servers

(6) also go to Activation spec's advanced properties, and set the maximum server sessions as 1 on both servers

(7) restart the server.

after restarting the server, the activation spec will be only running on one server, which assure the event squencing working

following jython script will let you change the stop or start MQ activation spec on another server

import sys

if len(sys.argv)<2:
print "Usage:changeASStatus.jy ASName ServerName action"
sys.exit()

if len(sys.argv)>1:
ASName=sys.argv[0]
serverName=sys.argv[1]

if len(sys.argv)>2:
Action=sys.argv[2]
else:
Action=""

print "--------------------------------------------------"
id=AdminControl.queryNames('type=J2CMessageEndpoint,ActivationSpec='+ASName+','+'process='+serverName+',*')

status=AdminControl.invoke(id,'getStatus')

print ASName+"'s current status:"+status
print "note: 1 (Active), 2 (Inactive), 3 (Stopped)"
print "-------------------------------------------------------"

if Action=="start":

print "starting the ActivationSpec "+ASName
try:
AdminControl.invoke(id,'resume')
print "ActivationSpec "+ASName+" started on server:"+serverName
except:
print "ActivationSpec "+ASNAme+" could not be started on server:"+serverName,sys.exc_info()[1]

if Action=="stop":

print "stopping the ActivationSpec "+ASName
try:
AdminControl.invoke(id,'pause')
print "ActivationSpec "+ASName+" stopped on server:"+serverName
except:
print "ActivationSpec "+ASNAme+" could not be stopped on server:"+serverName,sys.exc_info()[1]

if Action=="":
print ""

 

WPS - SCA trace causes the SCA module NullPointerException

Noticed that transaction rollback in the SystemOut.log, like:

[11/29/10 21:42:53:499 EST] 0000006a Recovery      I com.ibm.wbiserver.
management.FailedEventManagerMBean resubmitFailedEvents(List)
CWRCV0033I: The following number of failed events was resubmitted
successfully: "1".
[11/29/10 21:42:54:426 EST] 00000036 MA_IBISSTARSD I   begin
createOutb_Bpi_Msg_Stars: msg_id=1400010275, orderid=2010-11-29 16:48:
56.455674
[11/29/10 21:42:55:699 EST] 00000036 ESWorkBean    E com.ibm.wbiserver.
sequencing.work.ESWorkBean processMessageInBifurcatedSession
CWRES0016E: The message with msgId=8FD53ADC8A049ECB_6000087 and the
transaction will be rolled back. The exception is Fail to invoke [sca.
component.java.impl.Component1Impl3.public void sca.component.java.impl.
Component1Impl3.createOutb_Bpi_Msg_Stars(commonj.sdo.DataObject)] for
component [{MA_IBISSTARSDb}StarsBGtoBO].
[11/29/10 21:42:56:331 EST] 00000036 LocalExceptio E   CNTR0020E: EJB
threw an unexpected (non-declared) exception during invocation of
method "processMessageInBifurcatedSession" on bean "BeanId
(persistentLkMgr#persistentLkMgrEJB.jar#ESWork, null)". Exception data:
com.ibm.wbiserver.sequencing.exceptions.SequencingRuntimeException:
CWRES0015E: The message with msgId=8FD53ADC8A049ECB_6000087 failed
during processing. The error is Fail to invoke [sca.component.java.impl.
Component1Impl3.public void sca.component.java.impl.Component1Impl3.
createOutb_Bpi_Msg_Stars(commonj.sdo.DataObject)] for component
[{MA_IBISSTARSDb}StarsBGtoBO].
        at com.ibm.wbiserver.sequencing.work.ESWorkBean.processMessage
(ESWorkBean.java:177)
        at com.ibm.wbiserver.sequencing.work.ESWorkBean.
processMessageInBifurcatedSession(ESWorkBean.java:101)
        at com.ibm.wbiserver.sequencing.work.
EJSLocalStatelessESWork_ddc8f6c2.processMessageInBifurcatedSession
(Unknown Source)
        at com.ibm.wbiserver.sequencing.work.EsWorkImpl.run(EsWorkImpl.
java:235)
        at com.ibm.ws.asynchbeans.J2EEContext$RunProxy.run(J2EEContext.
java:268)
        at java.security.AccessController.doPrivileged(AccessController.
java:224)
        at javax.security.auth.Subject.doAs(Subject.java:495)
        at com.ibm.websphere.security.auth.WSSubject.doAs(WSSubject.
java:131)
        at com.ibm.websphere.security.auth.WSSubject.doAs(WSSubject.
java:89)
        at com.ibm.ws.asynchbeans.J2EEContext$DoAsProxy.run(J2EEContext.
java:335)
        at java.security.AccessController.doPrivileged(AccessController.
java:251)
        at com.ibm.ws.asynchbeans.J2EEContext.run(J2EEContext.java:777)
        at com.ibm.ws.asynchbeans.WorkWithExecutionContextImpl.go
(WorkWithExecutionContextImpl.java:222)
        at com.ibm.ws.asynchbeans.ABWorkItemImpl.run(ABWorkItemImpl.
java:159)
        at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1550)
Caused by: com.ibm.websphere.sca.ServiceRuntimeException: Fail to
invoke [sca.component.java.impl.Component1Impl3.public void sca.
component.java.impl.Component1Impl3.createOutb_Bpi_Msg_Stars(commonj.
sdo.DataObject)] for component [{MA_IBISSTARSDb}StarsBGtoBO]: caused
by: java.lang.NullPointerException
        at com.ibm.ws.sca.internal.java.handler.
JavaImplementationHandler.invokeSync(JavaImplementationHandler.java:473)
        at com.ibm.ws.sca.internal.java.handler.
JavaImplementationHandler.processMessage(JavaImplementationHandler.java:
306) 

The transactions kept failling. 

The root cause is the trace "SCA.*=all" caused this problem. After disabling the trace "SCA.*=all",
the transactions were going well. 

IBM provided an ifix to solve this issue JR37651 to solve this issue.

WebSphere - Exported EAR (Enhance EAR) could not started after deployed on WPS7

On WPS7, an exported EAR file from one WebSphere environment could mot started after deployed on another WebSphere environment. When you start the deployed module, you may see the following error in the log:

[12/21/10 16:56:31:809 EST] 0000002c FfdcProvider  W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted
on /IBM2/wpsProfiles/tx1Node01/logs/ffdc/uat01Cell.AppTarget.tx1Node01.0_6a306a30_10.12.21_16.56.31.6896290038486030221913.txt com.i
bm.ws.metadata.bindings.EJBModuleHelper.processBindingsFile 199
[12/21/10 16:56:31:818 EST] 0000002c EJBModuleHelp E   CNTR0146E: The META-INF/ibm-ejb-jar-bnd.xml bindings file contained in the MA
_RSDChangeTicketEJB.jar module has a configuration error at line number: 5, column number: 131.
[12/21/10 16:56:34:348 EST] 0000002c FfdcProvider  W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted
on /IBM2/wpsProfiles/tx1Node01/logs/ffdc/uat01Cell.AppTarget.tx1Node01.0_6a306a30_10.12.21_16.56.31.8253759099213016881532.txt com.i
bm.ws.runtime.component.EJBContainerImpl.createMetaData 3439
[12/21/10 16:56:34:356 EST] 0000002c EJBContainerI E   WSVR0039E: Unable to start EJB jar, MA_RSDChangeTicketEJB.jar: CNTR0146E: The
META-INF/ibm-ejb-jar-bnd.xml bindings file contained in the MA_RSDChangeTicketEJB.jar module has a configuration error at line numb
er: 5, column number: 131.
[12/21/10 16:56:34:359 EST] 0000002c DeployedAppli W   WSVR0206E: Module, MA_RSDChangeTicketEJB.jar, of application, MA_RSDChangeTic
ketApp.ear/deployments/MA_RSDChangeTicketApp, failed to start

Solution:

IBM provided a workaround for this issue:

1) Open Export EAR using any compression software (Ex. 7-Zip)
2) Navigate to "META-INF" folder. (Ex. MA_RSDChangeTicketApp.ear\META-INF\)
3) Open "MANIFEST.MF"
4) Add following line in "MANIFEST.MF"
GeneratedVersion: 7.0.0.0
5) Save "MANIFEST.MF"

this workaround works.

IBM are working on the ifix to solve this issue. will let you know when I get it.

 

WebSphere-Get Runtime JVM heapsize

The following script can be used to get the current JVM HeapSize:

serverJVM=AdminControl.completeObjectName(’type=JVM,process=server1,*’)

serverJVMObj=AdminControl.makeObjectName(serverJVM)

perf=AdminControl.completeObjectName(’type=Perf,process=server1,*’)

perfObj=AdminControl.makeObjectName(perf)

jvmObj=AdminControl.invoke_jmx(perfObj,’getStatsObject’,[serverJVMObj,java.lang.Boolean('false')],['javax.management.ObjectName','java.lang.Boolean'])

currentHeapsize=jvmObj.getStatistic(’HeapSize’).getCurrent()
usedMemory=jvmObj.getStatistic(’UsedMemory’).getCount()
usage=float(usedMemory)/float(currentHeapsize)*100

print serverName+” –> “+str(currentHeapsize)+”K –> “+str(usedMemory)+”K –> “+”Usage:%.2f” % usage+”%”

This script will print current heap size and used memory of JVM.

WebSphere-Modify WAS Datasource Attributes Dynamically

Some WAS Datasource attributes can be modified dynamically without restarting the server.
Using the following command to change the attributes

(1) get the datasource id

set ds [$AdminControl queryNames type=DataSource,name=Datasource1, process=server1,*]

(2) change the datasource minimum connections from default value 1 to 0

$AdminControl setAttribute $ds minConnections 0

(3) get the minimum connections

$AdminControl getAttribute $ds minConnections

(4) list all datasource attributes and its operations

$Help attributes $ds
$Help operations $ds

(5) show the data source connection pool content

$AdminControl invoke $ds showPoolContents

WPS – Using esAdmin.sh to unlock the locked messages in the event sequencing queue

Sometimes, WPS event sequencing works not very well, and some messages are locked there and blocked the other messages to continue to process, using the esAdmin.sh to unlock or delete locks

Usage: esAdmin [-h hostname] [-p soapPortNumber] method parameters where hostname=’localhost’ and soapPortNumber=8880 by default.
Available methods are :
listAll
listLocks moduleName
listLocks moduleName componentName
listLocks moduleName componentName methodName
deleteLocks moduleName
unlock lockId

listAll is used to list all locked messages
unlock lockId will be used to unlock one locked messages. lockId can be acquired using listAll command.
deleteLocks is used to delete all lockes related to one module. remember that the application need to be stopped before using deleteLocks. the module name is SCA module name, not application name.

for example: esAdmin.sh deleteLocks moduleName

WPS-MQBinding Error CWSCA4100E: An MQMD property could not be accessed.

When you using MQ binding in WPS7, you may see the following error:

00000463 MQ            E   CWSCA4100E: An MQMD property could not be accessed. JNDI destinations used by t
he WebSphere MQ binding must have custom properties MDWRITE and MDREAD set to “YES” and custom property MSGBODY set to “MQ”.

00000463 SibMessage    W   [:] CWSJY0003W: JMSCC0109: A message driven bean threw a runtime exception ‘
Message : com.ibm.websphere.sca.ServiceRuntimeException: JNDI destinations used by the WebSphere MQ binding m
ust have custom properties MDWRITE and MDREAD set to ‘YES’ and custom property MSGBODY set to ‘MQ’: caused by: java.lang.NumberForma
tException: null
Class : class com.ibm.websphere.sca.ServiceRuntimeException

solution:

you have to set the following properties for MQ Queue defined in MQ resources

go to Resources->JMS->Queues, select the MQ queue you defined. then click “Custom Properties” in “Additional Properties” section

add the following properties for Sender MQ queue



add the following properties for Receiver MQ queue







then save the changes, and restart the server. The error should be eliminated.