Message Server NO running on sap ERP 7.4 HA

[jbarr1]

Buenas tardes,

Se me presenta el siguiente problema:

Tengo una instalación sap netweaver ERP 7.4
kernel 742 con sybase 16 en alta disponibilidad (Cluster) en servidores virtuales HP.

Ahora bien cuando el cluster esta en modo mantenimiento y levanto SAP de manera manual la aplicación funciona normalmente y correctamente pero cuando el cluster esta en modo activo y el mismo cluster debe levantar de manera automática la aplicación sap esta levanta pero NO levanta el message server que esta en la instancia ASCS01, leyendo notas encontré que debía eliminar el parámetro rdisp/wp_no_enq = 1 porque este parámetro no funciona para instalaciones en alta disponibilidad o cluster.

Podria alguien ayudarme con la solucion de este caso.

Según lo que he leído todo apunta a que es un problema de parámetro, por lo que les pregunto, ¿existe alguna información de los parámetros que funcionan cuando la aplicación esta instalada en alta disponibilidad.?

o en su defecto ¿como hacer para que el message server levante cuando el cluster sube automáticamente la aplicación.?


Nota: Para no interrumpir el servicio a los usuarios tengo el cluster en modo mantenimiento.

De antemano gracias por toda la colaboración prestada

Jose Barrera

[jamez]

Buenas,

No tengo experiencia en cluster con Sybase pero debería haber un fichero dev_ms con la traza de lo que ocurre en el Message Server.

Quizás ahí puedas ver que le sucede. Si no, súbelo y vemos si hay algo.

Saludos,

[jbarr1]

aca el log de sapstartsrv.log

---------------------------------------------------
trc file: "sapstartsrv.log", trc level: 0, release: "742"
---------------------------------------------------
pid 18353

[Thr 01] Tue Dec 22 08:48:56 2015
HistoryLog_Init: logfile (/usr/sap/PRD/ASCS01/work/history.glf) already exists, check parameter
No halib defined => HA support disabled
CCMS agent initialization for instance type SCS: return code 0.
CCMS agent start: return code 0.
Initializing SAPControl Webservice
Root PSE verification succeeded
[Thr 05] Tue Dec 22 08:48:57 2015
Auto PSE update thread started
Starting AutoRestart thread
AutoRestart thread started
Starting WebService SSL thread
Starting WebService thread
Webservice SSL thread started, listening on port 50114
Webservice SSL thread using system PKI credential
Webservice thread started, listening on port 50113
Trusted http connect via Unix domain socket '/tmp/.sapstream50113' enabled.
Trusted https connect via Unix domain socket '/tmp/.sapstream50114' enabled.

trusted unix domain socket user is starting SAP System at 2015/12/22 8:49:06
trusted unix domain socket user is stopping SAP System at 2015/12/22 8:57:16



log de sapstar.log

SAP-R/3-Startup Program Rel 742 V2.0 (2008/11/15)
-------------------------------------------------

Starting at 2015/12/22 08:49:07
Startup Profile: "/usr/sap/PRD/SYS/profile/PRD_ASCS01_clterpfa"

Setup Environment Variables
---------------------------
(18591) SETENV DIR_LIBRARY=/usr/sap/PRD/ASCS01/exe
(18591) SETENV LD_LIBRARY_PATH=/usr/sap/PRD/ASCS01/exe:/usr/sap/PRD/ASCS01/exe:/sybase/PRD/ASE-16_0/lib:/sybase/PRD/OCS-16_0/lib:/sybase/PRD/OCS-16_0/lib3p64:
(18591) SETENV SHLIB_PATH=/usr/sap/PRD/ASCS01/exe:/usr/sap/PRD/SYS/global/syb/hpia64/sybodbc:/sybase/PRD/ASE-16_0/lib:/sybase/PRD/OCS-16_0/lib:/sybase/PRD/OCS-16_0/lib3p:/usr/sap/PRD/SYS/exe/run:/usr/sap/PRD/SYS/exe/uc/hpia64
(18591) SETENV LIBPATH=/usr/sap/PRD/ASCS01/exe:
(18591) SETENV PATH=/usr/sap/PRD/ASCS01/exe:/sybase/PRD/ASE-16_0/jobscheduler/bin:/sybase/PRD/ASE-16_0/bin:/sybase/PRD/ASE-16_0/install:/sybase/PRD/OCS-16_0/bin:/usr/bin:/usr/ccs/bin:/usr/contrib/bin:/usr/contrib/Q4/bin:/opt/perl/bin:/opt/gvsd/bin:/opt/ipf/bin:/opt/nettladm/bin:/opt/fcms/bin:/opt/wbem/bin:/opt/wbem/sbin:/opt/sas/bin:/opt/wli/bin:/opt/graphics/common/bin:/opt/hpvm/bin:/opt/atok/bin:/usr/bin/X11:/usr/contrib/bin/X11:/opt/sec_mgmt/bastille/bin:/opt/caliper/bin:/opt/drd/bin:/opt/dsau/bin:/opt/dsau/sbin:/opt/resmon/bin:/opt/firefox:/opt/perf/bin:/opt/ignite/bin:/usr/contrib/kwdb/bin:/opt/perl_32/bin:/opt/perl_64/bin:/opt/prm/bin:/opt/propplus/bin:/opt/sfm/bin:/etc/cmcluster/scripts/tkit/vtn:/opt/swm/bin:/opt/sec_mgmt/spc/bin:/opt/ssh/bin:/opt/swa/bin:/opt/hpsmh/bin:/opt/thunderbird:/opt/sentinel/bin:/opt/langtools/bin:/opt/wlm/bin:/opt/gwlm/bin:/opt/omni/bin:/usr/sap/PRD/SYS/exe/uc/hpia64:/usr/sap/PRD/SYS/exe/run:/home/prdadm:.
(18591) SETENV SECUDIR=/usr/sap/PRD/ASCS01/sec

Execute Pre-Startup Commands
----------------------------
(18592) Local: /usr/sap/PRD/SYS/exe/uc/hpia64/sapcpe pf=/usr/sap/PRD/SYS/profile/PRD_ASCS01_clterpfa list:/usr/sap/PRD/SYS/exe/uc/hpia64/scs.lst
(18591) system(/usr/sap/PRD/SYS/exe/uc/hpia64/sapcpe pf=/usr/sap/PRD/SYS/profile/PRD_ASCS01_clterpfa list:/usr/sap/PRD/SYS/exe/uc/hpia64/scs.lst) returns 4
(18596) Local: /usr/sap/PRD/SYS/exe/uc/hpia64/sapcpe pf=/usr/sap/PRD/SYS/profile/PRD_ASCS01_clterpfa list:/usr/sap/PRD/SYS/exe/uc/hpia64/sapcrypto.lst
(18600) Local: rm -f ms.sapPRD_ASCS01
(18602) Local: ln -s -f /usr/sap/PRD/ASCS01/exe/msg_server ms.sapPRD_ASCS01
(18604) Local: rm -f en.sapPRD_ASCS01
(18606) Local: ln -s -f /usr/sap/PRD/ASCS01/exe/enserver en.sapPRD_ASCS01

Starting Programs
-----------------
(18610) Starting: local ms.sapPRD_ASCS01 pf=/usr/sap/PRD/SYS/profile/PRD_ASCS01_clterpfa
(18610) New Child Process created.
(18610) Starting local Command:
Command: ms.sapPRD_ASCS01
pf=/usr/sap/PRD/SYS/profile/PRD_ASCS01_clterpfa
(18611) Starting: local en.sapPRD_ASCS01 pf=/usr/sap/PRD/SYS/profile/PRD_ASCS01_clterpfa
(18611) New Child Process created.
(18611) Starting local Command:
Command: en.sapPRD_ASCS01
pf=/usr/sap/PRD/SYS/profile/PRD_ASCS01_clterpfa
(18591) Waiting for Child Processes to terminate.

(18591) **** 2015/12/22 08:57:16 Caught Signal to Stop all Programs. ****

(18591) Parent Shutdown at 2015/12/22 08:57:18
(18591) Stop Child Process: 18611
(18591) Wait 300 sec for Process 18611 terminating...
(18591) Stop Child Process: 18610
(18591) Wait 300 sec for Process 18610 terminating...

Execute Post-Shutdown Commands
------------------------------
(18591) Exiting with Return-Code 0. (Ok)




log de dev_ms


---------------------------------------------------
trc file: "dev_ms.new", trc level: 1, release: "742"
---------------------------------------------------

[Thr 01] Tue Dec 22 08:49:34 2015
[Thr 01] SAP_RELEASE = 742
[Thr 01] compatibility level = 0
[Thr 01] Server state INIT
[Thr 01] MsSSetTrcLog: trc logging active, max size = 52428800 bytes
[Thr 01] * SWITCH TRC-RESOLUTION from 1 to 1
systemid 274 (HP (IA-64) with HP-UX)
relno 7420
patchlevel 0
patchno 114
intno 20020600
make multithreaded, Unicode, 64 bit, optimized
pid 14652


[Thr 01] Tue Dec 22 08:49:34 2015
[Thr 01] ***LOG Q01=> MsSInit, MSStart (Msg Server 1 14652) [msxxserv.c 2484]
[Thr 01] SigISetDefaultAction : default handling for signal SIGCHLD
[Thr 01] load acl file = /usr/sap/PRD/SYS/global/ms_acl_info
[Thr 01] MsGetOwnIpAddr: my host addresses are :
[Thr 01] 1 : [10.x.xx.8] saperpfa02 (HOSTNAME)
[Thr 01] 2 : [127.0.0.1] localhost (LOCALHOST)
[Thr 01] 3 : [10.1.xx.53] clterpfa (NILIST)
[Thr 01] 4 : [10.1.xx.8] 10.1.33.8 (NILIST)
[Thr 01] 5 : [192.168.xx.xx] 192.168.xx.xx (NILIST)
[Thr 01] 6 : [10.10.0.7] 10.10.0.7 (NILIST)
[Thr 01] MsHttpInit: full qualified hostname = saperpfa02
[Thr 01] HTTP logging is switch off
[Thr 01] set HTTP state to LISTEN
[Thr 01] *** HTTP port 8101 state LISTEN ***
[Thr 01] *** I listen to port sapmsPRD (3601) ***
[Thr 01] *** I listen to internal port 3901 (3901) ***
[Thr 01] *** HTTP port 8101 state LISTEN ***
[Thr 01] CUSTOMER KEY: >P1721684070<
[Thr 01] compatibility level = 0
[Thr 01] build version=742.2015.04.20
[Thr 01] read logon groups from previously stored file PRD_msg_server_adtl_storage
[Thr 01] Server state ACTIVE

[Thr 01] Tue Dec 22 08:50:01 2015
[Thr 01] LOGIN: clterpfa_PRD_00 [127.0.0.1] (DIA UPD BTC SPO UP2 ICM )

[Thr 01] Tue Dec 22 12:25:36 2015
[Thr 01] LOGOUT: clterpfa_PRD_00 [10.1.31.53] (DIA UPD BTC SPO UP2 ICM )



agradezco cualquier ayuda


saludos

[jamez]

Te explico lo que veo con lo que has enviado:

Según el log de sapstar.log

Arranca message server y enqueue server
Starting at 2015/12/22 08:49:07
...
(18610) Starting: local ms.sapPRD_ASCS01 pf=/usr/sap/PRD/SYS/profile/PRD_ASCS01_clterpfa
(18611) Starting local Command:
Command: en.sapPRD_ASCS01
....
Manda señal de parada a las 08:57
(18591) **** 2015/12/22 08:57:16 Caught Signal to Stop all Programs. ****
Para el ms y el en
(18591) Stop Child Process: 18611
(18591) Stop Child Process: 18610

Y según el log de dev_ms, éste llega al estado ACTIVE
[Thr 01] Server state ACTIVE

Por lo que parece que el error no está en el ms sino más adelante, quizás algun problema de memoria con algún dw (disp+work).

Te recomiendo que revises los logs de los dw (ordenalos por tamaño y empieza por el que más ocupa), y además realices un chequeo de parámetros y memoria usando el sappfpar, mira la web siguiente



Lo más destacado del resultado estará al final
Errors detected..................: ?
Warnings detected................: ?

Lo que sea, nos pones y veremos que pistas vemos.

Saludos,

[jbarr1]

muestro el resultado de la ejecucion del commando sappfpar:

saperpfa01rdadm 38> sappfpar check pf=/sapmnt/PRD/profile/PRD_DVEBMGS00_clterpfa


================================================================================
== Checking profile: /sapmnt/PRD/profile/PRD_DVEBMGS00_clterpfa
================================================================================



***WARNING: Unexpected parameter: ipc/shm_psize_01 =-10
***WARNING: Unexpected parameter: ipc/shm_psize_02 =-10
***WARNING: Unexpected parameter: ipc/shm_psize_03 =-10
***WARNING: Unexpected parameter: ipc/shm_psize_04 =-10
***WARNING: Unexpected parameter: ipc/shm_psize_07 =-10
***WARNING: Unexpected parameter: ipc/shm_psize_12 =-10
***WARNING: Unexpected parameter: ipc/shm_psize_14 =-10
***WARNING: Unexpected parameter: ipc/shm_psize_16 =-10
***WARNING: Unexpected parameter: ipc/shm_psize_18 =-10
***WARNING: Unexpected parameter: ipc/shm_psize_21 =-10
***WARNING: Unexpected parameter: ipc/shm_psize_30 =-10
***WARNING: Unexpected parameter: ipc/shm_psize_41 =-40
***WARNING: Unexpected parameter: ipc/shm_psize_42 =-40
***WARNING: Unexpected parameter: ipc/shm_psize_43 =-40
***WARNING: Unexpected parameter: ipc/shm_psize_44 =-40
***WARNING: Unexpected parameter: ipc/shm_psize_45 =-40
***WARNING: Unexpected parameter: ipc/shm_psize_46 =-40
***WARNING: Unexpected parameter: ipc/shm_psize_47 =-40
***WARNING: Unexpected parameter: ipc/shm_psize_48 =-40
***WARNING: Unexpected parameter: ipc/shm_psize_49 =-40
***WARNING: Unexpected parameter: ipc/shm_psize_51 =-40
***WARNING: Unexpected parameter: ipc/shm_psize_52 =-40
***WARNING: Unexpected parameter: ipc/shm_psize_56 =-40
***WARNING: Unexpected parameter: ipc/shm_psize_57 =-40
***WARNING: Unexpected parameter: ipc/shm_psize_58 =-40
***WARNING: Unexpected parameter: rdisp/appc_ca_blk_no =0

Maximum shared memory size > 20G
Shared memory disposition overview
================================================================
Shared memory pools
Key: 10 Pool
Size configured.....: 2147483647 (2048.0 MB)
Size min. estimated.: 778804098 ( 742.7 MB)
Advised Size........: 782000000 ( 745.8 MB)

Key: 40 Pool for database buffers
Size configured.....: 2147483647 (2048.0 MB)
Size min. estimated.: 1186572534 (1131.6 MB)
Advised Size........: 1190000000 (1134.9 MB)

Shared memories inside of pool 10
Key: 1 Size: 2048 ( 0.0 MB) System administration
Key: 2 Size: 68844304 ( 65.7 MB) Disp. administration tables
Key: 3 Size: 208896000 ( 199.2 MB) Disp. communication areas
Key: 4 Size: 582400 ( 0.6 MB) statistic area
Key: 7 Size: 33376 ( 0.0 MB) Update task administration
Key: 11 Size: 500000 ( 0.5 MB) Factory calender buffer
Key: 12 Size: 6000000 ( 5.7 MB) TemSe Char-Code convert Buf.
Key: 13 Size: 60500000 ( 57.7 MB) Alert Area
Key: 14 Size: 4400128 ( 4.2 MB) Presentation buffer
Key: 16 Size: 32384 ( 0.0 MB) Semaphore activity monitoring
Key: 18 Size: 4148580 ( 4.0 MB) Paging adminitration
Key: 20 Size: 366666854 ( 349.7 MB) Unused1
Key: 21 Size: 8141232 ( 7.8 MB) Unused2
Key: 30 Size: 55000 ( 0.1 MB) Taskhandler runtime admin.

Shared memories inside of pool 40
Key: 41 Size: 25010000 ( 23.9 MB) DB statistics buffer
Key: 42 Size: 229344992 ( 218.7 MB) DB TTAB buffer
Key: 43 Size: 569342392 ( 543.0 MB) DB FTAB buffer
Key: 44 Size: 159742392 ( 152.3 MB) DB IREC buffer
Key: 45 Size: 108542392 ( 103.5 MB) DB short nametab buffer
Key: 46 Size: 20480 ( 0.0 MB) DB sync table
Key: 47 Size: 3175424 ( 3.0 MB) DB CUA buffer
Key: 48 Size: 712000 ( 0.7 MB) Number range buffer
Key: 49 Size: 2968344 ( 2.8 MB) Spool admin (SpoolWP+DiaWP)
Key: 51 Size: 35000000 ( 33.4 MB) Extended memory admin.
Key: 52 Size: 180000 ( 0.2 MB) Message Server buffer
Key: 56 Size: 535936 ( 0.5 MB) Application statistics
Key: 57 Size: 1992294 ( 1.9 MB) Profilparameter in shared mem
Key: 58 Size: 4096 ( 0.0 MB) Enqueue ID for reset

Shared memories outside of pools
Key: 6 Size: 8444979200 (8053.8 MB) ABAP program buffer
Key: 8 Size: 1213727220 (1157.5 MB) Paging buffer
Key: 34 Size: 67108864 ( 64.0 MB) Enqueue table
Key: 54 Size: 773039104 ( 737.2 MB) Export/Import buffer
Key: 62 Size: 421527552 ( 402.0 MB) Memory pipes
Key: 63 Size: 409600 ( 0.4 MB) ICMAN shared memory
Key: 64 Size: 4227072 ( 4.0 MB) Online Text Repository Buf.
Key: 65 Size: 4227072 ( 4.0 MB) Export/Import Shared Memory
Key: 81 Size: 21760 ( 0.0 MB) Security Audit Log
Key: 1002 Size: 400000 ( 0.4 MB) Performance monitoring V01.0
Key: 58900100 Size: 4096 ( 0.0 MB) SCSA area

Nr of operating system shared memory segments: 13

Shared memory resource requirements estimated
================================================================
Total Nr of shared segments required.....: 13
System-imposed number of shared memories.: 512
Shared memory segment size required min..: 8444979200 (8053.8 MB)
Currently available maximum segment size.: 34359738368 (32768.0 MB)

Swap space requirements estimated
================================================
Shared memory....................: 12304.0 MB
..in pool 10 742.7 MB, 36% used
..in pool 40 1131.6 MB, 55% used
..not in pool: 10423.3 MB
Processes........................: 188.1 MB
Extended Memory .................: 51605.0 MB
------------------------------------------------
Total, minimum requirement.......: 64097.0 MB
Process local heaps, worst case..: 7629.4 MB
Total, worst case requirement....: 71726.4 MB

Errors detected..................: 0
Warnings detected................: 26

[jamez]

Pues no le veo un problema de parámetros ya que no salen Errores solo warnings.

Has podido mirar los logs de los dw's de mayor tamaño a menor en busca de errores en ellos ??
Esto has de mirarlo cuando te falle el arranque del HA, si lo miras ahora estando en mantenimiento los dw's funcionarán normalmente y tendrán unos logs bastante grandes.
O alomejor puedes mirar los dw*.old, si éstos, encajan con un arranque con HA y falló...

Siento no poder ayudarte mucho...

[jbarr1]

Para solventar este caso se eliminó el parámetro rdisp/wp_no_enq = 1 del profile.

Este parametro no debe estar configurado en una instancia siempre que la instalacion se este ejecutando sobre un cluster.

[jamez]

Perfecto !! Gracias por compartir.

Saludos,