Solution : https://service.sap.com/sap/support/notes/321238 (SAP Service marketplace login required)
Key words : 
maximum shared memory segment size = 960 mb  maximum number, os/390 unix system services planning book, configuring os/390 unix system services, 1752 mb  maximum mapped file size, shared memory segments = 1000  maximum amount, maximum mapped file size, shared memory space = 960 mb, bpxp012i fork syscall terminated, running sap tool memlimits, running sap tool ipclimits
Related Notes : 
       
| 387370 | DB2/390: Calculation virtual memory - OS/390 AppServ |