Out of memory error

[wangshun1121]

I tried to deploy latest release of cbioportal with docker, versions of docker images is below:

DOCKER_IMAGE_CBIOPORTAL=cbioportal/cbioportal:5.3.13
DOCKER_IMAGE_SESSION_SERVICE=cbioportal/session-service:0.6.1
DOCKER_IMAGE_MYSQL=mysql:5.7

I tried to lunch the app, but the session-service-container was always restarting.

I used following command to start a solo container to check this image:

docker run --rm -it -v /tmp:/tmp cbioportal/session-service:0.6.1 bash

in this container, I tried to start /app.war :

java -jar /app.war 

and got following info:

[0.006s][warning][os,thread] Failed to start thread "GC Thread#0" - pthread_create failed (EPERM) for attributes: stacksize: 1024k, guardsize: 4k, detached.
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Cannot create worker GC thread. Out of system resources.
# An error report file with more information is saved as:
# //hs_err_pid9.log

hs_err_pid9.log was attached here!

hs_err_pid9.log

The key information of this log is at the head of this log file:

#
# There is insufficient memory for the Java Runtime Environment to continue.
# Cannot create worker GC thread. Out of system resources.
# Possible reasons:
#   The system is out of physical RAM or swap space
#   The process is running with CompressedOops enabled, and the Java Heap may be blocking the growth of the native heap
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
#   JVM is running with Unscaled Compressed Oops mode in which the Java heap is
#     placed in the first 4GB address space. The Java Heap base address is the
#     maximum limit for the native heap growth. Please use -XX:HeapBaseMinAddress
#     to set the Java Heap base and to place the Java Heap above 4GB virtual address.
# This output file may be truncated or incomplete.
#
#  Out of Memory Error (workerManager.hpp:70), pid=9, tid=9
#
# JRE version:  (11.0.18+10) (build )
# Java VM: OpenJDK 64-Bit Server VM (11.0.18+10, mixed mode, sharing, tiered, compressed oops, g1 gc, linux-amd64)
# Core dump will be written. Default location: Core dumps may be processed with "/usr/share/apport/apport -p%p -s%s -c%c -d%d -P%P -u%u -g%g -- %E" (or dumping to //core.9)
#

System info is here:

OS:DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.2 LTS"
uname:Linux 5.15.0-78-generic #85~20.04.1-Ubuntu SMP Mon Jul 17 09:42:39 UTC 2023 x86_64
OS uptime: 7 days 22:26 hours
libc:glibc 2.35 NPTL 2.35 
rlimit (soft/hard): STACK 8192k/infinity , CORE infinity/infinity , NPROC infinity/infinity , NOFILE 1048576/1048576 , AS infinity/infinity , CPU infinity/infinity , DATA infinity/infinity , FSIZE infinity/infinity , MEMLOCK 64k/64k
load average:0.33 0.20 0.18

/proc/meminfo:
MemTotal:       197940460 kB
MemFree:        18550468 kB
MemAvailable:   190977672 kB
Buffers:         3793108 kB
Cached:         159687920 kB
SwapCached:       132200 kB
Active:         103051660 kB
Inactive:       62834060 kB
Active(anon):     662604 kB
Inactive(anon):  1740444 kB
Active(file):   102389056 kB
Inactive(file): 61093616 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:      31999996 kB
SwapFree:       29066748 kB

...
...
...

Process Memory:
Virtual Size: 430160K (peak: 430160K)
Resident Set Size: 18312K (peak: 18312K) (anon: 5492K, file: 12820K, shmem: 0K)
Swapped out: 0K
C-Heap outstanding allocations: 4240K, retained: 55K
glibc malloc tunables: (default)

/proc/sys/kernel/threads-max (system-wide limit on the number of threads):
1545803
/proc/sys/vm/max_map_count (maximum number of memory map areas a process may have):
65530
/proc/sys/kernel/pid_max (system-wide limit on number of process identifiers):
4194304

cbioportal/session-service:0.5.0 can be launched easily, but the 0.6.1 release returned insufficient memory ... how to solve this?

[wangshun1121]

OK, here I get the solution: https://ojson.com/archives/docker-pthread-create-failed-stacksize

Make sure update docker engine to latest release, or otherwise, lunch container using following command:

docker run --rm -it -v /tmp:/tmp  --security-opt seccomp=unconfined cbioportal/session-service:0.6.1 bash