PHP-FPM7 SIGSEGV - core dumped

manpreet Best Answer 3 years ago

I'm having some issues with just a single site that keeps segfaulting php-fpm7.

Can't seem to pinpoint the exact moment when this occurs, i browse around on the site and wait and after a while when something is closed the core dump is produced (This happens very frequently).

[Fri Aug 11 14:31:37 2017] php-fpm[15202]: segfault at 7f68cdd238c5 ip 0000562a8248055f sp 00007ffc22d8ea70 error 4 in php-fpm7.0[562a82214000+45e000]
[Fri Aug 11 14:36:50 2017] php-fpm[18507]: segfault at 7f68cd5d0345 ip 0000562a8248055f sp 00007ffc22d8ea70 error 4 in php-fpm7.0[562a82214000+45e000]
[Fri Aug 11 14:37:17 2017] php-fpm[18658]: segfault at 7f68cd5b6345 ip 0000562a8248055f sp 00007ffc22d8ea70 error 4 in php-fpm7.0[562a82214000+45e000]
[Fri Aug 11 14:38:06 2017] php-fpm[18884]: segfault at 7f68cd5286e5 ip 0000562a8248055f sp 00007ffc22d8ea70 error 4 in php-fpm7.0[562a82214000+45e000]
[Fri Aug 11 14:38:21 2017] php-fpm[18944]: segfault at 7f68cd5fc525 ip 0000562a8248055f sp 00007ffc22d8ea70 error 4 in php-fpm7.0[562a82214000+45e000]
[Fri Aug 11 14:39:26 2017] php-fpm[19447]: segfault at 7f68cd30c915 ip 0000562a8248055f sp 00007ffc22d8ea70 error 4 in php-fpm7.0[562a82214000+45e000]
[Fri Aug 11 14:42:34 2017] php-fpm[21669]: segfault at 7f68cd5575c5 ip 0000562a8248055f sp 00007ffc22d8ea70 error 4 in php-fpm7.0[562a82214000+45e000]
[Fri Aug 11 14:47:29 2017] php-fpm[24499]: segfault at 7f68cd536685 ip 0000562a8248055f sp 00007ffc22d8ea70 error 4 in php-fpm7.0[562a82214000+45e000]
[Fri Aug 11 14:48:30 2017] php-fpm[25072]: segfault at 7f68cd554725 ip 0000562a8248055f sp 00007ffc22d8ea70 error 4 in php-fpm7.0[562a82214000+45e000]
[Fri Aug 11 14:48:54 2017] php-fpm[24750]: segfault at 7f68ccd5a5c5 ip 0000562a8248055f sp 00007ffc22d8ea70 error 4 in php-fpm7.0[562a82214000+45e000]
[Fri Aug 11 14:49:49 2017] php-fpm[25857]: segfault at 7f68cd5f6ba5 ip 0000562a8248055f sp 00007ffc22d8ea70 error 4 in php-fpm7.0[562a82214000+45e000]

From the stacktrace it looks like some soap request is destroyed and that's where the crash occurs but i can't even find a soap request in the site source.

First thought would be to install php7.0-dev but i already have that installed from the error in the stacktrace: /usr/src/builddir/Zend/zend_string.h: No such file or directory

Php-FPM version

PHP 7.0.20-1~dotdeb+8.2 (fpm-fcgi)
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2017 Zend Technologies
    with Zend OPcache v7.0.20-1~dotdeb+8.2, Copyright (c) 1999-2017, by Zend Technologies

PHP-FPM Pool configuration

[mysite]
user = mysite
group = mysite
prefix = /data/sites/mysite
listen = /var/run/php-fpm-mysite.sock
chdir = /data/sites/mysite/public_html
php_admin_value[disable_functions] = exec,passthru,shell_exec,system
listen.owner = www-data
listen.group = www-data
pm = ondemand
pm.process_idle_timeout = 10s
pm.max_children = 4
pm.max_requests = 200
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 2
php_value[session.save_path] = /tmp/php/sessions
php_value[session_save_path] = /tmp/php/sessions

PHP-Modules

ii  dh-php                         0.21-1~dotdeb+8.1                all          debhelper add-on to handle PHP PECL extensions
ii  php-common                     1:42-1~dotdeb+8.1                all          Common files for PHP packages
ii  php7.0-cli                     7.0.20-1~dotdeb+8.2              amd64        command-line interpreter for the PHP scripting language
ii  php7.0-common                  7.0.20-1~dotdeb+8.2              amd64        documentation, examples and common module for PHP
ii  php7.0-curl                    7.0.20-1~dotdeb+8.2              amd64        CURL module for PHP
ii  php7.0-dev                     7.0.20-1~dotdeb+8.2              amd64        Files for PHP7.0 module development
ii  php7.0-fpm                     7.0.20-1~dotdeb+8.2              amd64        server-side, HTML-embedded scripting language (FPM-CGI binary)
ii  php7.0-gd                      7.0.20-1~dotdeb+8.2              amd64        GD module for PHP
ii  php7.0-imagick                 3.4.3-1~dotdeb+8.1               amd64        Provides a wrapper to the ImageMagick library
ii  php7.0-imap                    7.0.20-1~dotdeb+8.2              amd64        IMAP module for PHP
ii  php7.0-intl                    7.0.20-1~dotdeb+8.2              amd64        Internationalisation module for PHP
ii  php7.0-json                    7.0.20-1~dotdeb+8.2              amd64        JSON module for PHP
ii  php7.0-mbstring                7.0.20-1~dotdeb+8.2              amd64        MBSTRING module for PHP
ii  php7.0-mcrypt                  7.0.20-1~dotdeb+8.2              amd64        libmcrypt module for PHP
ii  php7.0-mysql                   7.0.20-1~dotdeb+8.2              amd64        MySQL module for PHP
ii  php7.0-opcache                 7.0.20-1~dotdeb+8.2              amd64        Zend OpCache module for PHP
ii  php7.0-readline                7.0.20-1~dotdeb+8.2              amd64        readline module for PHP
ii  php7.0-soap                    7.0.20-1~dotdeb+8.2              amd64        SOAP module for PHP
ii  php7.0-xml                     7.0.20-1~dotdeb+8.2              amd64        DOM, SimpleXML, WDDX, XML, and XSL module for PHP
ii  php7.0-zip                     7.0.20-1~dotdeb+8.2              amd64        Zip module for PHP

Platform

Kubernetes/Docker container inside Google Container Engine

NGINX version & build

nginx version: nginx/1.10.3
built with OpenSSL 1.0.1t  3 May 2016
TLS SNI support enabled
configure arguments: --with-cc-opt='-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-z,relro -Wl,-z,now' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --modules-path=/usr/lib/nginx/modules --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_v2_module --with-http_dav_module --with-file-aio --with-threads --with-http_addition_module --with-http_geoip_module=dynamic --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_secure_link_module --with-http_sub_module --with-http_xslt_module=dynamic --with-stream=dynamic --with-stream_ssl_module --with-mail=dynamic --with-mail_ssl_module --add-dynamic-module=/usr/src/builddir/debian/modules/nginx-auth-pam --add-module=/usr/src/builddir/debian/modules/nginx-dav-ext-module --add-module=/usr/src/builddir/debian/modules/nginx-echo --add-module=/usr/src/builddir/debian/modules/nginx-upstream-fair --add-module=/usr/src/builddir/debian/modules/ngx_http_substitutions_filter_module --add-module=/usr/src/builddir/debian/modules/nginx-cache-purge --add-module=/usr/src/builddir/debian/modules/nginx-x-rid-header --with-ld-opt=-lossp-uuid

Stacktrace

Program terminated with signal SIGSEGV, Segmentation fault.
#0  zend_string_release (s=0x7fd20d50c4c0) at /usr/src/builddir/Zend/zend_string.h:269
269     /usr/src/builddir/Zend/zend_string.h: No such file or directory.
(gdb) bt full
#0  zend_string_release (s=0x7fd20d50c4c0) at /usr/src/builddir/Zend/zend_string.h:269
No locals.
#1  zend_hash_destroy (ht=0x55d590f01c30) at /usr/src/builddir/Zend/zend_hash.c:1277
        p = 0x55d590f01c90
        end = 0x55d590f01d10
#2  0x00007fd215897c21 in delete_type_persistent (zv=) at /usr/src/builddir/ext/soap/php_schema.c:2459
        type = 0x55d590f01b10
#3  0x000055d58e956e55 in zend_hash_destroy (ht=0x55d590ef8380) at /usr/src/builddir/Zend/zend_hash.c:1264
        p = 0x55d590f003c8
        end = 0x55d590f00aa8
#4  0x00007fd21589f121 in delete_psdl_int (p=) at /usr/src/builddir/ext/soap/php_sdl.c:3130
        tmp = 0x55d590ef8270
#5  0x00007fd21589f18f in delete_psdl (zv=0x55d590f211a0) at /usr/src/builddir/ext/soap/php_sdl.c:3150
No locals.
#6  0x000055d58e956ef6 in zend_hash_destroy (ht=0x55d590efcf40) at /usr/src/builddir/Zend/zend_hash.c:1275
        p = 0x55d590f211a0
        end = 0x55d590f21200
#7  0x00007fd21586f71b in zm_shutdown_soap (type=, module_number=40) at /usr/src/builddir/ext/soap/soap.c:598
No locals.
#8  0x000055d58e94c577 in module_destructor (module=module@entry=0x55d590c19f20) at /usr/src/builddir/Zend/zend_API.c:2505
No locals.
#9  0x000055d58e94460c in module_destructor_zval (zv=) at /usr/src/builddir/Zend/zend.c:620
        module = 0x55d590c19f20
#10 0x000055d58e9579c9 in _zend_hash_del_el_ex (prev=, p=, idx=, ht=) at /usr/src/builddir/Zend/zend_hash.c:1025
        tmp = {value = {lval = 94375745003296, dval = 4.6627813406802235e-310, counted = 0x55d590c19f20, str = 0x55d590c19f20, arr = 0x55d590c19f20, obj = 0x55d590c19f20, 
            res = 0x55d590c19f20, ref = 0x55d590c19f20, ast = 0x55d590c19f20, zv = 0x55d590c19f20, ptr = 0x55d590c19f20, ce = 0x55d590c19f20, func = 0x55d590c19f20, ww = {w1 = 2428608288, 
              w2 = 21973}}, u1 = {v = {type = 17 '\021', type_flags = 0 '\000', const_flags = 0 '\000', reserved = 0 '\000'}, type_info = 17}, u2 = {var_flags = 21973, next = 21973, 
            cache_slot = 21973, lineno = 21973, num_args = 21973, fe_pos = 21973, fe_iter_idx = 21973}}
#11 _zend_hash_del_el (p=0x55d590c0ed00, idx=39, ht=0x55d58edf0d60 ) at /usr/src/builddir/Zend/zend_hash.c:1049
        prev = 
#12 zend_hash_graceful_reverse_destroy (ht=ht@entry=0x55d58edf0d60 ) at /usr/src/builddir/Zend/zend_hash.c:1505
        idx = 39
        p = 0x55d590c0ed00
#13 0x000055d58e94a95c in zend_destroy_modules () at /usr/src/builddir/Zend/zend_API.c:1982
No locals.
#14 0x000055d58e9456a5 in zend_shutdown () at /usr/src/builddir/Zend/zend.c:856
No locals.
#15 0x000055d58e8e4c0b in php_module_shutdown () at /usr/src/builddir/main/main.c:2360
No locals.
#16 0x000055d58e7ca015 in main (argc=-1865301124, argv=0x55d590d1c2dc) at /usr/src/builddir/sapi/fpm/fpm/fpm_main.c:2021
        c = -1865319872
        file_handle = {handle = {fd = 591867904, fp = 0x7fd223473000, stream = {handle = 0x7fd223473000, isatty = 0, mmap = {len = 3799, pos = 0, map = 0x0, 
                buf = 0x7fd2260ed000 , old_handle = 0x0, old_closer = 0x0}, reader = 0x55d58e8fc140 <_php_stream_read>, 
              fsizer = 0x55d58e8e2990 <php_zend_stream_fsizer>, closer = 0x55d58e8e2970 <php_zend_stream_mmap_closer>}}, filename = 0x7fd223402000 "( @#\322\177", opened_path = 0x0, 
          type = ZEND_HANDLE_FILENAME, free_filename = 0 '\000'}
        max_requests = 200
        fcgi_fd = -1897990464
        __func__ = "main"

Any insight would really be appreciated, i can't seem to figure out what is causing this.