check_mysql_query segfaults with mysql-connector-c

This forum is intended for the discussion of Nagios plugin development. Feature requests, patches, bug fixes, and all types of development-related discussions are welcome!

NOTE: The SourceForge.net nagiosplug-devel mailing list has been deprecated in favor of this forum in order to expedite support and provide additional features not available on the old mailing list.

check_mysql_query segfaults with mysql-connector-c

Postby internethering » Thu Oct 04, 2018 2:35 am

Hi,

Gentoo switched from mysql shipped client libs to mysql-connector-c (https://dev.mysql.com/downloads/connector/c/). This causes two problems with check_mysql_query:

/usr/lib64/nagios/plugins/check_mysql_query -q "SELECT CASE WHEN variable_value = 'ON' THEN 0 ELSE 2 END FROM information_schema.global_status WHERE variable_name='SLAVE_RUNNING'" -u munin -w 0 -c 1
Speicherzugriffsfehler (Speicherabzug geschrieben)


GDB tell me:

Code: Select all
           PID: 30491 (check_mysql_que)
           UID: 0 (root)
           GID: 0 (root)
        Signal: 11 (SEGV)
     Timestamp: Thu 2018-11-15 14:40:05 CET (2s ago)
  Command Line: /usr/lib64/nagios/plugins/check_mysql_query -q SELECT CASE WHEN variable_value = 'ON' THEN 0 ELSE 2 END FROM information_schema.global_status WHERE variable_name='SLAVE_RUNNING' -u munin -w 0 -c 1
    Executable: /usr/lib64/nagios/plugins/check_mysql_query
Control Group: /user.slice/user-1000.slice/session-584.scope
          Unit: session-584.scope
         Slice: user-1000.slice
       Session: 584
     Owner UID: 1000 (rhering)
       Boot ID: 1402b547b1f842789ffb300c29f0e707
    Machine ID: 4a6906c2152dff421b9b5a385455c64e
      Hostname: alpha.mail
       Storage: /var/lib/systemd/coredump/core.check_mysql_que.0.1402b547b1f842789ffb300c29f0e707.30491.1542289205000000.lz4
       Message: Process 30491 (check_mysql_que) of user 0 dumped core.
               
                Stack trace of thread 30491:
                #0  0x00007f74fb668c47 memset (libcrypto.so.1.0.0)
                #1  0x00007f74fc4741ca my_aes_decrypt (libmysqlclient.so.18)
                #2  0x00007f74fc47063f n/a (libmysqlclient.so.18)
                #3  0x00007f74fc470b45 n/a (libmysqlclient.so.18)
                #4  0x00007f74fc470f6e my_search_option_files (libmysqlclient.so.18)
                #5  0x00007f74fc471735 my_load_defaults (libmysqlclient.so.18)
                #6  0x00007f74fc432ec8 mysql_read_default_options (libmysqlclient.so.18)
                #7  0x00007f74fc43a3fc mysql_real_connect (libmysqlclient.so.18)
                #8  0x00005571d9d80796 main (check_mysql_query)
                #9  0x00007f74fbe33ae7 __libc_start_main (libc.so.6)
                #10 0x00005571d9d80bfa _start (check_mysql_query)

GNU gdb (Gentoo 8.2 p1) 8.2
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/lib64/nagios/plugins/check_mysql_query...Reading symbols from /usr/lib64/debug//usr/lib64/nagios/plugins/check_mysql_query.debug...done.
done.

warning: core file may not match specified executable file.
[New LWP 30491]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/usr/lib64/nagios/plugins/check_mysql_query -q SELECT CASE WHEN variable_value'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f74fb668c47 in memset (__len=168, __ch=0, __dest=0x0) at /usr/include/bits/string_fortified.h:71
71        return __builtin___memset_chk (__dest, __ch, __len, __bos0 (__dest));
(gdb) bt
#0  0x00007f74fb668c47 in memset (__len=168, __ch=0, __dest=0x0) at /usr/include/bits/string_fortified.h:71
#1  EVP_CIPHER_CTX_init (ctx=0x0) at evp_enc.c:82
#2  0x00007f74fc4741ca in my_aes_decrypt () from /usr/lib64/libmysqlclient.so.18
#3  0x00007f74fc47063f in ?? () from /usr/lib64/libmysqlclient.so.18
#4  0x00007f74fc470b45 in ?? () from /usr/lib64/libmysqlclient.so.18
#5  0x00007f74fc470f6e in my_search_option_files () from /usr/lib64/libmysqlclient.so.18
#6  0x00007f74fc471735 in my_load_defaults () from /usr/lib64/libmysqlclient.so.18
#7  0x00007f74fc432ec8 in mysql_read_default_options () from /usr/lib64/libmysqlclient.so.18
#8  0x00007f74fc43a3fc in mysql_real_connect () from /usr/lib64/libmysqlclient.so.18
#9  0x00005571d9d80796 in main (argc=<optimized out>, argv=<optimized out>) at check_mysql_query.c:100


This problem is gone, if you rename the .mylogin.cnf (see https://dev.mysql.com/doc/refman/8.0/en ... ditor.html). I've atm no idea howto solve it.

---

2nd problem:

Code: Select all
           PID: 21044 (check_mysql_que)
           UID: 0 (root)
           GID: 0 (root)
        Signal: 11 (SEGV)
     Timestamp: Thu 2018-11-15 14:45:56 CET (3s ago)
  Command Line: /usr/lib64/nagios/plugins/check_mysql_query -q SELECT CASE WHEN variable_value = 'ON' THEN 0 ELSE 2 END FROM information_schema.global_status WHERE variable_name='SLAVE_RUNNING' -u munin -w 0 -c 1
    Executable: /usr/lib64/nagios/plugins/check_mysql_query
Control Group: /user.slice/user-1000.slice/session-586.scope
          Unit: session-586.scope
         Slice: user-1000.slice
       Session: 586
     Owner UID: 1000 (rhering)
       Boot ID: 1402b547b1f842789ffb300c29f0e707
    Machine ID: 4a6906c2152dff421b9b5a385455c64e
      Hostname: alpha.mail
       Storage: /var/lib/systemd/coredump/core.check_mysql_que.0.1402b547b1f842789ffb300c29f0e707.21044.1542289556000000.lz4
       Message: Process 21044 (check_mysql_que) of user 0 dumped core.
               
                Stack trace of thread 21044:
                #0  0x00007f7f8a455613 __rawmemchr_avx2 (libc.so.6)
                #1  0x00007f7f8a374192 _IO_str_init_static_internal (libc.so.6)
                #2  0x00007f7f8a3643fd _IO_vsscanf (libc.so.6)
                #3  0x00007f7f8a35d97c __sscanf (libc.so.6)
                #4  0x000055ba80c114c8 is_numeric (check_mysql_query)
                #5  0x000055ba80c1080a main (check_mysql_query)
                #6  0x00007f7f8a311ae7 __libc_start_main (libc.so.6)
                #7  0x000055ba80c10bfa _start (check_mysql_query)

GNU gdb (Gentoo 8.2 p1) 8.2
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/lib64/nagios/plugins/check_mysql_query...Reading symbols from /usr/lib64/debug//usr/lib64/nagios/plugins/check_mysql_query.debug...done.
done.

warning: core file may not match specified executable file.
[New LWP 21044]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/usr/lib64/nagios/plugins/check_mysql_query -q SELECT CASE WHEN variable_value'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  __rawmemchr_avx2 () at ../sysdeps/x86_64/multiarch/memchr-avx2.S:61
61              VPCMPEQ (%rdi), %ymm0, %ymm1
(gdb) bt
#0  __rawmemchr_avx2 () at ../sysdeps/x86_64/multiarch/memchr-avx2.S:61
#1  0x00007f7f8a374192 in _IO_str_init_static_internal (sf=sf@entry=0x7ffed8893cd0, ptr=ptr@entry=0x8f8f8f8f8f8f8f8f <error: Cannot access memory at address 0x8f8f8f8f8f8f8f8f>, size=size@entry=0, pstart=pstart@entry=0x0) at strops.c:41
#2  0x00007f7f8a3643fd in _IO_vsscanf (string=0x8f8f8f8f8f8f8f8f <error: Cannot access memory at address 0x8f8f8f8f8f8f8f8f>, format=0x55ba80c1805c "%f%c", args=args@entry=0x7ffed8893e00) at iovsscanf.c:40
#3  0x00007f7f8a35d97c in __sscanf (s=<optimized out>, format=format@entry=0x55ba80c1805c "%f%c") at sscanf.c:32
#4  0x000055ba80c114c8 in is_numeric (number=<optimized out>) at utils.c:245
#5  0x000055ba80c1080a in main (argc=<optimized out>, argv=<optimized out>) at check_mysql_query.c:148


I got this problem fixed:

Code: Select all
diff -urN a/plugins/check_mysql_query.c b/plugins/check_mysql_query.c
--- a/plugins/check_mysql_query.c       2018-11-15 14:25:30.110775355 +0100
+++ b/plugins/check_mysql_query.c       2018-11-15 14:25:49.477436877 +0100
@@ -136,18 +136,18 @@
                die (STATE_CRITICAL, "QUERY %s: Fetch row error - %s\n", _("CRITICAL"), error);
        }

-       /* free the result */
-       mysql_free_result (res);
-
-       /* close the connection */
-       mysql_close (&mysql);
-
        if (! is_numeric(row[0])) {
                die (STATE_CRITICAL, "QUERY %s: %s - '%s'\n", _("CRITICAL"), _("Is not a numeric"), row[0]);
        }

        value = strtod(row[0], NULL);

+       /* free the result */
+       mysql_free_result (res);
+
+       /* close the connection */
+       mysql_close (&mysql);
+
        if (verbose >= 3)
                printf("mysql result: %f\n", value);


some version info:
dev-db/mysql-connector-c: 6.1.11-r1
dev-db/mysql: 5.7.23-r2
net-analyzer/nagius-plugins: 2.2.1-r3
Last edited by internethering on Thu Nov 15, 2018 8:47 am, edited 1 time in total.
internethering
 
Posts: 3
Joined: Thu Oct 04, 2018 1:40 am

Re: check_mysql_query segfaults with mysql-connector-c

Postby scottwilkerson » Thu Oct 04, 2018 3:23 pm

Where did you get this plugin from?

internethering wrote:net-analyzer/monitoring-plugins: 2.2-r3


We are not the author of this, are you sure you are working with the nagios-plugins package and not something else?
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
User avatar
scottwilkerson
DevOps Engineer
 
Posts: 12609
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises

Re: check_mysql_query segfaults with mysql-connector-c

Postby internethering » Tue Oct 16, 2018 7:26 am

scottwilkerson wrote:We are not the author of this, are you sure you are working with the nagios-plugins package and not something else?


Ups, you're right. Correct address: https://github.com/monitoring-plugins/m ... ssues/1557
internethering
 
Posts: 3
Joined: Thu Oct 04, 2018 1:40 am

Re: check_mysql_query segfaults with mysql-connector-c

Postby scottwilkerson » Tue Oct 16, 2018 7:49 am

internethering wrote:
scottwilkerson wrote:We are not the author of this, are you sure you are working with the nagios-plugins package and not something else?


Ups, you're right. Correct address: https://github.com/monitoring-plugins/m ... ssues/1557


Locking thread
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
User avatar
scottwilkerson
DevOps Engineer
 
Posts: 12609
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises

Re: check_mysql_query segfaults with mysql-connector-c

Postby scottwilkerson » Thu Nov 15, 2018 8:28 am

Unlocked per user request
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
User avatar
scottwilkerson
DevOps Engineer
 
Posts: 12609
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises

Re: check_mysql_query segfaults with mysql-connector-c

Postby internethering » Thu Nov 15, 2018 8:49 am

Thanks. The problem exists for check_mysql_query from nagios-plugins.org too. I updated my first post with actual backtraces and a patch for one problem.

By the way, check_mysql has the problem with .mylogin.cnf too:

Code: Select all
Core was generated by `/usr/lib64/nagios/plugins/check_mysql -H localhost -n'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f7f2250ac47 in memset (__len=168, __ch=0, __dest=0x0) at /usr/include/bits/string_fortified.h:71
71        return __builtin___memset_chk (__dest, __ch, __len, __bos0 (__dest));
(gdb) bt
#0  0x00007f7f2250ac47 in memset (__len=168, __ch=0, __dest=0x0) at /usr/include/bits/string_fortified.h:71
#1  EVP_CIPHER_CTX_init (ctx=0x0) at evp_enc.c:82
#2  0x00007f7f233161ca in my_aes_decrypt () from /usr/lib64/libmysqlclient.so.18
#3  0x00007f7f2331263f in ?? () from /usr/lib64/libmysqlclient.so.18
#4  0x00007f7f23312b45 in ?? () from /usr/lib64/libmysqlclient.so.18
#5  0x00007f7f23312f6e in my_search_option_files () from /usr/lib64/libmysqlclient.so.18
#6  0x00007f7f23313735 in my_load_defaults () from /usr/lib64/libmysqlclient.so.18
#7  0x00007f7f232d4ec8 in mysql_read_default_options () from /usr/lib64/libmysqlclient.so.18
#8  0x00007f7f232dc3fc in mysql_real_connect () from /usr/lib64/libmysqlclient.so.18
#9  0x000055d336ef2ca8 in main (argc=<optimized out>, argv=<optimized out>) at check_mysql.c:143
internethering
 
Posts: 3
Joined: Thu Oct 04, 2018 1:40 am

Re: check_mysql_query segfaults with mysql-connector-c

Postby scottwilkerson » Thu Nov 15, 2018 5:17 pm

Are you passing the -f flag with the full path to the file?
Code: Select all
-f, --file=STRING
    Read from the specified client options file
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
User avatar
scottwilkerson
DevOps Engineer
 
Posts: 12609
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises


Return to Nagios Plugin Development

Who is online

Users browsing this forum: No registered users and 1 guest