Thread object leak

[laurynas-biveinis]

Valgrind (or LeakSanitizer) on a 5.6 server shutdown with HS installed gives the following errors (to see the actual stacktraces, comment out dlclose() call in sql_plugin.cc, or HS will be unloaded by the time Valgrind tries to resolve the stacktraces):

2,560 bytes in 16 blocks are indirectly lost in loss record 275 of 345
   at 0x4C2E0EF: operator new(unsigned long) (vg_replace_malloc.c:333)
   by 0x148734C2: dena::database::create_context(bool) volatile (database.cpp:200)
   by 0x148819D0: dena::hstcpsvr_worker::hstcpsvr_worker(dena::hstcpsvr_worker_arg const&) (hstcpsvr_worker.cpp:287)
   by 0x14884D47: dena::hstcpsvr_worker_i::create(dena::hstcpsvr_worker_arg const&) (hstcpsvr_worker.cpp:943)
   by 0x1488C741: dena::worker_throbj::worker_throbj(dena::hstcpsvr_worker_arg const&) (hstcpsvr.cpp:30)
   by 0x1488CD1A: dena::thread<dena::worker_throbj>::thread<dena::hstcpsvr_worker_arg>(dena::hstcpsvr_worker_arg const&, unsigned long) (thread.hpp:22)
   by 0x1488C142: dena::hstcpsvr::start_listen[abi:cxx11]() (hstcpsvr.cpp:122)
   by 0x1487FB9A: daemon_handlersocket_init(void*) (handlersocket.cpp:84)
   by 0x828808: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1159)
   by 0x82A9A3: mysql_install_plugin(THD*, st_mysql_lex_string const*, st_mysql_lex_string const*) (sql_plugin.cc:1912)
   by 0x81B4E6: mysql_execute_command(THD*) (sql_parse.cc:5422)
   by 0x81F444: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:6972)
   by 0x81092E: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1441)
   by 0x80F7A4: do_command(THD*) (sql_parse.cc:1053)
   by 0x7D27F1: do_handle_one_connection(THD*) (sql_connect.cc:1541)
   by 0x7D2289: handle_one_connection (sql_connect.cc:1444)

2,944 bytes in 16 blocks are indirectly lost in loss record 276 of 345
   at 0x4C2E0EF: operator new(unsigned long) (vg_replace_malloc.c:333)
   by 0x14884D35: dena::hstcpsvr_worker_i::create(dena::hstcpsvr_worker_arg const&) (hstcpsvr_worker.cpp:943)
   by 0x1488C741: dena::worker_throbj::worker_throbj(dena::hstcpsvr_worker_arg const&) (hstcpsvr.cpp:30)
   by 0x1488CD1A: dena::thread<dena::worker_throbj>::thread<dena::hstcpsvr_worker_arg>(dena::hstcpsvr_worker_arg const&, unsigned long) (thread.hpp:22)
   by 0x1488C142: dena::hstcpsvr::start_listen[abi:cxx11]() (hstcpsvr.cpp:122)
   by 0x1487FB9A: daemon_handlersocket_init(void*) (handlersocket.cpp:84)
   by 0x828808: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1159)
   by 0x82A9A3: mysql_install_plugin(THD*, st_mysql_lex_string const*, st_mysql_lex_string const*) (sql_plugin.cc:1912)
   by 0x81B4E6: mysql_execute_command(THD*) (sql_parse.cc:5422)
   by 0x81F444: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:6972)
   by 0x81092E: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1441)
   by 0x80F7A4: do_command(THD*) (sql_parse.cc:1053)
   by 0x7D27F1: do_handle_one_connection(THD*) (sql_connect.cc:1541)
   by 0x7D2289: handle_one_connection (sql_connect.cc:1444)
   by 0xB958CF: pfs_spawn_thread (pfs.cc:1860)
   by 0x58A16F9: start_thread (pthread_create.c:333)

12,800 bytes in 16 blocks are indirectly lost in loss record 289 of 345
   at 0x4C2E0EF: operator new(unsigned long) (vg_replace_malloc.c:333)
   by 0x14873CFD: dena::dbcontext::init_thread(void const*, int volatile&) (database.cpp:333)
   by 0x14881CBE: dena::(anonymous namespace)::thr_init::thr_init(std::auto_ptr<dena::dbcontext_i> const&, int volatile&) (hstcpsvr_worker.cpp:311)
   by 0x14881D2D: dena::hstcpsvr_worker::run() (hstcpsvr_worker.cpp:324)
   by 0x1488C7AA: dena::worker_throbj::operator()() (hstcpsvr.cpp:32)
   by 0x1488E050: dena::thread<dena::worker_throbj>::thread_main(void*) (thread.hpp:71)
   by 0x58A16F9: start_thread (pthread_create.c:333)
   by 0x645EB5C: clone (clone.S:109)

131,072 bytes in 16 blocks are indirectly lost in loss record 311 of 345
   at 0x4C2E0EF: operator new(unsigned long) (vg_replace_malloc.c:333)
   by 0x1487E4DF: __gnu_cxx::new_allocator<char>::allocate(unsigned long, void const*) (new_allocator.h:104)
   by 0x1487DC1B: __gnu_cxx::__alloc_traits<std::allocator<char> >::allocate(std::allocator<char>&, unsigned long) (alloc_traits.h:182)
   by 0x1487D301: std::_Vector_base<char, std::allocator<char> >::_M_allocate(unsigned long) (stl_vector.h:170)
   by 0x1487C0A1: std::vector<char, std::allocator<char> >::_M_fill_insert(__gnu_cxx::__normal_iterator<char*, std::vector<char, std::allocator<char> > >, unsigned long, char const&) (vector.tcc:491)
   by 0x1487AB61: std::vector<char, std::allocator<char> >::insert(__gnu_cxx::__normal_iterator<char*, std::vector<char, std::allocator<char> > >, unsigned long, char const&) (stl_vector.h:1073)
   by 0x14879C44: std::vector<char, std::allocator<char> >::resize(unsigned long, char) (stl_vector.h:716)
   by 0x148736EC: dena::dbcontext::dbcontext(dena::database volatile*, bool) (database.cpp:226)
   by 0x148734D8: dena::database::create_context(bool) volatile (database.cpp:200)
   by 0x148819D0: dena::hstcpsvr_worker::hstcpsvr_worker(dena::hstcpsvr_worker_arg const&) (hstcpsvr_worker.cpp:287)
   by 0x14884D47: dena::hstcpsvr_worker_i::create(dena::hstcpsvr_worker_arg const&) (hstcpsvr_worker.cpp:943)
   by 0x1488C741: dena::worker_throbj::worker_throbj(dena::hstcpsvr_worker_arg const&) (hstcpsvr.cpp:30)
   by 0x1488CD1A: dena::thread<dena::worker_throbj>::thread<dena::hstcpsvr_worker_arg>(dena::hstcpsvr_worker_arg const&, unsigned long) (thread.hpp:22)
   by 0x1488C142: dena::hstcpsvr::start_listen[abi:cxx11]() (hstcpsvr.cpp:122)
   by 0x1487FB9A: daemon_handlersocket_init(void*) (handlersocket.cpp:84)
   by 0x828808: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1159)

737,280 bytes in 6 blocks are indirectly lost in loss record 325 of 345
   at 0x4C2E0EF: operator new(unsigned long) (vg_replace_malloc.c:333)
   by 0x1488A980: __gnu_cxx::new_allocator<epoll_event>::allocate(unsigned long, void const*) (new_allocator.h:104)
   by 0x1488A267: __gnu_cxx::__alloc_traits<std::allocator<epoll_event> >::allocate(std::allocator<epoll_event>&, unsigned long) (alloc_traits.h:182)
   by 0x1488949F: std::_Vector_base<epoll_event, std::allocator<epoll_event> >::_M_allocate(unsigned long) (stl_vector.h:170)
   by 0x14887BE4: std::vector<epoll_event, std::allocator<epoll_event> >::_M_fill_insert(__gnu_cxx::__normal_iterator<epoll_event*, std::vector<epoll_event, std::allocator<epoll_event> > >, unsigned long, epoll_event const&) (vector.tcc:491)
   by 0x14886AA5: std::vector<epoll_event, std::allocator<epoll_event> >::insert(__gnu_cxx::__normal_iterator<epoll_event*, std::vector<epoll_event, std::allocator<epoll_event> > >, unsigned long, epoll_event const&) (stl_vector.h:1073)
   by 0x14885E42: std::vector<epoll_event, std::allocator<epoll_event> >::resize(unsigned long, epoll_event) (stl_vector.h:716)
   by 0x14881BF4: dena::hstcpsvr_worker::hstcpsvr_worker(dena::hstcpsvr_worker_arg const&) (hstcpsvr_worker.cpp:301)
   by 0x14884D47: dena::hstcpsvr_worker_i::create(dena::hstcpsvr_worker_arg const&) (hstcpsvr_worker.cpp:943)
   by 0x1488C741: dena::worker_throbj::worker_throbj(dena::hstcpsvr_worker_arg const&) (hstcpsvr.cpp:30)
   by 0x1488CD1A: dena::thread<dena::worker_throbj>::thread<dena::hstcpsvr_worker_arg>(dena::hstcpsvr_worker_arg const&, unsigned long) (thread.hpp:22)
   by 0x1488C142: dena::hstcpsvr::start_listen[abi:cxx11]() (hstcpsvr.cpp:122)
   by 0x1487FB9A: daemon_handlersocket_init(void*) (handlersocket.cpp:84)
   by 0x828808: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1159)
   by 0x82A9A3: mysql_install_plugin(THD*, st_mysql_lex_string const*, st_mysql_lex_string const*) (sql_plugin.cc:1912)
   by 0x81B4E6: mysql_execute_command(THD*) (sql_parse.cc:5422)

887,168 (512 direct, 886,656 indirect) bytes in 16 blocks are definitely lost in loss record 326 of 345
   at 0x4C2E0EF: operator new(unsigned long) (vg_replace_malloc.c:333)
   by 0x1488C129: dena::hstcpsvr::start_listen[abi:cxx11]() (hstcpsvr.cpp:122)
   by 0x1487FB9A: daemon_handlersocket_init(void*) (handlersocket.cpp:84)
   by 0x828808: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1159)
   by 0x82A9A3: mysql_install_plugin(THD*, st_mysql_lex_string const*, st_mysql_lex_string const*) (sql_plugin.cc:1912)
   by 0x81B4E6: mysql_execute_command(THD*) (sql_parse.cc:5422)
   by 0x81F444: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:6972)
   by 0x81092E: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1441)
   by 0x80F7A4: do_command(THD*) (sql_parse.cc:1053)
   by 0x7D27F1: do_handle_one_connection(THD*) (sql_connect.cc:1541)
   by 0x7D2289: handle_one_connection (sql_connect.cc:1444)
   by 0xB958CF: pfs_spawn_thread (pfs.cc:1860)
   by 0x58A16F9: start_thread (pthread_create.c:333)
   by 0x645EB5C: clone (clone.S:109)

1,228,800 bytes in 10 blocks are possibly lost in loss record 333 of 345
   at 0x4C2E0EF: operator new(unsigned long) (vg_replace_malloc.c:333)
   by 0x1488A980: __gnu_cxx::new_allocator<epoll_event>::allocate(unsigned long, void const*) (new_allocator.h:104)
   by 0x1488A267: __gnu_cxx::__alloc_traits<std::allocator<epoll_event> >::allocate(std::allocator<epoll_event>&, unsigned long) (alloc_traits.h:182)
   by 0x1488949F: std::_Vector_base<epoll_event, std::allocator<epoll_event> >::_M_allocate(unsigned long) (stl_vector.h:170)
   by 0x14887BE4: std::vector<epoll_event, std::allocator<epoll_event> >::_M_fill_insert(__gnu_cxx::__normal_iterator<epoll_event*, std::vector<epoll_event, std::allocator<epoll_event> > >, unsigned long, epoll_event const&) (vector.tcc:491)
   by 0x14886AA5: std::vector<epoll_event, std::allocator<epoll_event> >::insert(__gnu_cxx::__normal_iterator<epoll_event*, std::vector<epoll_event, std::allocator<epoll_event> > >, unsigned long, epoll_event const&) (stl_vector.h:1073)
   by 0x14885E42: std::vector<epoll_event, std::allocator<epoll_event> >::resize(unsigned long, epoll_event) (stl_vector.h:716)
   by 0x14881BF4: dena::hstcpsvr_worker::hstcpsvr_worker(dena::hstcpsvr_worker_arg const&) (hstcpsvr_worker.cpp:301)
   by 0x14884D47: dena::hstcpsvr_worker_i::create(dena::hstcpsvr_worker_arg const&) (hstcpsvr_worker.cpp:943)
   by 0x1488C741: dena::worker_throbj::worker_throbj(dena::hstcpsvr_worker_arg const&) (hstcpsvr.cpp:30)
   by 0x1488CD1A: dena::thread<dena::worker_throbj>::thread<dena::hstcpsvr_worker_arg>(dena::hstcpsvr_worker_arg const&, unsigned long) (thread.hpp:22)
   by 0x1488C142: dena::hstcpsvr::start_listen[abi:cxx11]() (hstcpsvr.cpp:122)
   by 0x1487FB9A: daemon_handlersocket_init(void*) (handlersocket.cpp:84)
   by 0x828808: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1159)
   by 0x82A9A3: mysql_install_plugin(THD*, st_mysql_lex_string const*, st_mysql_lex_string const*) (sql_plugin.cc:1912)
   by 0x81B4E6: mysql_execute_command(THD*) (sql_parse.cc:5422)

This is caused by hstcpsvr::threads, which is auto_ptrcontainer<std::vector<worker_thread_type *> > (effectively std::vector<foo *>), having clear() method, which clears the container without deleting the member pointers, which are owned by the container.

HandlerSocket bundled with Percona Server 5.6 has merged the fix at percona/percona-server#937.