Hi,
I'm migrating 389DS from 1.2.11 to 1.4.0.11 on Debian Buster. I have two plug-ins which I would like to use with new server, they compile ok. But server crashes when they are about to be used.
What is actual documentation? I've found
https://access.redhat.com/documentation/en-us/red_hat_directory_server/10/ht...
But I'm not sure it this is latest for plugins. For server itself it is not, it speaks about obsoleted Admin Console.
Thanks
Sorry the plugin guide has not been maintained in a long time. There was a discussion to just remove it. Can you provide the stack trace from the crash? I'm sure we help get it straightened out...
On 8/28/20 6:32 AM, Jan Tomasek wrote:
Hi,
I'm migrating 389DS from 1.2.11 to 1.4.0.11 on Debian Buster. I have two plug-ins which I would like to use with new server, they compile ok. But server crashes when they are about to be used.
What is actual documentation? I've found
https://access.redhat.com/documentation/en-us/red_hat_directory_server/10/ht...
But I'm not sure it this is latest for plugins. For server itself it is not, it speaks about obsoleted Admin Console.
Thanks
389-users mailing list -- 389-users@lists.fedoraproject.org To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject....
Hi Mark,
On 8/28/20 2:51 PM, Mark Reynolds wrote:
Sorry the plugin guide has not been maintained in a long time. There was a discussion to just remove it. Can you provide the stack trace from the crash? I'm sure we help get it straightened out...
you are very kind. My C knowledge is kinda outdated, it's about 20years I last time created something bigger in C.
I'm fighting with gdb how to be able trace debug 389 ds with plugin loaded:
root@ldap33:~# gdb /usr/sbin/ns-slapd
(gdb) run -d 65536 -D /etc/dirsrv/slapd-ldap33 -i /var/run/dirsrv/slapd-ldap33.pid Starting program: /usr/sbin/ns-slapd -d 65536 -D /etc/dirsrv/slapd-ldap33 -i /var/run/dirsrv/slapd-ldap33.pid [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [31/Aug/2020:11:14:36.396980567 +0200] - DEBUG - syntax-plugin - => bin_init ... [31/Aug/2020:11:15:11.443569660 +0200] - ERR - altpass-plugin - do_pre_bind: 1 [31/Aug/2020:11:15:11.445011559 +0200] - ERR - altpass-plugin - do_pre_bind: 2 [31/Aug/2020:11:15:11.446302153 +0200] - ERR - altpass-plugin - do_pre_bind: 3 [31/Aug/2020:11:15:11.447546080 +0200] - ERR - altpass-plugin - do_pre_bind: 4 [31/Aug/2020:11:15:11.448848356 +0200] - ERR - altpass-plugin - do_pre_bind: 5 [31/Aug/2020:11:15:11.450387903 +0200] - ERR - altpass-plugin - do_pre_bind: 6 [31/Aug/2020:11:15:11.451510488 +0200] - ERR - altpass-plugin - do_pre_bind: 7 [31/Aug/2020:11:15:11.453559193 +0200] - ERR - altpass-plugin - do_pre_bind: 8 [31/Aug/2020:11:15:11.454709087 +0200] - ERR - altpass-plugin - do_pre_bind: 9 [31/Aug/2020:11:15:11.455636136 +0200] - ERR - altpass-plugin - do_pre_bind: 9a [31/Aug/2020:11:15:11.456657442 +0200] - ERR - altpass-plugin - do_pre_bind: 9a: filter=(memberNisNetgroup=2001:718:1:6::134:138)
Thread 17 "ns-slapd" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fffd0ff9700 (LWP 28079)] 0x00007ffff4318b52 in do_pre_bind () from /usr/lib/x86_64-linux-gnu/dirsrv/plugins/altpass-plugin.so (gdb) bt #0 0x00007ffff4318b52 in do_pre_bind () at /usr/lib/x86_64-linux-gnu/dirsrv/plugins/altpass-plugin.so #1 0x00007ffff4318f3b in pre_bind () at /usr/lib/x86_64-linux-gnu/dirsrv/plugins/altpass-plugin.so #2 0x00007ffff7f0c409 in None () at /usr/lib/x86_64-linux-gnu/dirsrv/libslapd.so.0 #3 0x00007ffff7f0c654 in plugin_call_plugins () at /usr/lib/x86_64-linux-gnu/dirsrv/libslapd.so.0 #4 0x000055555556907e in None () #5 0x000055555557045a in None () #6 0x00007ffff7c13ec7 in None () at /usr/lib/x86_64-linux-gnu/libnspr4.so #7 0x00007ffff7bb3fa3 in start_thread (arg=<optimized out>) at pthread_create.c:486 #8 0x00007ffff77ed4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
When I set breakpoint at start of do_pre_bind():
(gdb) b do_pre_bind Breakpoint 1 at 0x7ffff431879a (gdb) r The program being debugged has been started already. Start it from the beginning? (y or n) y
[Switching to Thread 0x7fffd0ff9700 (LWP 28116)]
Thread 17 "ns-slapd" hit Breakpoint 1, 0x00007ffff431879a in do_pre_bind () from /usr/lib/x86_64-linux-gnu/dirsrv/plugins/altpass-plugin.so (gdb)
(gdb) l 1 ../sysdeps/x86_64/crti.S: No such file or directory. (gdb)
Tips how to properly set debug environment would be very welcome. I was unable to locate crti.S anywhere in debian packages https://packages.debian.org/search?searchon=contents&keywords=x86_64%2Fc...
Source code around SIGSEGV place:
slapi_entry_free(user_entry); user_entry = NULL; log_fatal("do_pre_bind: 9\n"); // Find corresponding service(s) char filter[200]; snprintf(filter, sizeof(filter), "(memberNisNetgroup=%s)", clientIP); log_fatal("do_pre_bind: 9a\n"); log_fatal("do_pre_bind: 9a: filter=%s\n", filter); find_entries(conf->group_suffix, filter, attributes, &matching_services);
function find_entries() is never entered.
Thanks
On 31 Aug 2020, at 19:27, Jan Tomasek jan@tomasek.cz wrote:
Hi Mark,
On 8/28/20 2:51 PM, Mark Reynolds wrote:
Sorry the plugin guide has not been maintained in a long time. There was a discussion to just remove it. Can you provide the stack trace from the crash? I'm sure we help get it straightened out...
you are very kind. My C knowledge is kinda outdated, it's about 20years I last time created something bigger in C.
I'm fighting with gdb how to be able trace debug 389 ds with plugin loaded:
root@ldap33:~# gdb /usr/sbin/ns-slapd
(gdb) run -d 65536 -D /etc/dirsrv/slapd-ldap33 -i /var/run/dirsrv/slapd-ldap33.pid Starting program: /usr/sbin/ns-slapd -d 65536 -D /etc/dirsrv/slapd-ldap33 -i /var/run/dirsrv/slapd-ldap33.pid [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [31/Aug/2020:11:14:36.396980567 +0200] - DEBUG - syntax-plugin - => bin_init ... [31/Aug/2020:11:15:11.443569660 +0200] - ERR - altpass-plugin - do_pre_bind: 1 [31/Aug/2020:11:15:11.445011559 +0200] - ERR - altpass-plugin - do_pre_bind: 2 [31/Aug/2020:11:15:11.446302153 +0200] - ERR - altpass-plugin - do_pre_bind: 3 [31/Aug/2020:11:15:11.447546080 +0200] - ERR - altpass-plugin - do_pre_bind: 4 [31/Aug/2020:11:15:11.448848356 +0200] - ERR - altpass-plugin - do_pre_bind: 5 [31/Aug/2020:11:15:11.450387903 +0200] - ERR - altpass-plugin - do_pre_bind: 6 [31/Aug/2020:11:15:11.451510488 +0200] - ERR - altpass-plugin - do_pre_bind: 7 [31/Aug/2020:11:15:11.453559193 +0200] - ERR - altpass-plugin - do_pre_bind: 8 [31/Aug/2020:11:15:11.454709087 +0200] - ERR - altpass-plugin - do_pre_bind: 9 [31/Aug/2020:11:15:11.455636136 +0200] - ERR - altpass-plugin - do_pre_bind: 9a [31/Aug/2020:11:15:11.456657442 +0200] - ERR - altpass-plugin - do_pre_bind: 9a: filter=(memberNisNetgroup=2001:718:1:6::134:138) Thread 17 "ns-slapd" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fffd0ff9700 (LWP 28079)] 0x00007ffff4318b52 in do_pre_bind () from /usr/lib/x86_64-linux-gnu/dirsrv/plugins/altpass-plugin.so (gdb) bt #0 0x00007ffff4318b52 in do_pre_bind () at /usr/lib/x86_64-linux-gnu/dirsrv/plugins/altpass-plugin.so #1 0x00007ffff4318f3b in pre_bind () at /usr/lib/x86_64-linux-gnu/dirsrv/plugins/altpass-plugin.so #2 0x00007ffff7f0c409 in None () at /usr/lib/x86_64-linux-gnu/dirsrv/libslapd.so.0 #3 0x00007ffff7f0c654 in plugin_call_plugins () at /usr/lib/x86_64-linux-gnu/dirsrv/libslapd.so.0 #4 0x000055555556907e in None () #5 0x000055555557045a in None () #6 0x00007ffff7c13ec7 in None () at /usr/lib/x86_64-linux-gnu/libnspr4.so #7 0x00007ffff7bb3fa3 in start_thread (arg=<optimized out>) at pthread_create.c:486 #8 0x00007ffff77ed4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
When I set breakpoint at start of do_pre_bind():
Reading this trace, it looks like you are missing debug symbols or devel information. Honestly, I'm not sure how to get this on debian, maybe "pkgname-dbgsym" aka 389-ds-dbgsym or similar needs to be installed?
(gdb) b do_pre_bind Breakpoint 1 at 0x7ffff431879a (gdb) r The program being debugged has been started already. Start it from the beginning? (y or n) y [Switching to Thread 0x7fffd0ff9700 (LWP 28116)] Thread 17 "ns-slapd" hit Breakpoint 1, 0x00007ffff431879a in do_pre_bind () from /usr/lib/x86_64-linux-gnu/dirsrv/plugins/altpass-plugin.so (gdb) (gdb) l 1 ../sysdeps/x86_64/crti.S: No such file or directory. (gdb)
Tips how to properly set debug environment would be very welcome. I was unable to locate crti.S anywhere in debian packages https://packages.debian.org/search?searchon=contents&keywords=x86_64%2Fc...
Source code around SIGSEGV place:
Well, you need to also compile your plugin with debug symbols too in order to see what's going on in there ... I think you can add -g3 to your gcc invocation to do this. There isn't really a penalty to having debug symbols around even for production, so feel free to leave -g3 in your compiler calls.
slapi_entry_free(user_entry); user_entry = NULL; log_fatal("do_pre_bind: 9\n"); // Find corresponding service(s) char filter[200]; snprintf(filter, sizeof(filter), "(memberNisNetgroup=%s)", clientIP); log_fatal("do_pre_bind: 9a\n"); log_fatal("do_pre_bind: 9a: filter=%s\n", filter); find_entries(conf->group_suffix, filter, attributes, &matching_services);
function find_entries() is never entered.
If there is a segfault around here, then it would likely be in the snprintf call. clientIP may be NULL, which could cause this. You could check this with "if (!clientIP) { ... }" and then log a message.
Hope that helps,
Thanks
Jan Tomasek aka Semik http://www.tomasek.cz/
389-users mailing list -- 389-users@lists.fedoraproject.org To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject....
— Sincerely,
William Brown
Senior Software Engineer, 389 Directory Server SUSE Labs
Hi William,
Reading this trace, it looks like you are missing debug symbols or devel information. Honestly, I'm not sure how to get this on debian, maybe "pkgname-dbgsym" aka 389-ds-dbgsym or similar needs to be installed?
Debian way is to add an extra repository, which contains all -dbgsym packages. It is described here: https://wiki.debian.org/HowToGetABacktrace
The problem was in the declaration of variable method:
static int do_pre_bind(Slapi_PBlock *pb, char* errmsg) { static const char* attributes[] = {"cn", NULL};
plugin_config_t* conf; int rc, method; ... conf = &s_conf; ... if (slapi_pblock_get(pb, SLAPI_BIND_TARGET, &dn) != 0 || slapi_pblock_get(pb, SLAPI_BIND_METHOD, &method) != 0 ...
Calling 'slapi_pblock_get(pb, SLAPI_BIND_METHOD, &method)' causes overwrite of conf.
In https://access.redhat.com/documentation/en-us/red_hat_directory_server/10/ht... is return of 'slapi_pblock_get(pb, SLAPI_BIND_METHOD, &method)' still 'int'
But in the source: https://pagure.io/389-ds-base/blob/master/f/ldap/servers/slapd/pblock.c#_157...
is used ber_tag_t
After I changed the declaration: ber_tag_t method;
Plugin started work. I need to deeply test it, but it looks good.
I appreciate your kind way of helping me.
Thanks a lot!
On 1 Sep 2020, at 23:35, Jan Tomasek jan@tomasek.cz wrote:
Hi William,
Reading this trace, it looks like you are missing debug symbols or devel information. Honestly, I'm not sure how to get this on debian, maybe "pkgname-dbgsym" aka 389-ds-dbgsym or similar needs to be installed?
Debian way is to add an extra repository, which contains all -dbgsym packages. It is described here: https://wiki.debian.org/HowToGetABacktrace
The problem was in the declaration of variable method:
static int do_pre_bind(Slapi_PBlock *pb, char* errmsg) { static const char* attributes[] = {"cn", NULL};
plugin_config_t* conf; int rc, method; ... conf = &s_conf; ... if (slapi_pblock_get(pb, SLAPI_BIND_TARGET, &dn) != 0 || slapi_pblock_get(pb, SLAPI_BIND_METHOD, &method) != 0 ...
Calling 'slapi_pblock_get(pb, SLAPI_BIND_METHOD, &method)' causes overwrite of conf.
In https://access.redhat.com/documentation/en-us/red_hat_directory_server/10/ht... is return of 'slapi_pblock_get(pb, SLAPI_BIND_METHOD, &method)' still 'int'
But in the source: https://pagure.io/389-ds-base/blob/master/f/ldap/servers/slapd/pblock.c#_157...
is used ber_tag_t
After I changed the declaration: ber_tag_t method;
Plugin started work. I need to deeply test it, but it looks good.
Ahhhhhh I suspect ber_tag_t is 8 bytes not 4 then. You might find that building your plugins with address sanitiser would help pick up this kind of thing. We use ASAN during development to detect exactly this kind of problem, and it's really helped us improve the quality of our code, so if you have plugins it could be well worth your time to get that working.
One of the awful parts about the problem you hit here is that slapi_pblock_get transforms everything to void * pointers, which means that in pblock.c, if the input pointer was the wrong size/width, we'll just blast it with out thought - exactly what happened to you here. The pblock has long been something I want to change in the project, but so much relies on it, it's hard to see it being changed to be safer.
As well, if your plugins are generic or could be broadly applicable, they could be contributed too, but that's up to you :)
I appreciate your kind way of helping me.
Any time mate, if you ever have questions please let us know!
Thanks a lot!
Jan Tomasek aka Semik http://www.tomasek.cz/
— Sincerely,
William Brown
Senior Software Engineer, 389 Directory Server SUSE Labs
At the minimum, you'll probably need to recompile the plugins against the newer versions/headers to eliminate some possible issues that could be occurring.
On 28 Aug 2020, at 20:32, Jan Tomasek jan@tomasek.cz wrote:
Hi,
I'm migrating 389DS from 1.2.11 to 1.4.0.11 on Debian Buster. I have two plug-ins which I would like to use with new server, they compile ok. But server crashes when they are about to be used.
What is actual documentation? I've found
https://access.redhat.com/documentation/en-us/red_hat_directory_server/10/ht...
But I'm not sure it this is latest for plugins. For server itself it is not, it speaks about obsoleted Admin Console.
Thanks
Jan Tomasek aka Semik http://www.tomasek.cz/
389-users mailing list -- 389-users@lists.fedoraproject.org To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject....
— Sincerely,
William Brown
Senior Software Engineer, 389 Directory Server SUSE Labs
389-users@lists.fedoraproject.org