Skip to content

add paging to ldap search #22

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

add paging to ldap search #22

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ed026bb
add paging to ldap search
TheoAndersen Mar 1, 2018
59ef5e7
add ldap protocol version as CLI option
TheoAndersen Mar 7, 2018
ffbcee3
fix missing p in ldap
TheoAndersen Apr 24, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 29 additions & 8 deletions sync_ldap_groups_to_svn_authz.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,8 @@ def bind():
"""This function will bind to the LDAP instance and return an ldapobject."""

ldapobject = ldap.initialize(url)

ldapobject.protocol_version = ldap.VERSION3 #paging results only apply to LDAP v3
Copy link
Owner

@whitlockjc whitlockjc Mar 5, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to turn this into a CLI option? If you can manipulate these options as you have without affecting other AD versions or LDAP servers, it wouldn't matter but I'm not sure.

Copy link
Author

@TheoAndersen TheoAndersen Mar 7, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want, sure i can make the ldap protocol version, into a cli option (with 3 being the default)

Copy link

@troelsarvin troelsarvin Mar 7, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LDAP 3 has existed since 1997, and LDAP 2 officially died in 2003. That's a while back... So I assert it's not worth the added complexity to add an LDAP version parameter.

ldapobject.set_option(ldap.OPT_REFERRALS, 0) #pagiing result only works without referrals enabled
ldapobject.bind_s(bind_dn, bind_password)

if verbose:
Expand All @@ -132,6 +133,7 @@ def search_for_groups(ldapobject):

groups = []
result_set = get_ldap_search_resultset(base_dn, group_query, ldapobject)


if (len(result_set) == 0):
if not silent:
Expand Down Expand Up @@ -179,16 +181,35 @@ def get_groups(ldapobject):

def get_ldap_search_resultset(base_dn, group_query, ldapobject, scope=ldap.SCOPE_SUBTREE):
"""This function will return a query result set."""
cookie = ''
page_size = 1000
criticality = False
result_set = []
result_id = ldapobject.search(base_dn, scope, group_query)

while 1:
result_type, result_data = ldapobject.result(result_id, 0)
if (result_type == ldap.RES_SEARCH_ENTRY):
result_set.append(result_data)
elif (result_type == ldap.RES_SEARCH_RESULT):
break
first_pass = True
pg_ctrl = ldap.controls.SimplePagedResultsControl(criticality, page_size, cookie)

#run though <page_size> results at a time, to avoid exceeding ldap size limit
while first_pass or pg_ctrl.cookie:
first_pass = False
try:
msgid = ldapobject.search_ext(base_dn, scope, group_query, serverctrls=[pg_ctrl])
except:
sys.stderr.write("ERROR: \n" + str(sys.exc_info()[0]))
sys.exit(1)

while True:
result_data = None
result_type, result_data, msgid, serverctrls = ldapobject.result3(msgid, all=0)

if serverctrls is not None and len(serverctrls) > 0:
pg_ctrl.cookie = serverctrls[0].cookie

if (result_type == ldap.RES_SEARCH_ENTRY):
result_set.append(result_data)
elif (result_type == ldap.RES_SEARCH_RESULT):
break

return result_set

# get_ldap_search_resultset()
Expand Down