Jump to content

User:BryanDavis/LDAP

From Wikitech

Notes on using LDAP.

Easy CLI queries

I have a shell alias for running ldapsearch which tells it to use paginated queries with a large page size, no prompting before fetching the next page, and no line wrapping: 

$ alias ldap='ldapsearch -xLLL -P 3 -E pr=5000/noprompt -o ldif-wrap=no -b"dc=wikimedia,dc=org"'
$ ldap uid=bd808 cn
dn: uid=bd808,ou=people,dc=wikimedia,dc=org
cn: BryanDavis

Decoding base64 entries

LDAP returns non-ascii values as base64 encoded strings. Developer accounts with unicode characters in their cn (wikitech username) values are a common example of this in our directory tree. Ldapsearch displays base64 values with two colons after the attribute name. I have a shell alias that ldapsearch results can be piped through which looks for these double colons and decodes the string. 

$ alias un64='awk '\''BEGIN{FS=":: ";c="base64 -d"}{if(/\w+:: /) {print $2 |& c; close(c,"to"); c |& getline $2; close(c); printf("%s:: \"%s\"\n", $1, $2); next} print $0 }'\'''
$ ldap uid=smccandlish displayName
dn: uid=smccandlish,ou=people,dc=wikimedia,dc=org
displayName:: 4oCUIDxmb250IGZhY2U9IlRyZWJ1Y2hldCBNUyI+JycnW1tVc2VyOlNNY0NhbmRsaXNofFNNY0NhbmRsaXNoXV0nJycgJm5ic3A7PHNwYW4gc3R5bGU9IndoaXRlLXNwYWNlOm5vd3JhcDsiPltbVXNlciB0YWxrOlNNY0NhbmRsaXNofFRhbGvih5JdXSDJljxzdXA+PGJpZz7iip08L2JpZz48L3N1cD7XmzxzdXA+4oqZPC9zdXA+w74gPC9zcGFuPiA8c21hbGw+W1tTcGVjaWFsOkNvbnRyaWJ1dGlvbnMvU01jQ2FuZGxpc2h8Q29udHJpYi5dXTwvc21hbGw+PC9mb250PiA=
$ ldap uid=smccandlish displayName | un64
dn: uid=smccandlish,ou=people,dc=wikimedia,dc=org
displayName:: "— <font face="Trebuchet MS">'''[[User:SMcCandlish|SMcCandlish]]''' &nbsp;<span style="white-space:nowrap;">[[User talk:SMcCandlish|Talk⇒]] ɖ<sup><big>⊝</big></sup>כ<sup>⊙</sup>þ </span> <small>[[Special:Contributions/SMcCandlish|Contrib.]]</small></font> "

Manual changes

ldap-modify.sh
#!/usr/bin/env bash
set -euxo pipefail
LDIF=${1:?Expected LDIF file to apply}

ldapmodify -v \
    -H 'ldap://ldap-rw.eqiad.wikimedia.org:389' \
    -D 'uid=novaadmin,ou=people,dc=wikimedia,dc=org' \
    -y .ldap-password \
    -f $LDIF

$ ldap-modify.sh $LDIF_FILE

One way to find the password for uid=novaadmin is in the /etc/mediawiki/WikitechPrivateSettings.php file on a cloudweb server.

Developer accounts in LDAP

I did an info dump on getting Developer account data out of the LDAP directory in phab:T386120. Rather than lose track of that write up in the depths of Phabricator I decided to republish some of it here.

Developer account information is stored in an LDAP directory maintained in the Wikimedia production network. More details on the service itself can be found in SRE/LDAP. The LDAP directory is used in Cloud VPS and Toolforge to provide Unix account information and ssh public keys to virtual machines. This is the same LDAP directory that backs the IDM (Bitu) and CAS-SSO (IDP) services.

Developer accounts are objectClass=posixAccount entities stored in the ou=people,dc=wikimedia,dc=org subtree. Cloud VPS project membership is recorded with groups named cn=project-$CLOUD_VPS_ID,ou=groups,dc=wikimedia,dc=org where $CLOUD_VPS_ID is the OpenStack project id which was the same as the OpenStack project name for years (for example "tools") but is now a UUID.

If you wanted to get the shell account name, legacy Wikitech username, account creation date, SUL account information, and email address of every Toolforge maintainer you could do something like this using my LDAP shell aliases: 

$ ldap '(&(objectClass=posixAccount)(memberOf=cn=project-tools,ou=groups,dc=wikimedia,dc=org))' uid cn createTimestamp wikimediaGlobalAccountId wikimediaGlobalAccountName mail | un64
(...snip...)

dn: uid=bd808,ou=people,dc=wikimedia,dc=org
uid: bd808
cn: BryanDavis
createTimestamp: 20130729163514Z
wikimediaGlobalAccountId: 12874
wikimediaGlobalAccountName: BryanDavis
mail: bdavis@wikimedia.org

(...snip...)

The un64 helper can be needed to decode non-ASCII cn values which are stored by LDAP in base-64 encoded strings.

The example above shows some of the core data for my Developer account:

  • dn: uid=bd808,ou=people,dc=wikimedia,dc=org -- the "dn" is the primary key for an LDAP record.
  • uid: bd808 -- "uid" in our environment is the account's shell name.
  • cn: BryanDavis -- "cn" in our environment is the account's "common name" which was also historically the user's Wikitech account name prior to October 2024.
  • createTimestamp: 20130729163514Z -- this Developer account was created 2013-07-29 16:35:51 UTC.
  • wikimediaGlobalAccountId: 12874 -- OAuth verified associated SUL account id.
  • wikimediaGlobalAccountName: BryanDavis -- OAuth verified associated SUL account username. Note that the username may have changed via a global account rename since being stored in LDAP. The SUL account id is invariant and can be used to find the current account name in the centralauth.globaluser database table.
  • mail: bdavis@wikimedia.org -- The Developer account's email address.

Command line tools can be nice for making quick lookups. The output format can be challenging to work with if you really want to create a CSV or TSV data file or to drive other automation. In that case I tend to write small Python scripts that use the ldap3 library to access the LDAP directory. Here is an example:

example.py
import ldap3
import yaml

cfg = yaml.safe_load(open("/etc/ldap.yaml")) # Toolforge bastions and Kubernetes containers have this file at runtime
conn = ldap3.Connection(cfg["servers"], auto_bind=True, read_only=True)
base = "ou=people,{basedn}".format(basedn=cfg["basedn"])
selector = "(&{})".format(
    "".join(
        [
            "(objectClass=posixAccount)",
            "(memberOf=cn=project-tools,ou=groups,dc=wikimedia,dc=org)",
        ]
    )
)

r = conn.extend.standard.paged_search(
    base,
    selector,
    attributes=[
        "uid",
        "cn",
        "createTimestamp",
        "wikimediaGlobalAccountId",
        "wikimediaGlobalAccountName",
        "mail",
    ],
    paged_size=256,
    time_limit=5,
    generator=True,
)
for user in r:
    # Do interesting stuff with the Developer account here
    print(user)

To run this script on Toolforge I would typically use some tool account I have access to (like https://toolsadmin.wikimedia.org/tools/id/bd808-test) and run things from inside a webservice python3.11 shell session: 

$ ssh login.toolforge.org
$ become $MY_TOOL_NAME
$ webservice python3.11 shell
tools.MY_TOOL_NAME@shell-1234567890:~$ python3 -m venv venv
tools.MY_TOOL_NAME@shell-1234567890:~$ ./venv/bin/pip3 install ldap3 pyyaml
tools.MY_TOOL_NAME@shell-1234567890:~$ vim example.py
  # Paste in the script
  # :wq
tools.MY_TOOL_NAME@shell-1234567890:~$ ./venv/bin/python3 example.py
Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=User:BryanDavis/LDAP&oldid=2271697"