Search Platform/Weekly Updates/2024-05-03

Summary

As the Search Update Pipeline work is ramping down, we're are onboarding more team members to work on the WDQS Graph Split.

Improve multilingual zero-results rate is going to be on pause until July, while one of our team member is on sabbatical.

We had to deal with the follow up of an incident with completion indices, root cause was tracked to a network issue.

What we've accomplished

Improve multilingual zero-results rate

• Analysis of normalization of Arabic family languages, write up on https://www.mediawiki.org/wiki/User:TJones_(WMF)/Notes/Normalization_for_Arabic_Script_Across_Languages - https://phabricator.wikimedia.org/T72899

WDQS graph splitting

• Work started on automating the data reload process - https://phabricator.wikimedia.org/T349069
• Onboarding a new team member on the project

Operations

• Incident follow up on missing documents in the search completion indices in codfw, root cause seem to be a network issue (https://phabricator.wikimedia.org/T363516#9748908), but we should also increase the reliability of the indices update in face of errors - https://phabricator.wikimedia.org/T363516
• Late events in the Search update pipeline - https://phabricator.wikimedia.org/T359580