Search Platform/Weekly Updates/2023-09-08

Summary

We are close to complete the feature work on the Search Update Pipeline. We will shift our focus on deployment, which is unlikely to be completed this quarter (at risk).

The latest batch of improvements to zero-result rate seem to be working well, but have a significant impact on the indexing time. We need some work to optimize this.

What we've accomplished

Search Update Pipeline

• Ongoing work on deployment helm charts
• routing writes to various elasticsearch clusters
• integration of Flink with Zookeeper
• Decision made on how to handle deduplication of events

Improve multilingual zero-results rate

• Ongoing work on performance optimization of the new analysis chains. The current improvements in analysis do improve the functionality, but it comes at a fairly high cost during re-indexing. We need to find the right balance here.

Finalize Search SLOs

• Patch for SLO collection from the SearchVue extension is blocked on review: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/SearchVue/+/948166

Misc

• Reimport the image suggestions dataset https://phabricator.wikimedia.org/T345545