Jump to content

Search Platform/Weekly Updates/2023-09-15

From Wikitech

Summary

We now track all the indicators we need for Search SLO. Next step, we will create dashboards.

The work on Search Update Pipeline needs to address a few remaining features, in particular dealing with late events and deduplication of events is a bit more complex than expected. We're moving forward on testing the deployment.

Performance improvements on Improve multilingual zero-results rate is ongoing.

We've finally removed the use of "whitelist" in the Wikidata Query Service code base, a good step in using inclusive language in our work.

What we've accomplished

Search Update Pipeline (SUP)

  • We're dropping support for Java 8 in the SUP. This will enable us to simplify the code a little bit (dropping compatibility libraries that we are currently using) and allow us to use newer features of Java. On the other hand, this will prevent us for deploying SUP on Yarn.
  • We are being hit by a bug in Flink (https://issues.apache.org/jira/browse/FLINK-28758) that prevents us from properly testing the deployment of Flink with Zookeeper. This is raising the priority of migrating the WDQS Streaming Updater from older Kafka / Flink integration to newer APIs - https://phabricator.wikimedia.org/T326914

Improve multilingual zero-results rate

Search SLOs

Operations

Misc