url-downloader
url-downloader is a Squid non-caching proxy for HTTP requests. It was originally created for MediaWiki's upload-by-URL functionality, these days it is being used by any service that wants to access resources outside the boundaries of the wikimedia production realm. It is not a highly available service and as such no assumptions should be made in code regarding availability.
Service
- See also url_downloader in Puppet.
- For eqiad: Hosted on
urldownloader100[34].wikimedia.org
- For codfw: Hosted on
urldownloader200[34].wikimedia.org
In both datacenters only one url-downloader is active at any time, and is pointed at via the DNS record for url-downloader.$DC.wikimedia.org
, and exposed at url-downloader.$DC.wikimedia.org:8080
.
url-downloader.wikimedia.org:8080
points to eqiad for backward-compatibility.
In MediaWiki configuration, it would be typically obtained via $wmgLocalServices['urldownloader']
(which also provides an appropriate proxy URL on beta).
See also
- HTTP proxy (another set of squid proxies for slightly different use cases)
- T254011: Why do we have 2 sets of squid proxies?