Help talk:Toolforge/Web/Archives/2020

Please do not post any new comments on this page. This is a discussion archive See current discussion or the archives index.

Location of static files for uWSGI apps?

Under "Python (uWSGI)", subsection "Default uwsgi configuration", it says, Static files are located in $HOME/www/python/src/static/, but that sees to not be the case. I'm not sure what the intent was, but I got it to work by adding a symlink from $HOME/www/static -> $HOME/www/python/src/static/. RoySmith (talk) 02:05, 17 January 2020 (UTC)

Files in the $HOME/www/static directory are available to be served by https://tools-static.wmflabs.org as documented at Help:Toolforge/Web#Static_file_server. Uwsgi is capable of serving static files, but I can not find any evidence in the history of the current webservice command that the $HOME/www/python/src/static/ directory mentioned in Help:Toolforge/Web#Default_uwsgi_configuration was ever configured for static asset service. I dug through the history of the older webservice2 in the ops/puppet.git repository and did not find uwsgi configuration for static files there either. The item was added to the docs in Special:Diff/1837297 by MichaelSchoenitzer, maybe they can help explain? --BryanDavis (talk) 02:52, 17 January 2020 (UTC)

I've got the tools-static.wmflabs.org mechanism working, but I'd like to avoid that if possible. Having the static files served up under a different host name from the rest of the app complicates things (i.e. constructing urls). At the moment, I'm going with having my own uwsgi.ini:

[uwsgi]
check-static = /data/project/spi-tools-dev/www/static

but it would be nice to avoid that bit of configuration. RoySmith (talk) 03:07, 17 January 2020 (UTC)

The $HOME/www/python/uwsgi.ini settings you mention seem like the right solution. If we did add a check-static mount to the default config, I think that having it be under the $HOME/www/python/src directory would be a bad idea. Something like $HOME/www/python/static could be a reasonable setting, but honestly dealing with static files is so application or framework specific it seems like setting a global default would be unlikely to help many people. As an example, the location that MichaelSchoenitzer documented is completely correct for the Flask framework and would not require any uWSGI configuration. For the Django framework the location could be correct, but would require config inside the Django app. --BryanDavis (talk) 04:23, 17 January 2020 (UTC)

I reverted the misleading edit. --BryanDavis (talk) 04:30, 17 January 2020 (UTC)

Thanks; eliminating that cleared up a lot of confusion. I agree that the default uWSGI config should be framework agnostic. My current thought is that rather than adding my own uwsgi.ini, I'm going to do this entirely inside django. You can get the tool name by parsing __file__ inside of settings.py, and from there it's easy to build the tools-static.wmflabs.org url, and a trivial context processor can make that available to every template. That's a little bit of work, but not as bad as I originally thought it would be. This has the advantage of keeping all the configuration inside the source tree. RoySmith (talk) 04:43, 17 January 2020 (UTC)

A working example

I've got this all sorted out now. It turns out to be pretty simple, once you see how it's done. The gist is:

1. Parse the tool-name out of the path to your django settings file.
2. Use that to build a STATIC_URL for tools-static.wmflabs.org.
3. Add django.template.context_processors.static to TEMPLATES['OPTIONS']['context_processors'].

You can now do something like:

<link rel="stylesheet" href="{{ STATIC_URL }}/whatever.css">

in any template. No need for mucking with uwsgi.ini files. Here's a diff for reference. This works Python 3.7 and Django-2.2. If you're using an older Python, change the f-string substitution to %-formatting, or whatever. RoySmith (talk) 18:25, 17 January 2020 (UTC)

Python (uWSGI/k8s), egg files and binary dependencies

I'll leave this here in case anyone finds it helpful. I had to deal with a Python egg which requires an additional .so dependency (distributed separately) in order to link on runtime. Section "Installing numpy / scipy / things with binary dependencies" covers Python wheels, but sadly it's not so straightforward with old-style .egg files ("ImportError: _library_name_.so: cannot open shared object file: No such file or directory").

At first, I tried to tweak the LD_LIBARY_PATH env var. Turns out k8s pods ignore anything set in the .bashrc file (excuse my lack of expertise, I'm sure this seems pretty obvious to an experienced user). Also, applying this workaround on venv/bin/activate didn't help. My Python app finally managed to see this variable via os.environ after I added env = LD_LIBRARY_PATH=/path/to/dir to www/python/uwsgi.ini, but still no luck on solving that ImportError.

In my case, hardcoding dlopen = /path/to/lib.so into www/python/uwsgi.ini did the trick:

[uwsgi]
dlopen = /path/to/lib.so

See uWSGI docs. Peter Bowman (talk) 00:32, 31 May 2020 (UTC)