Nova Resource:Video/Help/video2commons

Toolforge tools

video2commons
Website	toolforge:video2commons
Description	Transfers video and audio from external sites to Commons
Keywords	video, transfer
Author(s)	Zhuyifei1999talk
Maintainer(s)	Zhuyifei1999 (View all)
Source code	GitHub
License	GNU General Public License 3.0 or later
Issues	GitHub

{"name":"video2commons","repository":"GitHub","author":"Zhuyifei1999","url":"toolforge:video2commons","keywords":"video, transfer","description":"Transfers video and audio from external sites to Commons"}

This tool transfers video and audio from external sites to Commons. Web frontend is on Nova resource:Tools, and backend is on Nova resource:Video. Code is hosted on GitHub.

Web frontend

Webfrontend is on Toolforge under tool video2commons.

Setup

Ensure you're in your tool account; if not, $ become video2commons

Clone repo

Somehow, clone video2commons to the tool's home dir. Some methods can do this:

• Delete everything and clone it (be careful): $ rm -rf * && git clone https://github.com/toolforge/video2commons.git .
• Or, clone to a tmp dir and copy git configuration back: $ git clone https://github.com/toolforge/video2commons.git tmp && mv tmp/.git . && rm -rf tmp && git reset --hard

Setup environment

Inside a shell of the tool's k8s container (launch with $ webservice --backend=kubernetes python3.11 shell): Setup virtualenv:

$ python3 -m venv $HOME/www/python/venv
$ # echo source ~/www/python/venv/bin/activate >> .bash_profile
$ source ~/www/python/venv/bin/activate
$ pip install --upgrade pip wheel

Install necessary components:

• $ pip install -r video2commons/backend/requirements.txt
  $ pip install -r video2commons/frontend/requirements.txt
  

• Anything else?

Create a config.json similar to config.json.example, and fill the necessary configs. Make sure the file is chmodded 600 before writing it!

Start the webservice

Start the frontend (outside of the container):

$ toolforge webservice --backend=kubernetes python3.11 start

Updating

Run $ git pull. No local commits should exist.

• If any python code is changed, uwsgi must be restarted: $ webservice --backend=kubernetes python2 restart (and a downtime under a minute might be caused)
• Otherwise, nothing must be done.

Backend

Backends are celery workers on video cluster encoding instances.

Setup

Ensure Nova Resource:Video/Help#Setup is done.

Use puppet to configure the rest:

1. Get the puppet configuration: $ wget https://github.com/toolforge/video2commons/raw/master/puppet/backend.pp
2. Ensure the first few lines of the file is correct. And, just in case, make sure nothing in the code is evil.
3. Apply them: $ sudo puppet apply backend.pp --debug

Switching off

Run once (and only once. DO NOT run a second time unless something is already terribly broken!):

sudo su -
. /srv/v2c/venv/bin/activate
celery -A video2commons.backend.worker --workdir=/srv/v2c multi stop --pidfile="/var/run/v2ccelery/%N.pid" 2
exit

This will put celery to gracefully shutdown (i.e. it will stop handling new jobs, stop all idle workers, and wait for each worker to finish its job). Running a second time WILL KILL all the workers.