Jump to content

Nova Resource:Project-proxy/SAL

From Wikitech

2024-09-18

  • 12:02 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.add_user_to_project (exit_code=0) for user 'aborrero' in role 'member'
  • 12:02 aborrero@cloudcumin1001: START - Cookbook wmcs.vps.add_user_to_project for user 'aborrero' in role 'member'

2024-06-25

  • 13:16 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server maps-proxy-03
  • 13:15 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server maps-proxy-03
  • 13:15 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.migrate_floating_ip (exit_code=0) for address 185.15.56.55 to server 'maps-proxy-04'
  • 13:15 taavi@cloudcumin1001: START - Cookbook wmcs.vps.migrate_floating_ip for address 185.15.56.55 to server 'maps-proxy-04'
  • 12:42 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server proxy-03
  • 12:40 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server proxy-03

2024-06-17

  • 10:28 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server project-proxy-puppetserver-1
  • 10:26 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server project-proxy-puppetserver-1
  • 10:26 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server project-proxy-acme-chief-02
  • 10:24 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server project-proxy-acme-chief-02
  • 10:23 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server maps-proxy-04
  • 10:22 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server maps-proxy-04
  • 10:18 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server proxy-04
  • 10:17 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server proxy-04

2023-11-24

  • 08:47 taavi: switch from project-proxy-acme-chief-01 (buster) to -02 (bookworm)

2023-10-16

  • 09:18 taavi: move main floating ip to a keepalived backed port T316982

2023-10-12

  • 11:41 taavi: configure keepalived ip for main project-proxy service T316982
  • 11:05 taavi: resize proxy-04 g3.cores2.ram4.disk20 to match proxy-03

2023-09-29

  • 07:53 wm-bot2: dcaro@urcuchillay END (FAIL) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=99)
  • 07:53 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.openstack.cloudvirt.vm_console

2023-09-21

  • 07:39 taavi: migrating storage from sqlite to mariadb T316982

2023-02-14

  • 08:18 taavi: remove proxies referring to maps-tiles1 to get the maps proxy back up

2022-10-13

  • 14:00 andrewbogott: added proxy-api security group to proxy0[34], opening the proxy API to the public for T319312

2022-09-14

  • 05:42 taavi: root@proxy-03:~# systemctl restart nginx # T316975

2022-09-09

  • 08:36 taavi: configure metricsinfra jobs to scrape prometheus nginx exporter

2022-09-08

  • 19:58 taavi: deployed some 'global' rate limiting rules

2022-09-03

  • 11:02 taavi: hard reboot proxy-03

2022-07-29

  • 15:55 taavi: updated neutron firewall rules to allow traffic to the api from 208.80.152.0/22 (all of eqiad + codfw) instead of individual per-cloud-host rules T314152

2022-06-01

  • 15:00 taavi: update warper.wmflabs.org mapping per irc request

2022-02-14

  • 21:49 andrewbogott: replacing maps-01 and maps-02 with maps-03 and maps-04 for bullseye upgrades. Active traffic is now on maps-03.
  • 21:22 andrewbogott: moving backups off of NFS and onto cinder with https://gerrit.wikimedia.org/r/c/operations/puppet/+/762509
  • 18:26 andrewbogott: deleting backups in /data/project/backup older than 400 days

2022-02-02

  • 17:05 arturo: add ws-export-test.wmcloud.org to XFF allowlist (af9e473) (T279111)

2022-01-22

  • 18:08 taavi: add maps proxy names requested in T299775
  • 11:16 taavi: add wma.wmcloud.org and *.wma.wmcloud.org to wma certificate SANs T299775

2022-01-21

  • 15:43 taavi: update maps-proxy mappings from maps-wma to maps-wma2 per request on T299585

2021-12-06

  • 08:33 majavah: deleting stretch proxies (proxy-01 and -02)

2021-11-18

  • 13:40 majavah: [codfw1dev] testing keystone authentication patches

2021-11-17

  • 19:33 majavah: [codfw1dev] replace proxy-01 instance with proxy-02 running bullseye

2021-11-11

  • 13:21 majavah: failing over to proxy-03 T295235

2021-11-09

  • 10:52 majavah: create new pair of proxies on bullseye T295235

2021-11-07

  • 08:45 majavah: increase quota by 4 cpu and 8G ram T295245

2021-10-07

  • 07:35 arturo: manually rebase labs/private.git, it had merge conflicts

2021-09-02

  • 01:24 bstorm: rebooting maps-proxy-02 because it is unresponsive

2021-05-06

  • 09:34 arturo: systemctl restart acme-chief (to workaround a bug)
  • 09:24 arturo: rebase & resolve merge conflicts in labs/private.git

2021-04-15

  • 09:15 arturo: refresh hiera XFF entry for ws-export.wmcloud.org T279111

2021-04-14

2020-09-08

  • 10:05 arturo: remove /var/lib/acme-chief/certs/* to force acme-chief generating new certs instead of renewing them (T262237)
  • 09:35 arturo: refresh some hiera settings in the `project-proxy-acme-chief` puppet tab
  • 09:23 arturo: cleanup old apt sources.list entries referencing mitaka-jessie that prevents clean package upgrades in proxy-01
  • 09:18 arturo: upgrading acme-chief deb package from 0.25-1 to 0.28-1 on project-proxy-acme-chief-01 (T262237)
  • 09:18 arturo: upgrading acme-chief deb package from 0.25-1 to 0.28-1 on project-proxy-acme-chief-01 (T26223
  • 09:10 arturo: restart acme-chief service in roject-proxy-acme-chief-01 (T262237)

2020-07-06

  • 14:44 andrewbogott: switching proxy-01 and proxy-02 to use an acme-chief cert for *.wmflabs.org and *.wmcloud.org via "profile::wmcs::novaproxy::acme_certname: wmflabs"
  • 14:38 andrewbogott: moving proxy-01 and proxy-02 to the project-local puppetmaster via the hiera setting "puppetmaster: project-proxy-puppetmaster-01.project-proxy.eqiad.wmflabs"

2020-06-16

  • 21:33 bstorm_: setting "profile::wmcs::nfsclient::mode: soft" on the project T102369 T127559

2020-04-15

2020-03-25

2019-11-07

  • 09:53 arturo: replacing SSL cert for star.wmflabs.org - for real this time (T237066)

2019-11-06

  • 09:57 arturo: replacing SSL cert for star.wmflabs.org (T237066)

2019-07-24

  • 10:15 arturo: reallocating proxy-02 from cloudvirt1027 to cloudvirt1028 (T227539)

2019-06-05

  • 22:50 Krenair: Added cloudcontrol1004 IP to match cloudcontrol1003 rule in 'proxy' security group rules for port 5668 T225168
  • 22:44 Krenair: Updating 'proxy' security group rules for port 5668 to remove decommissioned IP - 208.80.154.147 californium T189921
  • 22:36 Krenair: Updating 'proxy' security group rules for port 5668 to remove decommissioned IPs - 208.80.154.136 silver, 208.80.155.117 labs-ns0, 208.80.152.32 virt0 (!), 208.80.153.48 labtestservices2001, 208.80.154.92 labcontrol1001
  • 22:14 Krenair: Per jeh's investigation, added cloudservices1004 IP to match cloudservices1003 rule in 'proxy' security group rules for port 5668

2019-03-12

  • 03:46 bd808: Restarted uwsgi-invisible-unicorn on proxy-01
  • 03:21 bd808: Removed redis sets with no record in the backing database (T133554)
  • 03:00 bd808: Fixed domains with trailing dot (T218064)
  • 01:13 bd808: Deleted dangling backend records in /etc/dynamicproxy-api/data.db (T218064)

2019-01-29

  • 17:21 arturo: add myself as projectadmin to be able to debug some stuff

2019-01-23

  • 19:41 andrewbogott: deleting old eqiad-region proxy nodes novaproxy-01 and novaproxy-02

2019-01-16

  • 14:21 andrewbogott: stopping old VPS proxies in eqiad — T213540

2018-12-21

  • 14:59 bstorm_: Added 172.16.0.0/21 to the one-off port listed by those proxy hosts' security group

2018-11-27

  • 20:14 andrewbogott: moving traffic back to novaproxy-01
  • 19:57 andrewbogott: moving novaproxy-01 to labvirt1001
  • 19:55 andrewbogott: moving traffic to novaproxy-02

2018-11-02

  • 18:22 arturo: T206223 certificate renewal for *.wmflabs.org by manual copy of private key to /etc/ for nginx

2018-10-23

  • 13:23 gtirloni: Added gtirloni to the project

2018-06-06

  • 17:56 andrewbogott: switching primary proxy to novaproxy-01
  • 17:41 andrewbogott: switching primary proxy to novaproxy-02

2018-03-20

  • 21:42 chasemp: novaproxy-01:/var/log# rm -f *\.gz
  • 21:42 chasemp: novaproxy-01:/var# aptitude clean

2018-02-12

  • 22:07 andrewbogott: opened up port 5668 to labweb1001 and 1002 in the 'proxy' security group

2018-02-09

  • 01:05 bd808: Removed inactive users Ryan Lane and Coren
  • 01:04 bd808: Removed Yuvipanda at user request (T186289)

2017-11-16

  • 15:19 chasemp: disable puppet

2017-08-31

  • 21:12 madhuvishy: Updated star.wmflabs.org cert, ran puppet and restarted nginx in novaproxy-01 and 02 (Copied the private key manually to /etc/ssl/private) T174611

2017-08-03

  • 14:48 chasemp: remove openstack::clientlib from novaproxy-02 it is not on novaproxy-01. no one is sure why it is there atm. hopefully this doesn't explode.

2017-07-13

  • 17:15 bd808: Upgraded nginx-common

2017-06-21

  • 17:05 andrewbogott: moving novaproxy ip back to nova-proxy-01
  • 16:59 andrewbogott: moving the active proxy IP to novaproxy-02

2017-06-05

2017-04-21

  • 05:37 bd808: Added BryanDavis (self) as admin

2017-03-20

  • 21:21 andrewbogott: switching primary back to novaproxy-01
  • 21:12 andrewbogott: migrating novaproxy-01 to labvirt1010
  • 21:11 andrewbogott: switching primary proxy to novaproxy-02

2016-10-24

  • 21:43 yuvipanda: move public floating IP back to novaproxy-01

2016-10-20

  • 22:09 yuvipanda: failover novaproxy to novaproxy-02, upgrade kernel to 4.4, reboot

2015-10-05

  • 20:15 yuvipanda: delete dynamicproxy-gateway

2015-09-29

  • 22:12 yuvipanda: deleting dynamicproxy-01 and -02, starting afresh with jessie

June 19

  • 22:16 YuviPanda: restarted dynamicproxy-api

May 7

  • 23:56 yuvipanda: restarted dynamicproxy API again
  • 22:25 yuvipanda: restarted dynamicproxy-gateawy

April 23

  • 20:23 YuviPanda: restarted dynamicproxy-api

March 30

  • 23:58 YuviPanda: rebooted dynamicproxy-api on dynamicproxy-gateway

March 3

  • 11:21 YuviPanda: restarted dynamicproxy-api on dynamicproxy-gateway

February 23

  • 17:03 andrewbogott: restarted dynamicproxy-api

October 30

  • 19:30 mutante: restarting nginx on dynamicproxy-gateway - disabled SSLv3

October 8

  • 16:17 YuviPanda: restarted dynamicproxy-api on dynamicproxy-gateway

July 16

July 9

  • 19:31 YuviPanda: added scfc_de as projectadmin

May 10

  • 14:49 YuviPanda: Upgraded nginx to 1.7.0 to get SPDY/3.1 on dynamicproxy-gatewaya

April 17

  • 15:12 andrewbogott: rebooting dynamic proxy to move logging to a bigger partition

April 16

  • 14:29 andrewbogott: removed some users who may not have signed an NDA

April 9

  • 14:03 Coren: replaces SSL certificate for the general labs webproxy

April 8

  • 05:33 Ryan_Lane: restart nginx on affected nodes
  • 05:07 Ryan_Lane: upgraded libssl on all nodes

April 16