Nova Resource:Gitlab-runners/SAL

15:41 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_project_to_ovs (exit_code=0)
15:27 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_project_to_ovs
15:26 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server gitlab-runners-puppetserver-01
15:25 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server gitlab-runners-puppetserver-01

20:53 andrewbogott: replaced gitlab-runners-puppetmaster-01 with gitlab-runners-puppetserver-01 and moved secrets to /srv/git/labs/private. Shut down gitlab-runners-puppetmaster-01 and it can be deleted once projectadmins are convinced that all is well.

21:55 mutante: root@runner-1030:/var/lib/docker/volumes# rm -rf runner-m4mqfjvt-project-860-concurrent-2-cache-c33bcaa1fd2c77edfc3893b41966cea8 ; rm -rf runner-m4mqfjvt-project-860-concurrent-3-cache-c33bcaa1fd2c77edfc3893b41966cea8 (T333586)
21:52 mutante: root@runner-1030:/var/lib/docker/volumes# rm -rf runner-m4mqfjvt-project-1177-concurrent-2-cache-c33bcaa1fd2c77edfc3893b41966cea8 (T333586)

17:02 mutante: pausing runner-1014, then will remove it and create new bullseye runner runner 1025 to replace it
16:34 mutante: pausing runner-1013, then will remove it and create new bullseye runner to replace it

20:29 mutante: - create new instance runner-1023, wait for intial puppet run, then rm -rf /var/lib/puppet/ssl ; run puppet; sign new request on gitlab-runners-puppetmaster-01.gitlab-runners (normal procedure for fresh instance with local puppetmaster)
20:20 mutante: deleting instance runner-1012, creating instance runner-1023 to replace it
19:06 mutante: deleting instance runner-1011, creating instance runner-1022 to replace it
18:25 mutante: pausing runner-1011 in gitlab UI from accepting new jobs, then deleting instance in Horizon UI to replace it with another bullseye instance T297659

22:03 mutante: - deleting instance runner-1020 and recreating it with the same name but flavor g3.cores8.ram24.disk20 T297659
22:01 mutante: - deleting instance runner-1008 in Horizon and also deleting it in gitlab admin UI about the same time T297659
20:59 mutante: - pausing runner-1008 from accepting new jobs, hoping it will finish all existing jobs already queued and once that is down to 0 I can replace it with a new runner on bullseye (T297659)

23:52 mutante: creating instance runner-1020 with bullseye for testing purposes, unfortunately not enough quota to just use the same as all other runners, could only do small flavor T297659

09:23 arturo: bump quotas to instances 12 cores 82 ram 249856 gigabytes 600 volumes 10 (T293832)