15:59 dhinus: zone transfer "16.172.in-addr.arpa." from noauth-project to cloudinfra T380495
15:21 dhinus: zone transfer analytics.db.svc.eqiad.wmflabs. from noauth-project to cloudinfra T380493
15:21 dhinus: zone transfer web.db.svc.eqiad.wmflabs. from noauth-project to cloudinfra T380493
15:21 dhinus: zone transfer svc.eqiad.wmflabs. from noauth-project to cloudinfra T380490
14:22 dhinus: zone transfer eqiad.wmflabs. from noauth-project to cloudinfra T380488
2025-03-17
10:19 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_user_from_project (exit_code=0) for user 'rook'
10:19 aborrero@cloudcumin1001: START - Cookbook wmcs.vps.remove_user_from_project for user 'rook'
2024-11-25
12:28 dhinus: migrated designate zone db.svc.eqiad.wmflabs. from noauth-project to cloudinfra T380491
2024-11-18
16:59 dhinus: puppetserver ca clean --certname wikilabels-staging-02.wikilabels.eqiad.wmflabs (project does not exist anymore)
16:59 dhinus: puppetserver ca clean --certname wikilabels-backups-01.wikilabels.eqiad.wmflabs (project does not exist anymore)
2024-11-11
16:06 dhinus: revoke expired cert for mwv-builder-03.mediawiki-vagrant.eqiad.wmflabs (puppetserver ca clean from cloudinfra-cloudvps-puppetserver-1)
15:59 dhinus: revoke expired cert for deployment-poolcounter06.deployment-prep.eqiad.wmflabs (puppetserver ca clean from cloudinfra-cloudvps-puppetserver-1)
2024-10-22
09:05 arturo: restart puppetserver service in T377803
2024-09-18
11:47 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.add_user_to_project (exit_code=0) for user 'aborrero' in role 'member'
11:47 aborrero@cloudcumin1001: START - Cookbook wmcs.vps.add_user_to_project for user 'aborrero' in role 'member'
2024-07-17
15:50 bd808: Added jhathaway as a project member to support PCC work
15:26 bd808: Running /usr/local/sbin/puppet-facts-upload on cloudinfra-cloudvps-puppetserver-1.cloudinfra.eqiad1.wikimedia.cloud per request by jhathaway
2024-06-24
19:35 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_project_to_ovs (exit_code=0)
15:09 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance cloud-puppetmaster-05
15:09 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance cloud-puppetmaster-05
2024-04-04
14:12 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance cloudinfra-acme-chief-01
14:12 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance cloudinfra-acme-chief-01
14:11 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance cloudinfra-internal-puppetmaster-02
14:11 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance cloudinfra-internal-puppetmaster-02
2024-04-03
11:30 wmbot~taavi@runko: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance mx-out03
11:29 wmbot~taavi@runko: START - Cookbook wmcs.vps.remove_instance for instance mx-out03
11:28 wmbot~taavi@runko: END (FAIL) - Cookbook wmcs.vps.remove_instance (exit_code=99) for instance mx-out03
11:28 wmbot~taavi@runko: START - Cookbook wmcs.vps.remove_instance for instance mx-out03
11:28 wmbot~taavi@runko: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance mx-out04
11:28 wmbot~taavi@runko: START - Cookbook wmcs.vps.remove_instance for instance mx-out04
11:28 wmbot~taavi@runko: END (FAIL) - Cookbook wmcs.vps.remove_instance (exit_code=99) for instance mx-out03
11:28 wmbot~taavi@runko: START - Cookbook wmcs.vps.remove_instance for instance mx-out03
11:27 wmbot~taavi@runko: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance mx-out04
11:27 wmbot~taavi@runko: START - Cookbook wmcs.vps.remove_instance for instance mx-out04
2024-03-27
12:23 taavi: hard reboot cloudinfra-cloudvps-puppetserver-1
2024-03-25
18:34 andrewbogott: moved mx records to point to service names (mx-out-a.wmcloud.org and mx-out-b.wmcloud.org) and assoviated those service names with new mx servers mx-out05 and mx-out06
2024-02-21
14:08 taavi: update cloudinfra-idp-1 to java 17 to fix CAS class loading errors on idp.wmcloud.org T357749
2023-12-01
09:52 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_user_from_project (exit_code=0) for user 'jbond' (T352508)
09:52 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_user_from_project for user 'jbond' (T352508)
14:07 wm-bot2: Increased quotas by 1 instances - cookbook ran by dcaro@vulcanus
2022-07-20
07:03 dcaro: manually running fsck through costole on puppetmaster-03 do to disk errors (T313380)
06:54 dcaro: rebooting puppetmaster-03 do to disk errors
2022-02-16
14:44 taavi: delete old cloudinfra-db instances (cloudinfra-db01 cloudinfra-db02)
12:10 taavi: enable gtid (master_use_gtid=current_pos) on cloudinfra-db04
2022-02-13
17:14 taavi: encapi db switchover: cloudinfra-db01 -> cloudinfra-db03
11:38 taavi: generate new profile::mariadb::grants::cloudinfra::repl_pass, previous one seems to have been lost in the 2020-06-04 labs/private incident
10:48 taavi: shutdown and delete ntp-01 and ntp-02 running stretch, replaced by -03 and -04
2022-02-11
15:41 taavi: switch floating ip 185.15.56.3 from ntp-01 to ntp-03
15:18 taavi: switch floating ip 185.15.56.27 from ntp-02 to ntp-04
10:29 arturo: re-arm keyholder in cloud-cumin-02 (password in pwstore)
10:13 arturo: re-arm keyholder in cloud-cumin-01
2019-11-09
18:30 Krenair: Drop internal-puppetmaster old cherry-pick of https://gerrit.wikimedia.org/r/c/operations/puppet/+/545567 which was preventing rebase and blocking cloud-puppetmaster machines from getting puppet commits themselves - new version of that change will have appeared there in the rebase
2019-10-23
09:55 arturo: manually restart mariadb in cloudinfra-db02 to fix replication
09:41 arturo: the cloudinfra-db01 VM was rebooted bc the hypervisor rebooted
09:41 arturo: manually start mariadb in cloudinfra-db01
2019-10-17
16:29 jeh: cleanup old fullstackd puppet certs
12:52 arturo: add phamhi as user and projectadmin
2019-09-09
18:23 Krenair: Started mariadb service on cloudinfra-db02 - was not running for some reason
2019-04-08
14:17 andrewbogott: moved cloudinfra-internal-puppetmaster01 to cloudvirt1026
2019-04-02
20:07 andrewbogott: moving cloudinfra-db02 to cloudvirt1017
2019-03-26
17:43 andrewbogott: removing "profile::ldap::client::labs::restricted_to: ops" from project puppet to allow volunteer admin access
17:21 andrewbogott: adding Krenair as projectadmin as per T218448
2019-01-07
21:08 gtirloni: upgraded kernel and rebooted mx-out{01,02}
