User:Jhedden/notes/cloudvirt-nic
Cloudvirt dual NIC configuration
eno1
The eno1 network interface is configured with an address on the cloud-hosts1-b-<site> VLAN and subnet. This interface is used for everything except virtual machine traffic.
The configuration for this interface is defined in /etc/network/interfaces.
# The primary network interface
allow-hotplug eno1
iface eno1 inet static
address 10.192.20.13/24
gateway 10.192.20.1
# dns-* options are implemented by the resolvconf package, if installed
dns-nameservers 10.3.0.1
dns-search codfw.wmnet
pre-up /sbin/ip token set ::10:192:20:13 dev eno1
up ip addr add 2620:0:860:118:10:192:20:13/64 dev eno1
eno2
The eno2 network interface is setup to use the cloud-instances2-b-<site> VLAN, but is not associated with any subnet. This interface is only used by Neutron to bridge network communication between virtual machines and external networks.
The configuration for this interface is defined in /etc/network/interfaces.
auto eno2.2105
iface eno2.2105 inet manual
up ip link set $IFACE up
down ip link set $IFACE down
The systemd service neutron-linuxbridge-agent will create a linux bridge attached to the eno2 network interface. You can view the linux bridge with the brctl command.
cloudvirt2002-dev:~$ sudo brctl show
bridge name bridge id STP enabled interfaces
brq05a5494a-18 8000.d09466936da1 no eno2.2105
When a virtual machine is running you'll see the tap interfaces for each virtual machine also attached to the same bridge.
cloudvirt2002-dev:~$ sudo brctl show
bridge name bridge id STP enabled interfaces
brq05a5494a-18 8000.d09466936da1 no eno2.2105
tap0c8ccf84-bc
tap7729745f-c0
Cloudvirt single NIC configuration
In a single NIC configuration the physical port on the switch for eno1 is reconfigured from an access port to a trunk port. This allows the eno1 network interface to communicate through the physical switch on different VLANs.
The cloud-hosts1-b-<site> VLAN should be set as the native VLAN and cloud-instances2-b-<site> as a member of the trunk group. Phab task information on switch config
Steps to change from a dual NIC to a single NIC configuration:
- Evacuate all virtual machines from the cloudvirt
- Update network interface mappings in hiera example patch
- Request the switch port to be reconfigured example
- Verify network connectivity to the cloudvirt (the primary interface on eno1 does not require any changes)
- Update /etc/network/interfaces swapping eno2.2105 with eno1.2105
- Run the puppet agent to update Neutron with the latest hiera data
- Reboot the host
OPTIONAL: If you choose not to reboot the host you can run the following commands instead:
# Stop the neutron linux bridge agent
$ sudo systemctl stop neutron-linuxbridge-agent
# Detach the eno2 interface from the bridge
$ sudo brctl delif br-internal eno2.2105
# Shutdown and delete the eno2.2105 device
$ sudo ip link set dev eno2.2105 down
$ sudo ip link delete eno2.2105
# Restart networking
$ sudo systemctl restart networking
# Start the neutron linux bridge agent
$ sudo systemctl start neutron-linuxbridge-agent
Configuration after reconfiguring the cloudvirt
cloudvirt2001-dev:~$ ip link show eno1
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether d0:94:66:93:97:77 brd ff:ff:ff:ff:ff:ff
cloudvirt2001-dev:~$ ip link show eno1.2105
57: eno1.2105@eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master brq05a5494a-18 state UP mode DEFAULT group default qlen 1000
link/ether d0:94:66:93:97:77 brd ff:ff:ff:ff:ff:ff
cloudvirt2001-dev:~$ sudo brctl show
bridge name bridge id STP enabled interfaces
brq05a5494a-18 8000.d09466939777 no eno1.2105