Help:Cloud VPS Instances
This page contains information on how to set up a Cloud VPS instance.
What is a Cloud VPS instance?
An instance is a virtual machine (VM). Every instance belongs to a Cloud VPS project.
Difference between a Cloud VPS project and an instance
- A Cloud VPS project can be defined as the group of users and resources (like instances, security groups, floating IPs, Puppet groups, etc.) associated with a piece of software hosted on Cloud VPS. A project is a security concept. It's a group of users, a subset of which are given extra permissions as defined by the role: project member (formerly "projectadmin"). Cloud VPS (and Toolforge) projects are meant to reflect real-world endeavors, like "tools" or "bots".
- An instance is just one resource that a Cloud VPS project might use. An instance is a virtual machine. Every instance belongs to a project. We are using EC2/OpenStack terminology here. When creating a new instance, the user can decide how much memory and storage space the virtual machine will have. See Help:Instances for more details.
Why create a Cloud VPS instance?
Cloud VPS is a flexible option for those who need it. It is equivalent to having your own server(s).
Before requesting a Cloud VPS instance, we recommend you consider using Toolforge instead. Toolforge has many of the same features as Cloud VPS, and Toolforge users do not have to manage the full virtual environment. Toolforge users also have access to toolsdb, wiki replicas, and a number of other services.
See Help:Cloud Services Introduction to help decide which service is right for you.
Who can create a Cloud VPS instance?
Project members (formerly called project admins). You must be a Project member in a project to create and manage its instances.
What are the responsibilities of project members?
Project members must subscribe to the cloud-announce and cloud mailing lists and follow the maintenance or communication actions requested by the WMCS Admins. Failure to respond may result in unexpected loss of data or service interruptions.
Project members should delete unused instances and notify other members when a project is no longer active. See Cloud VPS instance lifecycle for more details.
Requesting Project member rights
To create instances, individuals must have project member rights on the specific project they want to have instances created for. The following people can grant Project member rights:
- Project members. Project members can grant project member rights on projects they belong to. You can find Project members listed on the individual pages for the projects.
- Cloud VPS admins. Cloud admins, including Wikimedia Cloud Services staff and trusted community volunteers, can manage membership and Project member status in any project.
Contacting project members and admins
Project members and members of the Cloud admin group can be contacted through the 'email a user' function or through their talk page on Wikitech. Please note: Many members of the Wikitech community are volunteers who may not check their email or talk pages regularly. You may need to check with more than one person before you receive a response.
Project access rights
Learn more about Cloud services user roles and rights.
Before you create a Cloud VPS instance
- Be a Project member in the project you intend to create an instance for.
- Make sure your project has an appropriate security group for the instance's networking needs.
- Be aware of your project's quotas (usage limits).
- Understand the Cloud VPS instance lifecycle.
Create a Cloud VPS instance
Working with Cloud VPS Instances
- Log in to https://horizon.wikimedia.org/.
- Switch to the Project where you will host your VM. This option is available at the top-left of your Cloud VPC portal.
- The interface should change and you should be able to view, launch and delete Instances. This implies that you have project member privileges.The highlighted column at the top left of the image above, provides you an option to select a project scope you can provision your machine in. You can be in one or multiple projects within openstack.
The steps below will guide you through setting up a VM on this interface:
- Fill out the form as presented below. Ignore the Availability zone option and leave Count as 1. Then click Next.
- Select an Operating System Image from the Available section.
- Do this by clicking the up-arrow icon to move the image into the Allocated section, and click Next.
- Select the Flavor. The flavor influences how much RAM, CPU and disk your VM gets. You are limited to a few predetermined combinations here. Click Next.
- Leave the Security Groups section as default. Click Next.
- Leave Server Groups as default.
- Clicking the Launch Instance button launches your new instance in the Spawning state, as seen in the instance overview.
- Click on an instance’s name to view its details. The output should be similar to the image below:
To configure this instance with a puppet role for a specific job, click Puppet Configuration. You can apply puppet classes and, optionally, Hiera config values:
Puppet Configuration for Cloud VPS instances
- Cloud VPS uses Puppet to manage instance configurations.
- A list of puppet classes can be found on doc.wikimedia.org.
- If you are a sudo user for the project, you can run
sudo puppet agent -tvor wait until puppet runs on your instance.
- When the puppet run finishes, you can log into it with SSH.
Logging into VMs with SSH
- Generate an SSH Key pair on your local machine
- Add the public key to your developer account in SSH key section on https://idm.wikimedia.org
- Then you can SSH into the instance
$ ssh -J <shell-name>@bastion.wmcloud.org <shell-name>@<instance>.<your-project>.eqiad1.wikimedia.cloud
where,
shell-name is the name you picked when signing up for you developer account.
instance is the instance name you picked when creating the VM.
your-project is the name of the project you are building under
Note: Copy and paste only the alpha-numeric texts generated by your SSH agent in the text box provided on your developer account.
Instance information
| Information type | Action | |
|---|---|---|
| Instance Details | The name, zone, and count of your instance | The name will also be your hostname. Name your instance using the following format project-instance, e.g, the project name is myproject; the instance is coolbot; the instance name is myproject-coolbot
|
| Instance Source | The version of the operating system | (Debian Stretch, Debian Buster, etc). Also known as a base image. |
| Instance Flavor | Contains a range of possible instance types indicating RAM, number of CPUs, and Hard Drive space. | Your most likely choice will be at the top of the list of system images available. See current flavors in following chart. This decision is not permanent, you can allocate more resources later by using the red Action "Resize Instance". |
| Security Groups | Contains a list of security groups | At minimum, the default security group should be selected.* |
| Server Groups | Contains a list of server groups | Server groups allow you to define logical groups of instances with advanced scheduling features. Note: avoid using the affinity server group policy. See the linked help page for why.
|
* If the Project members for the project have not created any additional security groups, besides the default group, you will only see the default security group listed.
Instance Sizes and Flavors
| Name | VCPUS | RAM | ROOT DISK | PUBLIC |
|---|---|---|---|---|
| g4.cores1.ram2.disk20 | 1 | 2 GB | 20 GB | Yes |
| g4.cores2.ram4.disk20 | 2 | 4 GB | 20 GB | Yes |
| g4.cores4.ram8.disk20 | 4 | 8 GB | 20 GB | Yes |
| g4.cores8.ram16.disk20 | 8 | 16 GB | 20 GB | Yes |
| g4.cores16.ram16.disk20 | 16 | 16 GB | 20 GB | Yes |
* Current as of June 2024
Increase quotas for projects
Quotas are the usage limits for projects. Quotas refer to one or more of CPU, RAM, disk storage, number of VM instances and/or floating IPs. The default quota for a new project allows up to 8 instances.
Projects can have their quota increased following the instructions on the "Cloud-VPS (Quota-requests)" Phabricator task.
The Cloud VPS Instance lifecycle
Instances are allocated resources (storage, RAM, CPU, power). Over time, these resources need to be used or reclaimed. WMCS admins periodically check projects and instances to ensure they are being used by active projects. Instances will be removed for projects that have been determined inactive.
Process for removing Cloud VPS instances
- Notice will be sent to cloud via cloud-announce at least 2 weeks before any expected action is taken. Emergencies may require more immediate intervention.
- If possible, instances will be suspended or shut down for at least 2 weeks (a total of a month) before further unsolicited actions are taken to allow the tenant to notice.
- Inactive projects (those without instances or any responsive users) will be removed (along with their quota)
Backups of Cloud VPS instances
No backups will be kept by Wikimedia Cloud Services.
Local backups
You can backup your server by copying the files you are interested to save on your local machine. Of course, this is not an automated process and you will need to store the data you are saving. Also, be mindful that backups may be very big and so you will be transferring a lot of data, which among other things will take a long time.
If you want to backup your root disk on your VPS server to a local directory named backup you can do the following:
rsync -aXvz --stats --info=progress2 --rsync-path="sudo /usr/bin/rsync" --exclude-from=./excludes.txt <server>:/ backup/`
The paths listed in the file excludes.txt will be ignored. This gist contains a common list of excluded paths that you usually do not need to backup since they are temporary files, cache, backups and other system files.
Communication and support
We communicate and provide support through several primary channels. Please reach out with questions and to join the conversation.
| Connect | Best for | |
|---|---|---|
| Phabricator Workboard | #Cloud-Services | Task tracking and bug reporting |
| IRC Channel | #wikimedia-cloud connect | General discussion and support |
| Mailing List | cloud@ | Information about ongoing initiatives, general discussion and support |
| Announcement emails | cloud-announce@ | Information about critical changes (all messages mirrored to cloud@) |
| News wiki page | News | Information about major near-term plans |
| Blog | Clouds & Unicorns | Learning more details about some of our work |