Skip to content

Commit

Permalink
Use new naming convention for VMs and cluster
Browse files Browse the repository at this point in the history
  • Loading branch information
@khanh-ph
khanh-ph committed Sep 26, 2023
1 parent f73d82f commit 7926c76
Show file tree
Hide file tree
Showing 6 changed files with 97 additions and 67 deletions.
28 changes: 15 additions & 13 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -72,13 +72,14 @@ The project provides several Terraform variables that allow you to customize the
| <a name="input_pm_timeout"></a> [pm\_timeout](#input\_pm\_timeout) | Timeout value (seconds) for proxmox API calls. | `number` | `600` | no |
| <a name="input_env_name"></a> [env\_name](#input\_env\_name) | The stage of the development lifecycle for the k8s cluster. Example: `prod`, `dev`, `qa`, `stage`, `test` | `string` | `"test"` | no |
| <a name="input_location"></a> [location](#input\_location) | The city or region where the cluster is provisioned | `string` | `null` | no |
| <a name="input_cluster_number"></a> [cluster\_number](#input\_cluster\_number) | The instance count for the k8s cluster, to differentiate it from other clusters. Example: `00`, `01` | `string` | `"00"` | no |
| <a name="input_cluster_number"></a> [cluster\_number](#input\_cluster\_number) | The instance count for the k8s cluster, to differentiate it from other clusters. Example: `00`, `01` | `string` | `"01"` | no |
| <a name="input_cluster_domain"></a> [cluster\_domain](#input\_cluster\_domain) | The cluster domain name | `string` | `"local"` | no |
| <a name="input_internal_net_name"></a> [internal\_net\_name](#input\_internal\_net\_name) | Name of the internal network bridge | `string` | `"vmbr1"` | no |
| <a name="input_internal_net_subnet_cidr"></a> [internal\_net\_subnet\_cidr](#input\_internal\_net\_subnet\_cidr) | CIDR of the internal network | `string` | `"10.0.1.0/24"` | no |
| <a name="input_ssh_private_key"></a> [ssh\_private\_key](#input\_ssh\_private\_key) | SSH private key in base64, will be used by Terraform client to connect to the VM after provisioning | `string` | n/a | yes |
| <a name="input_ssh_public_keys"></a> [ssh\_public\_keys](#input\_ssh\_public\_keys) | SSH public keys in base64 | `string` | n/a | yes |
| <a name="input_vm_user"></a> [vm\_user](#input\_vm\_user) | n/a | `string` | `"ubuntu"` | no |
| <a name="input_vm_sockets"></a> [vm\_sockets](#input\_vm\_sockets) | n/a | `number` | `1` | no |
| <a name="input_vm_user"></a> [vm\_user](#input\_vm\_user) | The default user for all VMs | `string` | `"ubuntu"` | no |
| <a name="input_vm_sockets"></a> [vm\_sockets](#input\_vm\_sockets) | Number of the CPU socket to allocate to the VMs | `number` | `1` | no |
| <a name="input_vm_max_vcpus"></a> [vm\_max\_vcpus](#input\_vm\_max\_vcpus) | The maximum CPU cores available per CPU socket to allocate to the VM | `number` | `2` | no |
| <a name="input_vm_cpu_type"></a> [vm\_cpu\_type](#input\_vm\_cpu\_type) | The type of CPU to emulate in the Guest | `string` | `"host"` | no |
| <a name="input_vm_os_disk_storage"></a> [vm\_os\_disk\_storage](#input\_vm\_os\_disk\_storage) | Default storage pool where OS VM disk is placed | `string` | n/a | yes |
Expand All @@ -87,21 +88,22 @@ The project provides several Terraform variables that allow you to customize the
| <a name="input_worker_node_data_disk_size"></a> [worker\_node\_data\_disk\_size](#input\_worker\_node\_data\_disk\_size) | The size of worker node data disk in Gigabyte | `string` | `10` | no |
| <a name="input_vm_ubuntu_tmpl_name"></a> [vm\_ubuntu\_tmpl\_name](#input\_vm\_ubuntu\_tmpl\_name) | Name of Cloud-init template Ubuntu VM | `string` | `"ubuntu-2204"` | no |
| <a name="input_bastion_ssh_ip"></a> [bastion\_ssh\_ip](#input\_bastion\_ssh\_ip) | IP of the bastion host, could be either public IP or local network IP of the bastion host | `string` | `""` | no |
| <a name="input_bastion_ssh_user"></a> [bastion\_ssh\_user](#input\_bastion\_ssh\_user) | n/a | `string` | `"ubuntu"` | no |
| <a name="input_bastion_ssh_port"></a> [bastion\_ssh\_port](#input\_bastion\_ssh\_port) | n/a | `number` | `22` | no |
| <a name="input_bastion_ssh_user"></a> [bastion\_ssh\_user](#input\_bastion\_ssh\_user) | The user to authenticate to the bastion host | `string` | `"ubuntu"` | no |
| <a name="input_bastion_ssh_port"></a> [bastion\_ssh\_port](#input\_bastion\_ssh\_port) | The SSH port number on the bastion host | `number` | `22` | no |
| <a name="input_create_kubespray_host"></a> [create\_kubespray\_host](#input\_create\_kubespray\_host) | n/a | `bool` | `true` | no |
| <a name="input_kubespray_image"></a> [kubespray\_image](#input\_kubespray\_image) | n/a | `string` | `"khanhphhub/kubespray:v2.22.0"` | no |
| <a name="input_kube_version"></a> [kube\_version](#input\_kube\_version) | Kubernetes version | `string` | `"v1.24.6"` | no |
| <a name="input_kube_network_plugin"></a> [kube\_network\_plugin](#input\_kube\_network\_plugin) | Choose network plugin (cilium, calico, kube-ovn, weave or flannel. Use cni for generic cni plugin) | `string` | `"calico"` | no |
| <a name="input_enable_nodelocaldns"></a> [enable\_nodelocaldns](#input\_enable\_nodelocaldns) | Enable nodelocal dns cache | `bool` | `false` | no |
| <a name="input_podsecuritypolicy_enabled"></a> [podsecuritypolicy\_enabled](#input\_podsecuritypolicy\_enabled) | pod security policy (RBAC must be enabled either by having 'RBAC' in authorization\_modes or kubeadm enabled) | `bool` | `false` | no |
| <a name="input_persistent_volumes_enabled"></a> [persistent\_volumes\_enabled](#input\_persistent\_volumes\_enabled) | Add Persistent Volumes Storage Class for corresponding cloud provider (supported: in-tree OpenStack, Cinder CSI, AWS EBS CSI, Azure Disk CSI, GCP Persistent Disk CSI) | `bool` | `false` | no |
| <a name="input_helm_enabled"></a> [helm\_enabled](#input\_helm\_enabled) | Helm deployment | `bool` | `false` | no |
| <a name="input_ingress_nginx_enabled"></a> [ingress\_nginx\_enabled](#input\_ingress\_nginx\_enabled) | Nginx ingress controller deployment | `bool` | `false` | no |
| <a name="input_argocd_enabled"></a> [argocd\_enabled](#input\_argocd\_enabled) | ArgoCD | `bool` | `false` | no |
| <a name="input_argocd_version"></a> [argocd\_version](#input\_argocd\_version) | ArgoCD version | `string` | `"v2.4.12"` | no |
| <a name="input_kube_network_plugin"></a> [kube\_network\_plugin](#input\_kube\_network\_plugin) | The network plugin to be installed on your cluster. Example: `cilium`, `calico`, `kube-ovn`, `weave` or `flannel` | `string` | `"calico"` | no |
| <a name="input_enable_nodelocaldns"></a> [enable\_nodelocaldns](#input\_enable\_nodelocaldns) | A boolean value that indicates whether to enable nodelocal dns cache on your cluster | `bool` | `false` | no |
| <a name="input_podsecuritypolicy_enabled"></a> [podsecuritypolicy\_enabled](#input\_podsecuritypolicy\_enabled) | A boolean value that indicates whether to enable pod security policy on your cluster (RBAC must be enabled either by having 'RBAC' in authorization\_modes or kubeadm enabled) | `bool` | `false` | no |
| <a name="input_persistent_volumes_enabled"></a> [persistent\_volumes\_enabled](#input\_persistent\_volumes\_enabled) | A boolean value that indicates whether to add Persistent Volumes Storage Class for corresponding cloud provider (supported: in-tree OpenStack, Cinder CSI, AWS EBS CSI, Azure Disk CSI, GCP Persistent Disk CSI) | `bool` | `false` | no |
| <a name="input_helm_enabled"></a> [helm\_enabled](#input\_helm\_enabled) | A boolean value that indicates whether to enable Helm on your cluster | `bool` | `false` | no |
| <a name="input_ingress_nginx_enabled"></a> [ingress\_nginx\_enabled](#input\_ingress\_nginx\_enabled) | A boolean value that indicates whether to enable Nginx ingress on your cluster | `bool` | `false` | no |
| <a name="input_argocd_enabled"></a> [argocd\_enabled](#input\_argocd\_enabled) | A boolean value that indicates whether to enable ArgoCD on your cluster | `bool` | `false` | no |
| <a name="input_argocd_version"></a> [argocd\_version](#input\_argocd\_version) | The ArgoCD version to be installed | `string` | `"v2.4.12"` | no |
| <a name="input_vm_k8s_control_plane"></a> [vm\_k8s\_control\_plane](#input\_vm\_k8s\_control\_plane) | Control Plane VM specification | `object({ node_count = number, vcpus = number, memory = number, disk_size = number })` | <pre>{<br> "disk_size": 20,<br> "memory": 1536,<br> "node_count": 1,<br> "vcpus": 2<br>}</pre> | no |
| <a name="input_vm_k8s_worker"></a> [vm\_k8s\_worker](#input\_vm\_k8s\_worker) | Worker VM specification | `object({ node_count = number, vcpus = number, memory = number, disk_size = number })` | <pre>{<br> "disk_size": 20,<br> "memory": 2048,<br> "node_count": 2,<br> "vcpus": 2<br>}</pre> | no |
| <a name="input_use_legacy_naming_convention"></a> [use\_legacy\_naming\_convention](#input\_use\_legacy\_naming\_convention) | A boolean value that indicates whether to use legacy naming convention for the VM and cluster name. If your cluster was provisioned using version <= 3.x, set it to `true` | `bool` | `false` | no |
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
## Blog posts
Expand Down
2 changes: 1 addition & 1 deletion modules/proxmox_ubuntu_vm/main.tf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ resource "proxmox_vm_qemu" "ubuntu_vm" {
target_node = var.pm_host
clone = var.vm_ubuntu_tmpl_name
qemu_os = "l26"
name = "${var.vm_name_prefix}-${format("%02d", count.index)}"
name = var.use_legacy_naming_convention ? "${var.vm_name_prefix}-${format("%02d", count.index)}" : "${var.vm_name_prefix}-${format("%02d", count.index + 1)}"
agent = 1
onboot = var.vm_onboot
os_type = "cloud-init"
Expand Down
5 changes: 5 additions & 0 deletions modules/proxmox_ubuntu_vm/variables.tf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,11 @@ variable "worker_node_data_disk_size" {
default = 10
}

variable "use_legacy_naming_convention" {
type = bool
default = false
}

#
# Local vars
#
Expand Down
54 changes: 37 additions & 17 deletions variables.tf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -54,11 +54,18 @@ variable "location" {
variable "cluster_number" {
type = string
description = "The instance count for the k8s cluster, to differentiate it from other clusters. Example: `00`, `01`"
default = "00"
default = "01"
}

variable "cluster_domain" {
type = string
description = "The cluster domain name"
default = "local"
}

locals {
cluster_name = var.location != null ? "k8s-${var.env_name}-${var.location}-${var.cluster_number}" : "k8s-${var.env_name}-${var.cluster_number}"
cluster_fqdn = "${local.cluster_name}.${var.cluster_domain}"
}

#
Expand Down Expand Up @@ -88,13 +95,15 @@ variable "ssh_public_keys" {
}

variable "vm_user" {
type = string
default = "ubuntu"
type = string
description = "The default user for all VMs"
default = "ubuntu"
}

variable "vm_sockets" {
type = number
default = 1
type = number
description = "Number of the CPU socket to allocate to the VMs"
default = 1
}

variable "vm_max_vcpus" {
Expand Down Expand Up @@ -149,13 +158,15 @@ variable "bastion_ssh_ip" {
}

variable "bastion_ssh_user" {
type = string
default = "ubuntu"
type = string
description = "The user to authenticate to the bastion host"
default = "ubuntu"
}

variable "bastion_ssh_port" {
type = number
default = 22
type = number
description = "The SSH port number on the bastion host"
default = 22
}

#
Expand All @@ -179,43 +190,43 @@ variable "kube_version" {
}
variable "kube_network_plugin" {
type = string
description = "Choose network plugin (cilium, calico, kube-ovn, weave or flannel. Use cni for generic cni plugin)"
description = "The network plugin to be installed on your cluster. Example: `cilium`, `calico`, `kube-ovn`, `weave` or `flannel`"
default = "calico"
}

variable "enable_nodelocaldns" {
type = bool
description = "Enable nodelocal dns cache"
description = "A boolean value that indicates whether to enable nodelocal dns cache on your cluster"
default = false
}
variable "podsecuritypolicy_enabled" {
type = bool
description = "pod security policy (RBAC must be enabled either by having 'RBAC' in authorization_modes or kubeadm enabled)"
description = "A boolean value that indicates whether to enable pod security policy on your cluster (RBAC must be enabled either by having 'RBAC' in authorization_modes or kubeadm enabled)"
default = false
}
variable "persistent_volumes_enabled" {
type = bool
description = "Add Persistent Volumes Storage Class for corresponding cloud provider (supported: in-tree OpenStack, Cinder CSI, AWS EBS CSI, Azure Disk CSI, GCP Persistent Disk CSI)"
description = "A boolean value that indicates whether to add Persistent Volumes Storage Class for corresponding cloud provider (supported: in-tree OpenStack, Cinder CSI, AWS EBS CSI, Azure Disk CSI, GCP Persistent Disk CSI)"
default = false
}
variable "helm_enabled" {
type = bool
description = "Helm deployment"
description = "A boolean value that indicates whether to enable Helm on your cluster"
default = false
}
variable "ingress_nginx_enabled" {
type = bool
description = "Nginx ingress controller deployment"
description = "A boolean value that indicates whether to enable Nginx ingress on your cluster"
default = false
}
variable "argocd_enabled" {
type = bool
description = "ArgoCD"
description = "A boolean value that indicates whether to enable ArgoCD on your cluster"
default = false
}
variable "argocd_version" {
type = string
description = "ArgoCD version"
description = "The ArgoCD version to be installed"
default = "v2.4.12"
}

Expand All @@ -232,4 +243,13 @@ variable "vm_k8s_worker" {
type = object({ node_count = number, vcpus = number, memory = number, disk_size = number })
description = "Worker VM specification"
default = { node_count = 2, vcpus = 2, memory = 2048, disk_size = 20 }
}

#
# Others
#
variable "use_legacy_naming_convention" {
type = bool
description = "A boolean value that indicates whether to use legacy naming convention for the VM and cluster name. If your cluster was provisioned using version <= 3.x, set it to `true`"
default = false
}
38 changes: 20 additions & 18 deletions vm-k8s-nodes.tf
View file
Original file line number Diff line number Diff line change
@@ -1,23 +1,24 @@
module "k8s_control_plane_nodes" {
source = "./modules/proxmox_ubuntu_vm"

node_count = var.vm_k8s_control_plane["node_count"]
pm_host = var.pm_host
vm_ubuntu_tmpl_name = var.vm_ubuntu_tmpl_name
vm_name_prefix = "${var.env_name}-k8s-cplane"
vm_max_vcpus = var.vm_max_vcpus
vm_vcpus = var.vm_k8s_control_plane["vcpus"]
vm_sockets = var.vm_sockets
vm_cpu_type = var.vm_cpu_type
vm_memory_mb = var.vm_k8s_control_plane["memory"]
vm_os_disk_storage = var.vm_os_disk_storage
vm_os_disk_size_gb = var.vm_k8s_control_plane["disk_size"]
vm_net_name = var.internal_net_name
vm_net_subnet_cidr = var.internal_net_subnet_cidr
vm_host_number = 10
vm_user = var.vm_user
vm_tags = "${var.env_name};terraform;k8s-control-plane"
ssh_public_keys = var.ssh_public_keys
node_count = var.vm_k8s_control_plane["node_count"]
pm_host = var.pm_host
vm_ubuntu_tmpl_name = var.vm_ubuntu_tmpl_name
vm_name_prefix = var.use_legacy_naming_convention ? "${var.env_name}-k8s-cplane" : "vm-${local.cluster_name}-cp"
vm_max_vcpus = var.vm_max_vcpus
vm_vcpus = var.vm_k8s_control_plane["vcpus"]
vm_sockets = var.vm_sockets
vm_cpu_type = var.vm_cpu_type
vm_memory_mb = var.vm_k8s_control_plane["memory"]
vm_os_disk_storage = var.vm_os_disk_storage
vm_os_disk_size_gb = var.vm_k8s_control_plane["disk_size"]
vm_net_name = var.internal_net_name
vm_net_subnet_cidr = var.internal_net_subnet_cidr
vm_host_number = 10
vm_user = var.vm_user
vm_tags = "${var.env_name};terraform;k8s-control-plane"
ssh_public_keys = var.ssh_public_keys
use_legacy_naming_convention = var.use_legacy_naming_convention
}

module "k8s_worker_nodes" {
Expand All @@ -26,7 +27,7 @@ module "k8s_worker_nodes" {
node_count = var.vm_k8s_worker["node_count"]
pm_host = var.pm_host
vm_ubuntu_tmpl_name = var.vm_ubuntu_tmpl_name
vm_name_prefix = "${var.env_name}-k8s-worker"
vm_name_prefix = var.use_legacy_naming_convention ? "${var.env_name}-k8s-worker" : "vm-${local.cluster_name}-worker"
vm_max_vcpus = var.vm_max_vcpus
vm_vcpus = var.vm_k8s_worker["vcpus"]
vm_sockets = var.vm_sockets
Expand All @@ -43,6 +44,7 @@ module "k8s_worker_nodes" {
add_worker_node_data_disk = var.add_worker_node_data_disk
worker_node_data_disk_storage = var.worker_node_data_disk_storage
worker_node_data_disk_size = var.worker_node_data_disk_size
use_legacy_naming_convention = var.use_legacy_naming_convention
}

output "k8s_control_plane" {
Expand Down
37 changes: 19 additions & 18 deletions vm-kubespray-host.tf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ locals {
{
kube_version = var.kube_version
kube_network_plugin = var.kube_network_plugin
cluster_name = local.cluster_name
cluster_name = local.cluster_fqdn
enable_nodelocaldns = var.enable_nodelocaldns
podsecuritypolicy_enabled = var.podsecuritypolicy_enabled
persistent_volumes_enabled = var.persistent_volumes_enabled
Expand All @@ -53,23 +53,24 @@ locals {
module "kubespray_host" {
source = "./modules/proxmox_ubuntu_vm"

node_count = var.create_kubespray_host ? 1 : 0
pm_host = var.pm_host
vm_ubuntu_tmpl_name = var.vm_ubuntu_tmpl_name
vm_name_prefix = "${var.env_name}-kubespray"
vm_max_vcpus = var.vm_max_vcpus
vm_vcpus = 2
vm_sockets = var.vm_sockets
vm_cpu_type = var.vm_cpu_type
vm_memory_mb = 2048
vm_os_disk_storage = var.vm_os_disk_storage
vm_os_disk_size_gb = 10
vm_net_name = var.internal_net_name
vm_net_subnet_cidr = var.internal_net_subnet_cidr
vm_host_number = 5
vm_user = var.vm_user
vm_tags = "${var.env_name};terraform;kubespray"
ssh_public_keys = var.ssh_public_keys
node_count = var.create_kubespray_host ? 1 : 0
pm_host = var.pm_host
vm_ubuntu_tmpl_name = var.vm_ubuntu_tmpl_name
vm_name_prefix = var.use_legacy_naming_convention ? "${var.env_name}-kubespray" : "vm-${local.cluster_name}-kubespray"
vm_max_vcpus = var.vm_max_vcpus
vm_vcpus = 2
vm_sockets = var.vm_sockets
vm_cpu_type = var.vm_cpu_type
vm_memory_mb = 2048
vm_os_disk_storage = var.vm_os_disk_storage
vm_os_disk_size_gb = 10
vm_net_name = var.internal_net_name
vm_net_subnet_cidr = var.internal_net_subnet_cidr
vm_host_number = 5
vm_user = var.vm_user
vm_tags = "${var.env_name};terraform;kubespray"
ssh_public_keys = var.ssh_public_keys
use_legacy_naming_convention = var.use_legacy_naming_convention
}

resource "null_resource" "setup_kubespray" {
Expand Down

0 comments on commit 7926c76

Please sign in to comment.