Tuesday, July 4, 2017

Kubernetes Notes (1): Allocatable Resources

1. What is allocatable resource?

The capacity property in Kubernetes node status represents the total mount of resources (such as CPU, memory) in a node. Some node resources may be reserved for Kubernetes components (Kube-Reserved), and some more may be reserved for other components (System-Reserved). They are allocatable resources when excluding reserved resources from the resource capacity, and it's the allocatable property of node status (https://github.com/kubernetes/kubernetes/blob/release-1.2/docs/proposals/node-allocatable.md).

The more pods deployed onto a node, the less resources available to accommodate containers. However, the allocatable resources reported by the command "kubectl describe node" keeps the same no matter how many pods have been deployed onto the node. It's count intuitive at the first glance. 
2. An issue about allocatable resource
The command "kubectl describe node" always reports 0 on capacity and allocatable CPU and memory of Windows nodes in Kubernetes 1.6.
Output of "kubectl describe node"

If Kubernetes fails to deploy Windows containers, it might be caused by the issue of allocatable property. When Kubernetes selects nodes to deploy containers, it checks whether the nodes have enough resources available, as shown in the following code (https://github.com/kubernetes/kubernetes/blob/master/plugin/pkg/scheduler/algorithm/predicates/predicates.go):

allocatable := nodeInfo.AllocatableResource()
if allocatable.MilliCPU < podRequest.MilliCPU+nodeInfo.RequestedResource().MilliCPU {
    predicateFails = append(predicateFails, NewInsufficientResourceError(v1.ResourceCPU, podRequest.MilliCPU, nodeInfo.RequestedResource().MilliCPU, allocatable.MilliCPU))
}
if allocatable.Memory < podRequest.Memory+nodeInfo.RequestedResource().Memory {
    predicateFails = append(predicateFails, NewInsufficientResourceError(v1.ResourceMemory, podRequest.Memory, nodeInfo.RequestedResource().Memory, allocatable.Memory))
}

In the code above, the variable allocatable is the allocatable property of node status, podRequest is the requested resource of the pod to be deployed (aggregated requested resources of all containers in the pod), and nodeInfo.RequestResource is the total resource of all pods already deployed onto the node. The node becomes a candidate to deploy new pods only when there is enough resource available.

Since the allocatable property of Windows nodes is always 0, the if expression in the code above gets false and Kubernetes fails to find nodes to deploy pods if containers request resources explicitly.

2 comments:

  1. Hi harry, i have the same problem. the resource allocation property for a windows node is always 0. so i can not deploy my pod.
    i have tried to modify the node config with "kubectl edit node xxx", e.g. set cpu to "1" and memory to "100M", but the changes will not be saved.

    do you know how to solve this problem? thank you very much.

    best regards,
    martin

    ReplyDelete
  2. Hi Harry, i have opened a ticket in github.there i have described my problems.

    https://github.com/kubernetes/kubernetes/issues/56637

    on linux, it is possible to set "kube-reserved", "system-reserved" cpu/memory, but on windows, i do not think it is possible to use these two options.

    i am curious about how the resource allocation is done for windows server 2016 node.

    do you have any idea?

    Thank you very much in advance.

    Martin

    ReplyDelete

Azure Container Instance (4): Integrating with Logic App

Azure Logic Apps are very useful tools to automate workflows and business processes. Azure Container Instances have been integrated with...