virtualkubelet

package
v0.0.0-...-75253b4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 16, 2025 License: Apache-2.0 Imports: 50 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	PodPhaseInitialize = "Initializing"
	PodPhaseCompleted  = "Completed"
)
View Source 
const (
	DefaultCPUCapacity    = "100"
	DefaultMemoryCapacity = "3000G"
	DefaultPodCapacity    = "10000"
	DefaultGPUCapacity    = "0"
	DefaultFPGACapacity   = "0"
	DefaultListenPort     = 10250
	NamespaceKey          = "namespace"
	NameKey               = "name"
	CREATE                = 0
	DELETE                = 1

	DefaultProtocol        = "TCP"
	DefaultWstunnelCommand = "" /* 245-byte string literal not displayed */
)

Variables

View Source 
var (
	KubeletVersion = "test"
)

Functions

func AddSessionContext 

func AddSessionContext(req *http.Request, sessionContext string)

func CheckIfAnnotationExists 

func CheckIfAnnotationExists(pod *v1.Pod, key string) bool

CheckIfAnnotationExists checks if a specific annotation (key) is available between the annotation of a pod

func GetResources 

func GetResources(config Config) v1.ResourceList

func GetSessionContextMessage 

func GetSessionContextMessage(sessionContext string) string

func LogRetrieval 

func LogRetrieval(
	ctx context.Context,
	config Config,
	logsRequest types.LogStruct,
	clientHTTPTransport *http.Transport,
	sessionContext string,
) (io.ReadCloser, error)

LogRetrieval performs a REST call to the InterLink API when the user ask for a log retrieval. Compared to create/delete/status request, a way smaller struct is marshalled and sent. This struct only includes a minimum data set needed to identify the job/container to get the logs from. Returns the call response and/or the first encountered error

func NodeCondition 

func NodeCondition(ready bool) []v1.NodeCondition
func NodeConditionWithInterlink(ready bool, interlinkStatus v1.ConditionStatus, interlinkReason, interlinkMessage string) []v1.NodeCondition
func PingInterLink(ctx context.Context, config Config) (bool, int, string, error)

PingInterLink pings the InterLink API and returns true if there's an answer. The second return value is given by the answer provided by the API. The third return value contains the response body from the ping call.

func PodPhase 

func PodPhase(_ Provider, phase string, podIP string) (v1.PodStatus, error)

func RemoteExecution 

func RemoteExecution(ctx context.Context, config Config, p *Provider, pod *v1.Pod, mode int8) error

RemoteExecution is called by the VK everytime a Pod is being registered or deleted to/from the VK. Depending on the mode (CREATE/DELETE), it performs different actions, making different REST calls. Note: for the CREATE mode, the function gets stuck up to 5 minutes waiting for every missing ConfigMap/Secret. If after 5m they are not still available, the function errors out

func SetDefaultResource 

func SetDefaultResource(config *Config)

func TracerUpdate 

func TracerUpdate(ctx *context.Context, name string, pod *v1.Pod)

Types

type Accelerator 

type Accelerator struct {
	// ResourceType specifies the type of accelerator (e.g., "nvidia.com/gpu", "xilinx.com/fpga")
	ResourceType string `yaml:"ResourceType"`
	// Model specifies the specific model or variant of the accelerator
	Model string `yaml:"Model"`
	// Available indicates how many units of this accelerator are available
	Available int `yaml:"Available"`
}

Accelerator represents a hardware accelerator (GPU, FPGA, etc.) available on the node.

type Config 

type Config struct {
	// InterlinkURL is the URL for connecting to the interLink API
	InterlinkURL string `yaml:"InterlinkURL"`
	// InterlinkPort specifies the port for the interLink API (for http/https)
	InterlinkPort string `yaml:"InterlinkPort"`
	// KubernetesAPIAddr is the Kubernetes API server address
	KubernetesAPIAddr string `yaml:"KubernetesApiAddr"`
	// KubernetesAPIPort specifies the Kubernetes API server port
	KubernetesAPIPort string `yaml:"KubernetesApiPort"`
	// KubernetesAPICaCrt is the CA certificate for Kubernetes API server verification
	KubernetesAPICaCrt string `yaml:"KubernetesApiCaCrt"`
	// DisableProjectedVolumes disables handling of Kubernetes projected volumes
	DisableProjectedVolumes bool `yaml:"DisableProjectedVolumes"`
	// JobScriptBuilderURL is an optional URL for an external job script builder
	JobScriptBuilderURL string `yaml:"JobScriptBuilderURL,omitempty"`
	// VKConfigPath is the path to the Virtual Kubelet configuration file
	VKConfigPath string `yaml:"VKConfigPath"`
	// VKTokenFile is the path to the token file for authenticating with the K8s API
	VKTokenFile string `yaml:"VKTokenFile"`
	// ServiceAccount is the name of the Kubernetes ServiceAccount to use
	ServiceAccount string `yaml:"ServiceAccount"`
	// Namespace specifies the Kubernetes namespace in which the Virtual Kubelet operates
	Namespace string `yaml:"Namespace"`
	// PodIP is the IP address assigned to the virtual node
	PodIP string `yaml:"PodIP"`
	// PodCIDR defines the CIDR range for pods assigned to the virtual node
	PodCIDR PodCIDR `yaml:"PodCIDR"`
	// VerboseLogging enables detailed logging output
	VerboseLogging bool `yaml:"VerboseLogging"`
	// ErrorsOnlyLogging restricts logging to error messages only
	ErrorsOnlyLogging bool `yaml:"ErrorsOnlyLogging"`
	// HTTP configures HTTP connection security
	HTTP HTTP `yaml:"HTTP"`
	// KubeletHTTP configures HTTP settings specific to Kubelet communication
	KubeletHTTP HTTP `yaml:"KubeletHTTP"`
	// KubeletCertFile is the path to the kubelet server certificate file (optional, for manual certificate management)
	KubeletCertFile string `yaml:"KubeletCertFile,omitempty"`
	// KubeletKeyFile is the path to the kubelet server key file (optional, for manual certificate management)
	KubeletKeyFile string `yaml:"KubeletKeyFile,omitempty"`
	// KubeletCSRSignerName specifies the signer name for CSR-based certificates (default: kubernetes.io/kubelet-serving)
	// Can be used with cert-manager: clusterissuers.cert-manager.io/<issuer-name>
	KubeletCSRSignerName string `yaml:"KubeletCSRSignerName,omitempty"`
	// Resources specifies compute resources available to the virtual node
	Resources Resources `yaml:"Resources"`
	// NodeLabels allows setting custom labels on the virtual node
	NodeLabels []string `yaml:"NodeLabels"`
	// NodeTaints allows setting taints on the virtual node
	NodeTaints []TaintSpec `yaml:"NodeTaints"`
	// TLS configures TLS/mTLS support for secure interLink API communication
	TLS TLSConfig `yaml:"TLS,omitempty"`
	// Network contains network-related settings for the virtual node
	Network Network `yaml:"Network,omitempty"`
	// SkipDownwardAPIResolution disables downward API resolution to enable scheduling pods with downward API
	SkipDownwardAPIResolution bool `yaml:"SkipDownwardAPIResolution,omitempty"`
	// DisableCSR disables CSR (CertificateSigningRequest) creation and uses self-signed certificates instead
	DisableCSR bool `yaml:"DisableCSR,omitempty"`
}

Config holds the complete configuration for the Virtual Kubelet provider. It defines how the virtual node connects to the Kubernetes cluster and interLink API.

func LoadConfig 

func LoadConfig(ctx context.Context, providerConfig string) (config Config, err error)

LoadConfig loads the given json configuration files and return a VirtualKubeletConfig struct

type Crtretriever 

type Crtretriever func(*tls.ClientHelloInfo) (*tls.Certificate, error)

func NewCertificateRetriever 

func NewCertificateRetriever(kubeClient kubernetes.Interface, signer string, nodeName string, nodeIP net.IP) (Crtretriever, error)

NewCertificateRetriever creates a certificate retriever that creates a single CSR and waits indefinitely for approval without any timeout. This implementation: - Creates ONE CSR on startup - Polls every 10 seconds checking if it's been approved (no 15-minute timeout) - Only creates a new CSR when the certificate is at 80% of its lifetime (near expiration) - Handles denied CSRs by creating a new one

func NewSelfSignedCertificateRetriever 

func NewSelfSignedCertificateRetriever(nodeName string, nodeIP net.IP) Crtretriever

newSelfSignedCertificateRetriever creates a new retriever for self-signed certificates.

type HTTP 

type HTTP struct {
	// Insecure indicates whether to skip certificate verification (use with caution)
	Insecure bool `yaml:"Insecure"`
	// CaCert is the path to the CA certificate for verifying server connections
	CaCert string `yaml:"CaCert"`
}

HTTP defines security settings for HTTP connections. It determines whether connections are insecure and holds CA certificates.

type MeshScriptTemplateData 

type MeshScriptTemplateData struct {
	WGInterfaceName       string
	WSTunnelExecutableURL string
	WireguardGoURL        string
	WgToolURL             string
	Slirp4netnsURL        string
	WGConfig              string
	DNSServiceIP          string
	RandomPassword        string
	IngressEndpoint       string
	WGMTU                 int
	PodCIDRCluster        string
	ServiceCIDR           string
	UnshareMode           string
}

type Network 

type Network struct {
	// EnableTunnel enables WebSocket tunneling for pod port exposure
	EnableTunnel bool `yaml:"EnableTunnel" default:"false"`
	// WildcardDNS specifies the DNS domain for generating tunnel endpoints
	WildcardDNS string `yaml:"WildcardDNS,omitempty"`
	// WSTunnelExecutableURL specifies the URL to download the wstunnel executable (default is "https://github.com/interlink-hq/interlink-artifacts/raw/main/wstunnel/v10.4.4/linux-amd64/wstunnel")
	WSTunnelExecutableURL string `yaml:"WSTunnelExecutable,omitempty"`
	// WstunnelTemplatePath is the path to a custom wstunnel template file
	WstunnelTemplatePath string `yaml:"WstunnelTemplatePath,omitempty"`
	// WstunnelCommand specifies the command template for setting up wstunnel clients
	WstunnelCommand string `yaml:"WstunnelCommand,omitempty"`
	// FullMesh enables full mesh networking with slirp4netns and WireGuard
	FullMesh bool `yaml:"FullMesh" default:"false"`
	// MeshScriptTemplatePath is the path to a custom mesh.sh template file
	MeshScriptTemplatePath string `yaml:"MeshScriptTemplatePath,omitempty"`
	// ServiceCIDR specifies the CIDR range for Kubernetes services
	ServiceCIDR string `yaml:"ServiceCIDR,omitempty"`
	// PodCIDRCluster specifies the CIDR range for pods in the main cluster
	PodCIDRCluster string `yaml:"PodCIDRCluster,omitempty"`
	// DNSServiceIP specifies the IP address of the DNS service (e.g., kube-dns)
	DNSServiceIP string `yaml:"DNSServiceIP,omitempty"`
	// WireguardGoURL specifies the URL to download wireguard-go binary (default is "https://github.com/interlink-hq/interlink-artifacts/raw/main/wireguard-go/v0.0.20201118/linux-amd64/wireguard-go")
	WireguardGoURL string `yaml:"WireguardGoURL,omitempty"`
	// WgToolURL specifies the URL to download wg tool binary (default is "https://github.com/interlink-hq/interlink-artifacts/raw/main/wgtools/v1.0.20210914/linux-amd64/wg")
	WgToolURL string `yaml:"WgToolURL,omitempty"`
	// Slirp4netnsURL specifies the URL to download slirp4netns binary (default is "https://github.com/interlink-hq/interlink-artifacts/raw/main/slirp4netns/v1.2.3/linux-amd64/slirp4netns")
	Slirp4netnsURL string `yaml:"Slirp4netnsURL,omitempty"`
	// UnsharedMode is the flag for unshared network mode in slirp4netns
	UnshareMode string `yaml:"UnshareMode,omitempty"`
}

Network configures networking features for the virtual node. It includes settings for tunneling and service exposure.

type PodCIDR 

type PodCIDR struct {
	// Subnet specifies the CIDR subnet for pod IP allocation (e.g., "10.10.0.0/24")
	Subnet string `yaml:"Subnet"`
	// MaxIP specifies the maximum IP address number to allocate (e.g., 250)
	MaxIP int `yaml:"MaxIP"`
	// MinIP specifies the minimum IP address number to allocate (e.g., 2)
	MinIP int `yaml:"MinIP"`
}

PodCIDR defines the CIDR range and IP allocation settings for pods on this node. This is used when pods need specific IP addresses within the node's network.

type PortMapping 

type PortMapping struct {
	Port     int32
	Name     string
	Protocol string
}

type Provider 

type Provider struct {
	// contains filtered or unexported fields
}

Provider defines the properties of the virtual kubelet provider

func NewProvider 

func NewProvider(
	ctx context.Context,
	providerConfig,
	nodeName,
	nodeVersion,
	operatingSystem string,
	internalIP string,
	daemonEndpointPort int32,
	clientHTTPTransport *http.Transport,
) (*Provider, error)

NewProvider creates a new Provider, which implements the PodNotifier and other virtual-kubelet interfaces

func NewProviderConfig 

func NewProviderConfig(
	config Config,
	nodeName string,
	nodeVersion string,
	operatingSystem string,
	internalIP string,
	daemonEndpointPort int32,
	clientHTTPTransport *http.Transport,
) (*Provider, error)

NewProviderConfig takes user-defined configuration and fills the Virtual Kubelet provider struct

func (*Provider) CreatePod 

func (p *Provider) CreatePod(ctx context.Context, pod *v1.Pod) error

CreatePod accepts a Pod definition and stores it in memory in p.pods

func (*Provider) DeletePod 

func (p *Provider) DeletePod(ctx context.Context, pod *v1.Pod) (err error)

DeletePod deletes the specified pod and drops it out of p.pods

func (*Provider) GetLogs 

func (p *Provider) GetLogs(ctx context.Context, namespace, podName, containerName string, opts api.ContainerLogOpts) (io.ReadCloser, error)

GetLogs implements the logic for interLink pod logs retrieval.

func (*Provider) GetNode 

func (p *Provider) GetNode() *v1.Node

GetNode return the Node information at the initiation of a virtual node

func (*Provider) GetPod 

func (p *Provider) GetPod(_ context.Context, _ string, _ string) (*v1.Pod, error)

func (*Provider) GetPodByUID 

func (p *Provider) GetPodByUID(ctx context.Context, namespace, name string, uid k8stypes.UID) (pod *v1.Pod, err error)

GetPodByUID returns a pod by name that is stored in memory.

func (*Provider) GetPodStatus 

func (p *Provider) GetPodStatus(_ context.Context, _ string, _ string) (*v1.PodStatus, error)

func (*Provider) GetPodStatusByUID 

func (p *Provider) GetPodStatusByUID(ctx context.Context, namespace, name string, uid k8stypes.UID) (*v1.PodStatus, error)

GetPodStatusByUID returns the status of a pod by name that is "running". returns nil if a pod by that name is not found.

func (*Provider) GetPods 

func (p *Provider) GetPods(ctx context.Context) ([]*v1.Pod, error)

GetPods returns a list of all pods known to be "running".

func (*Provider) GetStatsSummary 

func (p *Provider) GetStatsSummary(ctx context.Context) (*stats.Summary, error)

GetStatsSummary returns dummy stats for all pods known by this provider.

func (*Provider) NotifyNodeStatus 

func (p *Provider) NotifyNodeStatus(ctx context.Context, f func(*v1.Node))

NotifyNodeStatus runs once at initiation time and set the function to be used for node change notification (native of vk) it also starts a go routine for continously checking the node status and availability

func (*Provider) NotifyPods 

func (p *Provider) NotifyPods(_ context.Context, f func(*v1.Pod))

NotifyPods is called to set a pod notifier callback function. Also starts the go routine to monitor all vk pods

func (*Provider) Ping 

func (p *Provider) Ping(_ context.Context) error

Ping the kubelet from the cluster, this will always be ok by design probably

func (*Provider) RetrievePodsFromCluster 

func (p *Provider) RetrievePodsFromCluster(ctx context.Context) error

RetrievePodsFromCluster scans all pods registered to the K8S cluster and re-assigns the ones with a valid JobID to the Virtual Kubelet. This will run at the initiation time only

func (*Provider) UpdatePod 

func (p *Provider) UpdatePod(ctx context.Context, pod *v1.Pod) error

UpdatePod accepts a Pod definition and updates its reference.

type Resources 

type Resources struct {
	// CPU specifies the total CPU capacity (e.g., "100", "2000m")
	CPU string `yaml:"CPU,omitempty"`
	// Memory specifies the total memory capacity (e.g., "128Gi", "64000Mi")
	Memory string `yaml:"Memory,omitempty"`
	// Pods specifies the maximum number of pods this node can handle
	Pods string `yaml:"Pods,omitempty"`
	// Accelerators lists hardware accelerators available on this node
	Accelerators []Accelerator `yaml:"Accelerators"`
}

Resources defines the compute resources available to the virtual node. These values are reported to Kubernetes and used for pod scheduling decisions.

type TLSConfig 

type TLSConfig struct {
	// Enabled indicates whether TLS is enabled
	Enabled bool `yaml:"Enabled"`
	// CertFile is the path to the client certificate file for mTLS
	CertFile string `yaml:"CertFile,omitempty"`
	// KeyFile is the path to the client key file for mTLS
	KeyFile string `yaml:"KeyFile,omitempty"`
	// CACertFile is the path to the CA cert file for server verification
	CACertFile string `yaml:"CACertFile,omitempty"`
}

TLSConfig holds TLS/mTLS configuration for secure communication with interLink API.

type TaintSpec 

type TaintSpec struct {
	// Key is the taint key (e.g., "virtual-node.interlink/no-schedule")
	Key string `yaml:"Key"`
	// Value is the taint value
	Value string `yaml:"Value"`
	// Effect specifies the taint effect ("NoSchedule", "PreferNoSchedule", "NoExecute")
	Effect string `yaml:"Effect"`
}

TaintSpec defines a Kubernetes taint to be applied to the virtual node. Taints prevent pods from being scheduled unless they have matching tolerations.

type WstunnelTemplateData 

type WstunnelTemplateData struct {
	Name             string
	Namespace        string
	RandomPassword   string
	ExposedPorts     []PortMapping
	WildcardDNS      string
	WGPrivateKey     string
	ClientPublicKey  string
	WGMTU            int
	KeepaliveSecs    int
	ClientPrivateKey string // only if we generated it
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.