Node Groups

What is Node Groups?

A node group is a dedicated set of nodes with a specific GPU type.

There are three types of node groups:

• Lepton Managed Node Groups: The nodes are owned by Lepton. The customer can create workloads over it, view workload distribution, monitor node status, reboot nodes and pause scheduling over the nodes if needed.
• Customer Node Groups: The nodes are owned by customers. Thus customer can manage their nodes, view workload distribution, and monitor node status themselves.
• Monitoring-Only Nodes: The nodes designated solely for monitoring purposes and are not involved in workload orchestration.

Lepton Managed Node Groups

As of now, you need to open a ticket to get access to Lepton managed node groups. Head over the node groups page and click on "Create Node Group", select "Create From Lepton" to open a ticket.

You can provide a detailed comment and some attachments in the ticket so that we can review your request better.

Bring Your Own Account

Bring your own account into Lepton, enabling custom infrastructure management for enterprise workloads. You can manage your cloud accounts in your workspace.

Currently, we support the following account types, you can check the detailed guide for each account type:

• Lambda Labs
• Azure

Create and Use Node Group from Your Account

Create Node Group

Assume that you have successfully added your Lambda Labs account or Azure account to your workspace, you can now create a node group under the account.

1. Go to node groups page
2. Click on "Create Node Group", select "Create From Your Account"
3. Fill in the required fields and click on "Create", including:
   • Node Group Name
   • Provider
   • GPU Type

Use Node Group

After the creation, you can see a detailed information of the node group in the node group page.

Scale Nodes

By default, the node group is empty after creation. You need to scale the nodes manually by clicking on "Scale Nodes" button. Input the desired count and select a desired region in the provider section, and click on "Submit" to start the scaling process.

After submitting, you can see the nodes scaling status in the node group page.

Create Workloads from Node Group

You can then create workloads from a node group with available nodes. Take dedicated endpoint as an example, you can specify the node group in Resource section when creating a dedicated endpoint, and then you can see the available GPU type within that node group.