Lepton AI

lep photon

Manages photons locally and on the Lepton AI cloud.

Photon is at the core of Lepton AI's abstraction: it is a Python centric abstraction of an AI model or application, and provides a simple interface to specify dependencies, extra files, and environment variables. For more details, see leptonai.photon.Photon.

The photon command is used to create photons locally, push and fetch photons between local and remote, and run, list and delete photons either locally or remotely.

Usage

lep photon [OPTIONS] COMMAND [ARGS]...

Options

  • --help : Show this message and exit.

Commands

  • create : Creates a new photon in the local environment.
  • fetch : Fetch a photon from the workspace.
  • list : Lists all photons.
  • metadata : Returns the metadata json of the photon.
  • push : Push a photon to the workspace.
  • remove : Removes a photon.
  • run : Runs a photon.
  • scaffold : Creates a scaffolding main.py file for a new photon.

lep photon scaffold

Creates a scaffolding main.py file for a new photon. The file serves as a starting point that you can modify to create your own implementations. After implementing your photon, you can use lep photon create -n [name] -m main.py to create a photon from the file.

Usage

lep photon scaffold [OPTIONS]

Options

  • -n, --name TEXT : Name of the scaffolding file
  • --help : Show this message and exit.

lep photon create

Creates a new photon in the local environment. For specifics on the model spec, see leptonai.photon.Photon. To push a photon to the workspace, use lep photon push.

Developer note: insert a link to the photon documentation here.

Usage

lep photon create [OPTIONS]

Options

  • -n, --name TEXT : Name of the photon [required]
  • -m, --model TEXT : Model spec [required]
  • --help : Show this message and exit.

lep photon remove

Removes a photon. The behavior of this command depends on whether one has logged in to the Lepton AI cloud via lep login. If one has logged in, this command will remove the photon from the workspace. Otherwise, or of --local is explicitly specified, it will remove the photon from the local environment.

Usage

lep photon remove [OPTIONS]

Options

  • -n, --name TEXT : Name of the photon to delete. If --all is specified, all versions of the photon with this name will be deleted. Otherwise, remove the latest version of the photon with this name.
  • -l, --local : Remove local photons instead of remote.
  • -i, --id TEXT : The specific version id of the photon to remove.
  • -a, --all : Remove all versions of the photon.
  • --help : Show this message and exit.

lep photon list

Lists all photons. If one has logged in to the Lepton AI cloud via lep login, this command will list all photons in the Lepton AI cloud. Otherwise, or if --local is explicitly specified, it will list all photons in the local environment.

Usage

lep photon list [OPTIONS]

Options

  • -l, --local : If specified, list local photons
  • --pattern TEXT : Regular expression pattern to filter photon names
  • --public-photon : If specified, list photons from the public photon registry.
  • --help : Show this message and exit.

lep photon run

Runs a photon. If one has logged in to the Lepton AI cloud via lep login, the photon will be run on the cloud. Otherwise, or if --local is specified, the photon will be run locally.

Refer to the documentation for a more detailed description on the choices among --name, --model, --file and --id.

Usage

lep photon run [OPTIONS]

Options

  • -n, --name TEXT : Name of the photon to run.
  • -m, --model TEXT : Model spec of the photon. If model is specified, we will rebuild the photon before running.
  • -f, --file TEXT : Path to the specific .photon file to run.
  • -l, --local : If specified, run photon locally (note that this can only run locally stored photons)
  • -p, --port INTEGER : Port to run on.
  • -i, --id TEXT : ID of the photon (only required for remote).
  • --resource-shape TEXT : Resource shape for the deployment. Available types are: 'cpu.small', 'cpu.medium', 'cpu.large', 'gpu.t4', 'gpu.a10'.
  • --min-replicas INTEGER : Minimum number of replicas.
  • --max-replicas INTEGER : Maximum number of replicas.
  • --mount TEXT : Persistent storage to be mounted to the deployment, in the format STORAGE_PATH:MOUNT_PATH.
  • -dn, --deployment-name TEXT : Optional name for the deployment. If not specified, we will attempt to use the name (if specified) or id as the base name, and find the first non-conflict name by appending a number.
  • -e, --env TEXT : Environment variables to pass to the deployment, in the format NAME=VALUE.
  • -s, --secret TEXT : Secrets to pass to the deployment, in the format NAME=SECRET_NAME. If secret name is also the environment variable name, you can omit it and simply pass SECRET_NAME.
  • --public : If specified, the photon will be publicly accessible. See docs for details on access control.
  • --tokens TEXT : Additional tokens that can be used to access the photon. See docs for details on access control.
  • --no-traffic-timeout INTEGER : If specified, the deployment will be scaled down to 0 replicas after the specified number of seconds without traffic. Minimum is 60 seconds if set. Note that actual timeout may be up to 30 seconds longer than the specified value.
  • --target-gpu-utilization INTEGER : If min and max replicas are set, if the gpu utilization is higher than the target gpu utilization, autoscaler will scale up the replicas. If the gpu utilization is lower than the target gpu utilization, autoscaler will scale down the replicas. The value should be between 0 and 99.
  • --initial-delay-seconds INTEGER : If specified, the deployment will allow the specified amount of seconds for the photon to initialize before it starts the service. Usually you should not need this. If you have a deployment that takes a long time to initialize, set it to a longer value.
  • --include-workspace-token : If specified, the workspace token will be included as an environment variable. This is used when the photon code uses Lepton SDK capabilities such as queue, KV, objectstore etc. Note that you should trust the code in the photon, as it will have access to the workspace token.
  • --rerun : If specified, shutdown the deployment of the same deployment name and rerun it. Note that this may cause downtime of the photon if it is for production use, so use with caution. In a production environment, you should do photon create, push, and lep deployment update instead.
  • --public-photon : If specified, get the photon from the public photon registry. This is only supported for remote execution.
  • --help : Show this message and exit.

lep photon push

Push a photon to the workspace.

Usage

lep photon push [OPTIONS]

Options

  • -n, --name TEXT : Name of the photon [required]
  • --help : Show this message and exit.

lep photon metadata

Returns the metadata json of the photon.

Usage

lep photon metadata [OPTIONS]

Options

  • -n, --name TEXT : Name of the photon [required]
  • -i, --id TEXT : ID of the photon, if the photon is a remote one. If not specified, the latest photon will be used.
  • -l, --local : If specified, obtain metadata for the local photon.
  • --indent INTEGER : Indentation of the json.
  • --public-photon : If specified, get the photon from the public photon registry. This is only supported for remote execution.
  • --help : Show this message and exit.

lep photon fetch

Fetch a photon from the workspace.

Usage

lep photon fetch [OPTIONS]

Options

  • -i, --id TEXT : ID of the photon [required]
  • -f, --file TEXT : Path to the local .photon file
  • --help : Show this message and exit.