User:Accraze/MachineLearning/Kserve

Kserve

Serverless Inferencing on k8s

Requirements

Hardware

• Minimum cluster is 4 cpus and 8Gi memory

Software

• k8s 1.16 - 1.18

 * 1.19 is not suitable for kfserving (yet)

• istio service mesh

 \`cluster-local-gateway\`\` is required to serve
 cluster-internal traffic for transformer and explainer use cases.
 https://knative.dev/docs/install/installing-istio/#updating-your-install-to-use-cluster-local-gateway

• knative serving (and eventing if we want transformers/explainers)

• cert manager / letsencrypt

Commands

• Deploy / Update service:

 kubectl apply -f service.yaml -n kubeflow-user

• Delete/Remove model:

 kubectl delete -f service.yaml -n kubeflow-user

• Lookup info:

 kubectl describe inferenceservices.serving.kserve.io <model-name>

Links

• https://github.com/kserve/kserve