User:Accraze/MachineLearning/Kserve
Kserve
Serverless Inferencing on k8s
Requirements
Hardware
- Minimum cluster is 4 cpus and 8Gi memory
Software
- k8s 1.16 - 1.18
* 1.19 is not suitable for kfserving (yet)
- istio service mesh
\`cluster-local-gateway\`\` is required to serve cluster-internal traffic for transformer and explainer use cases. https://knative.dev/docs/install/installing-istio/#updating-your-install-to-use-cluster-local-gateway
- knative serving (and eventing if we want transformers/explainers)
- cert manager / letsencrypt
Commands
- Deploy / Update service:
kubectl apply -f service.yaml -n kubeflow-user
- Delete/Remove model:
kubectl delete -f service.yaml -n kubeflow-user
- Lookup info:
kubectl describe inferenceservices.serving.kserve.io <model-name>