Skip to content

Topic

Model routing and cascades

Serving designs that send each request to a cheaper or more capable model depending on difficulty or a confidence check, escalating only when the cheap answer fails.

Current clusters