Our product team has been busy this week and we’re excited to share some of the updates we’ve shipped. We thank you for your continued support and collaboration, since it directly helps us continue to ship great products for our customers.

Product Updates

Product Updates

Metadata-based routing

You can now set routing rules based on the metadata of a request, including:

• user or group id

• auth header information

• and more

Using a model-based classifier to route

You can now use a LLM-based classifier to route model requests. Any generative model available in Radiant can be used to make a routing decision. Some examples of how this can be used include:

• Using a small (and cheaper) locally-hosted LLM to route certain requests to a model based on cost or security considerations

• Using an LLM to route to specific models fine-tuned for a certain task

Redesigned Provider Interface 

We introduced a new provider interface to make it easier to navigate across providers, view models, monitor usage and set budget on a per provider or model level. 

The new provider interface lets administrators easily see which endpoints are consuming the most tokens in a specific model. 

Refined Endpoint tagging workflow 

This week we made tagging endpoints and workflows more powerful and intuitive.

• The tagging capability allows administrators to characterize endpoints and filter models, logs and usage using any number of parameters. 

• The refined workflow streamlines the processing of adding tags to an endpoints, creating and renaming tags. 

Collaboration with SkyPilot

This week we announced a collaboration with SkyPilot. SkyPilot is a rapidly growing open source project for training and serving Generative AI models. SBy combining a SkyPilot deployed inference server with Radiant’s control plane enterprises can achieve significant performance scale, and cost reduction while ensuring that all model usage comes with unified data controls, model permissions and audit logging. Read more on our blog.

New GenAI model integrations

• This week we added support for Claude 3 by Anthropic, which has turned out to be one of the first serious challenger to GPT-4 when it comes to capabilities in the enterprise context.

• Added support for Groq, a compute platform with highly optimized hardware for rapid LLM execution

About Radiant

Radiant is the Enterprise AI platform to take you from idea to production deployment. From security and governance to scaling and resiliency, we make it simple to make AI a critical part of your business. 

Try out a demo here, sign up here to get your own instance, or reach out to our founders directly at [email protected].

We’re also hiring. If you know of someone great that is interested in helping every company build AI into their products and operations, we'd love an introduction.