Skip to main content
Applies to:
  • Plan - Any
  • Deployment - Any
  • Use case - Online scoring automations that silently stop firing because the scorer's Gemini model is unavailable or misconfigured in the AI gateway

​
Summary

Issue: Automations configured to run scorers stop firing silently, with no UI error. Playground tests return Braintrust gateway error: Unable to detect source format, and backend logs show unknown model or provider not configured for Gemini models such as gemini-2.5-flash, gemini-2.5-pro, or gemini-3-flash-preview. Cause: The Gemini model configured in the scorer is unavailable or not routable through the currently enabled AI providers in your Braintrust org. Resolution: Make the required Vertex AI registry model available on the provider and update affected scorers to use its model ID.

​
Resolution steps

​
Step 1: Allow the required Vertex AI model

  1. Go to Settings > AI providers.
  2. Open the Vertex AI provider’s Setup tab.
  3. Under Models, set Registry models to All models, or use Selected models and select the model the scorer needs. Preserve your organization’s intended model restrictions.
  4. Click Update.

​
Step 2: Update the affected scorer

Open each affected scorer and change the configured model to one available in the Vertex registry (for example, publishers/google/models/gemini-2.5-flash or gemini-2.5-flash).

​
Step 3: Verify the automation fires

Trigger a manual test of the automation or apply the scorer directly to a log entry. Confirm that scores are written and no gateway error is returned.

​
If you prefer to avoid Vertex AI

​
Step 1: Switch to a different model provider

Update the scorer to use a model from a provider already configured and verified in your org (for example, an OpenAI or Anthropic model). Retry the automation test to confirm it runs successfully.

​
Notes

  • Silent failures occur because scorer LLM call errors at the gateway layer do not surface as visible errors in the automation UI.
  • If multiple automations stopped firing around the same date, check whether a model was removed or a provider configuration changed around that time.
  • If the playground returns Unable to detect source format, treat this as a signal of a misconfigured or unavailable model rather than a data formatting issue.