Business

Stop managing infrastructure: A new way to deploy AI agents and models

Standing up an agent as a production service on Kubernetes means five YAML files, a few hundred lines between them, and (in most enterprises) a ticket in someone else’s queue. On the Workload API it means one spec file, one command, and about five minutes to a live URL. No manifests, no kubectl, no namespace, nobody else in the loop.

AI workloads increasingly look like long-running services, not request/response models. Agents are the clearest example: they hold state, call tools, wait on LLM responses, and run for minutes or hours at a time. The same is true of inference servers, RAG pipelines, and the frontends that sit on top of them. In most enterprises, turning any of these into a production service means Kubernetes: namespaces, Deployments, Ingress rules, autoscaling policies, health probes, and a platform team in the loop for every change.

Today we’re announcing the general availability of the Workload API: a single layer for deploying and governing AI services on DataRobot. Bring a container image that serves HTTP; you get a stable URL with autoscaling, monitoring, and sharing, with a lifecycle you drive end to end through one API. For you, that means no Kubernetes manifests, no kubectl, and no platform ticket. The governance underneath is what buys you that: because every workload is a governed object by default, your platform team never has to choose between reviewing your deployment and handing you cluster credentials.

What stands between a working service and a production one

Shipping a long-running AI service on self-managed infrastructure typically requires:

  • A cluster, a namespace, and permissions to use them
  • Deployment manifests, Services, and Ingress configuration
  • Autoscaler tuning and node pool planning for GPUs
  • Liveness and readiness probes, wired up correctly
  • Log aggregation, metrics, and tracing, assembled from separate tools
  • A platform engineer involved in every version rollout

None of this is the service itself, and every item lands on someone. Either the AI developer learns Kubernetes, or a platform team fields the ticket. At enterprise scale, IT ends up choosing between two bad options: become the bottleneck for every AI deployment in the organization, or hand out cluster permissions to teams whose job is building agents, not operating infrastructure.

Generic serverless container platforms remove part of the setup, but they stop at the URL. What they don’t hand you is an identity: a governed object that carries sharing, monitoring, and an immutable production version, and that survives the trip from the thing you were iterating on to the thing your company depends on. They also don’t give you AI-native observability, an answer when a compliance team asks who can invoke a service and what it has been doing, or an autoscaler that understands KV-cache pressure instead of CPU. The Workload API keeps the one-command experience and adds the part that makes a service shippable inside a company.

Artifacts, workloads, and protons

A deployment layer is only useful if its model is small enough to hold in your head. The Workload API reduces the infrastructure surface to three objects:

  • Artifact   → what to run (image, port, entrypoint, env vars, probes)
  • Workload   → the governed identity (stable URL, sharing, monitoring)
  • Protons    → the running instance(s) backing the workload

The artifact describes what to run. The workload is the governed identity you hand to consumers. Protons are the execution. Scaling is a replica count. GPU selection is a bundle name rather than node pools and taints. The Workload concepts and Artifact concepts pages cover the full model.

The API is container-shaped by design. Agent services built on LangGraph, CrewAI, or custom orchestration run alongside model inference servers (NVIDIA NIM, vLLM), RAG pipelines, MCP servers, vector databases, and Streamlit or Gradio frontends. Any service that listens on HTTP fits, so an application and the services it depends on can run on one platform with one lifecycle.

Deploy in one command

Describe the workload in a spec file (YAML or JSON), then create it with the DataRobot CLI. One command creates the workload, schedules the container, and returns a stable endpoint URL.

# spec.yaml
name: support-agent
artifact:
  name: support-agent-artifact
  type: service
  spec:
    containerGroups:
      - name: default
        containers:
          - name: agent
            imageUri: your-registry/support-agent:1.0.0
            port: 8080
            primary: true
            readinessProbe: {path: /health, port: 8080, initialDelaySeconds: 5}
            environmentVars:
              - name: LOG_LEVEL
                value: info
              - source: dr-credential      # injected from the DataRobot credential store
                name: OPENAI_API_KEY
                drCredentialId: <credential-id>
                key: apiToken
runtime:
  containerGroups:
    - name: default
      replicaCount: 1
      containers:
        - name: agent
          resourceAllocation: {cpu: 1, memory: "512MB"}

dr workload create --spec-file spec.yaml

The spec has two halves. The artifact half carries everything that travels with the image: port, entrypoint, environment variables, probes. The runtime half carries what varies per deployment: replicas, CPU, memory. Note the environmentVars block: plain values are passed as-is, and secrets are injected by reference from the DataRobot credential store. The API key never appears in the spec, the image, or version control. Check progress and grab the URL:

dr workload status ${WORKLOAD_ID}     # submitted → launching → running
dr workload endpoint ${WORKLOAD_ID}   # the stable URL
dr workload logs ${WORKLOAD_ID}       # container logs

Once the status reaches running, the service is live on a stable URL. What you’ve created is a draft workload: a real endpoint with full monitoring, free to iterate on, and cleaned up automatically after 8 hours of inactivity. Production is one call away and this is the part with no equivalent on a generic container platform: promoting doesn’t redeploy anything. The workload ID, the endpoint URL, and everyone you shared it with all stay exactly as they are, and the artifact locks so production runs the bytes you tested. The thing you iterate on and the thing your company depends on are the same object.

There are other ways to run the same flow. Everything the CLI does maps to REST calls, so plain curl works. The DataRobot Pulumi provider and Terraform provider expose artifacts and workloads as native resources, so workloads can be managed as code: diffable, reviewable, and reproducible across environments. Code-to-Workload builds the container from source, with no Dockerfile or registry push. And the DataRobot Agent Skills plugin lets you create, scale, and debug workloads conversationally from Claude Code and Cowork.

Serve the models behind your agents, too

An agent is only as good as the model endpoint it calls. The Workload API runs generative AI models alongside your agent through two primary paths: seamless integration with NVIDIA NIM, and deploying open models directly from Hugging Face.

For NVIDIA NIM, microservices deploy as a first-class artifact type using the NIM Operator (currently available on self-managed DataRobot on OpenShift). Any model in the NGC catalog—such as Nemotron 3 Nano Omni—can be served with optimized GPU performance and managed weight caching. Alternatively, you can host open-source models directly from Hugging Face using inference servers like vLLM. In both cases, model weights are cached efficiently on persistent volumes, credentials are injected securely from the DataRobot store, and models run on identical GPU bundles with full autoscaling.

The result: your agent and the models powering it run side by side as governed, independent endpoints with unified observability and security.

Day-two operations through the same API

Deployment is one command. The operations that usually require Kubernetes expertise go through the same API::

  • Promoting to production. The agent starts as a draft: iterate freely while it behaves like a real service. When it’s ready, one call promotes it: the artifact locks (immutable and versioned, so production runs exactly what you tested), the draft TTL is removed, and the workload ID, endpoint, and sharing all stay the same. No redeployment, no environment migration.
  • Diagnosing a workload that won’t start. Every workload exposes a lifecycle event log and per-replica status, including container readiness, restart counts, and a log tail. Image pull failures and crash loops are visible through the API and CLI.
  • Observing what the service is doing. Container logs are collected out of the box, with no instrumentation required. For traces and metrics, instrument the container with OpenTelemetry: standard OTel instrumentation ships traces, metrics, and structured logs to DataRobot. For an agent, that means seeing individual LLM calls and tool invocations inside each request. 
  • Monitoring health and utilization. Service health, resource utilization, and quota consumption are tracked per workload with no instrumentation, in the same panes as the rest of the platform. In practice: you can see whether a replica is saturated or idle, whether a restart count is climbing, and whether you are about to hit an org-level scaling cap — before any of it becomes an incident.
  • Moderating traffic in real time. Guards from the DataRobot evaluation and moderation library attach to a workload and run in the request path, scoring quality, tracking token cost per call, and blocking unsafe or non-compliant responses before they reach a user. Same configuration surface as the guards on a DataRobot deployment, so an agent running as a workload is governed the same way a model is.
  • Shipping a new version. Replacing the artifact in a running workload rolls out the new container without dropping the endpoint. The URL stays the same.
  • Controlling access. Sharing is a property of the workload. Services deployed through the Workload API appear in the same governance and monitoring plane as an organization’s models and applications, and the platform runs wherever DataRobot runs, including VPC and on-premise environments. That’s the trade the Workload API makes possible: IT gets one governed surface for every AI service in the organization, and developers never touch a namespace.

Get started

A first workload takes about five minutes: one spec file, one command, and your container is live. Start with Tutorial: Hello, Workload!, then take a real service to production with sharing and monitoring.

The post Stop managing infrastructure: A new way to deploy AI agents and models appeared first on DataRobot.

Picture of John Doe
John Doe

Sociosqu conubia dis malesuada volutpat feugiat urna tortor vehicula adipiscing cubilia. Pede montes cras porttitor habitasse mollis nostra malesuada volutpat letius.

Related Article

Leave a Reply

Your email address will not be published. Required fields are marked *

X
"Hello! Let’s get started on your journey with us."
Site SearchBusiness ServicesBusiness Services

Meet Eve: Your AI Training Assistant

Welcome to Enlightening Methodology! We are excited to introduce Eve, our innovative AI-powered assistant designed specifically for our organization. Eve represents a glimpse into the future of artificial intelligence, continuously learning and growing to enhance the user experience across both healthcare and business sectors.

In Healthcare

In the healthcare category, Eve serves as a valuable resource for our clients. She is capable of answering questions about our business and providing "Day in the Life" training scenario examples that illustrate real-world applications of the training methodologies we employ. Eve offers insights into our unique compliance tool, detailing its capabilities and how it enhances operational efficiency while ensuring adherence to all regulatory statues and full HIPAA compliance. Furthermore, Eve can provide clients with compelling reasons why Enlightening Methodology should be their company of choice for Electronic Health Record (EHR) implementations and AI support. While Eve is purposefully designed for our in-house needs and is just a small example of what AI can offer, her continuous growth highlights the vast potential of AI in transforming healthcare practices.

In Business

In the business section, Eve showcases our extensive offerings, including our cutting-edge compliance tool. She provides examples of its functionality, helping organizations understand how it can streamline compliance processes and improve overall efficiency. Eve also explores our cybersecurity solutions powered by AI, demonstrating how these technologies can protect organizations from potential threats while ensuring data integrity and security. While Eve is tailored for internal purposes, she represents only a fraction of the incredible capabilities that AI can provide. With Eve, you gain access to an intelligent assistant that enhances training, compliance, and operational capabilities, making the journey towards AI implementation more accessible. At Enlightening Methodology, we are committed to innovation and continuous improvement. Join us on this exciting journey as we leverage Eve's abilities to drive progress in both healthcare and business, paving the way for a smarter and more efficient future. With Eve by your side, you're not just engaging with AI; you're witnessing the growth potential of technology that is reshaping training, compliance and our world! Welcome to Enlightening Methodology, where innovation meets opportunity!

[wpbotvoicemessage id="402"]