GLaDOS: the LLM layer

The first AI feature you ship in a Rails app usually lives in a service class. There’s an HTTP client, a prompt assembled inline, a JSON.parse wrapped in a rescue, a couple of retry blocks, and a Rails.logger.info so you can find the response in production. It works. You move on.

The second feature lives somewhere else. It has its own service class, its own retry policy, a slightly different error handler, a different log line, a slightly different way of squeezing structured output out of the model. By the fifth feature, you have five different patterns for the same five problems: retrying, logging, validating shape, surfacing failures to a human, chaining one call into the next. None of them are wrong, and none of them are the same.

Direct SDK integrations were quick to ship, but each feature needed its own maintenance. Changing providers or investigating a failure meant finding and understanding several implementations of the same work.

We extracted that shared work into GLaDOS, the internal pack that now handles every LLM call across our platform.

The prompt as a first-class object

The core abstraction in GLaDOS is Glados::Prompt. A feature defines a subclass, configures the provider and temperature, writes a system prompt and a user prompt, optionally declares a JSON schema for structured output, and calls it. The framework handles everything else.

class QualificationPrompt < Glados::Prompt
  config provider: :gemini, temperature: 0.0

  def user_prompt
    <<~PROMPT
      Evaluate whether this clinician meets the requirement.

      Requirement: #{requirement.text}
      Clinician evidence: #{evidence.as_json}
    PROMPT
  end

  json_schema do
    object do
      string :verdict, enum: %w[pass fail unknown]
      string :reason
    end
  end

  def self.on_success(result, qualification_ai_execution_id:, **)
    execution = QualificationAIExecution.find(qualification_ai_execution_id)
    QualificationAI::Callback::OnSuccess.new(execution:, result:).call
  end
end

That’s the whole feature-side surface. Building the request, calling the provider, validating the response, retrying on transient failures, writing the operation log, calling back into application code on success or failure --- all of it lives inside the framework. The author of QualificationPrompt writes the prompt and the schema. Nothing else.

Two execution modes sit on the same prompt class. .call runs synchronously and returns the validated result. .call_async enqueues a Sidekiq job that runs the prompt and dispatches the on_success callback when it lands. Same class, same definition. The choice is per call site.

Every execution produces a OperationLog row (Glados::OperationLog). The log carries the full request, the full response, the provider that served it, the latency, the token counts, and a pointer to a parent log if the call is part of a multi-step pipeline. Every AI feature in the codebase produces the same shape of audit trail, because they all go through the same executor.

Having one log format means we can investigate different AI features without first learning how each one records requests and failures.

Tools as classes, agents as loops

Single-turn prompts cover most cases. Some features need the model to call out, get information, and decide what to do next. GLaDOS supports that through .agentic_call, a multi-turn loop that runs the prompt against the model, executes any tool calls the model returns, feeds the results back in, and continues until the model emits a final answer or hits a max-iterations cap.

Tools are first-class. Each is a Glados::Tools::Function subclass with a description, a typed parameter schema, and a call class method. The framework handles the request-tool-result-request handshake.

class FindFacility < Glados::Tools::Function
  description "Search the verified facility database by name, city, and state."

  parameters do
    string :name, description: "Facility name as entered by the clinician."
    string :city, description: "City to narrow the search.", required: false
    string :state, description: "Two-letter state code.", required: false
    integer :page, description: "Page number, 25 results per page.", required: false
  end

  def self.call(name:, city: nil, state: nil, page: 1)
    Facility.verified.search(name:, city:, state:).page(page).limit(25).map(&:as_match)
  end
end

The production example sits behind our work history flow. A clinician types a facility name, the agent calls FindFacility, and the tool returns up to 25 verified facilities. If the first search doesn’t resolve a single confident match, the agent refines: different name, add a city, narrow by state, page through results, until it converges on the facility or determines no match exists. The tool call, the result, and the agent’s next turn are all stitched together in OperationLog, parent-and-child, so the whole trajectory is replayable from one row in the database.

We’ve written about why constraining agents through the same boundaries humans use is the right architectural choice in Agents that fill out forms. Tools-as-classes is the same idea on the inbound side. The agent doesn’t get a free SQL connection or a raw HTTP client. It gets a small, named set of functions with declared parameters, the same way a junior engineer gets a small, named set of methods on a service object.

Two providers, one interface

GLaDOS speaks to two model providers today through a factory pattern. The prompt class declares which it wants. Credential verification in particular has dual configurations and chooses at call time based on the credential type or current system load.

From the feature side, the provider is an implementation detail. Swapping it is a one-line change in the prompt class. The calling code doesn’t change. The schema doesn’t change. The callbacks don’t change.

That separation helps when a provider rate-limits us or a different model works better for a task. The change stays in the prompt’s configuration.

Six features, one framework

Six features currently use GLaDOS.

Feature	What GLaDOS provides	Prompt classes	Sync or async	Distinguishing detail
Credential verification	Three-step pipeline: extract, match, verify, chained via with_previous_log so step two reads step one’s OperationLog without anyone passing context by hand. Same shape handles skills checklists and reference forms.	Three (extract, match, verify)	Async	Pipeline chaining across OperationLog parent/child links.
Qualification	QualificationPrompt evaluation for the ambiguous rules a deterministic engine can’t decide---a free-text experience field against a structured requirement, a license format we haven’t seen before. Verdict plus reason returned as structured JSON.	QualificationPrompt	Sync	The AI half of the rules-and-AI architecture in Automatic Qualifications.
Interaction summarization	After a call, SMS thread, or email exchange wraps up, SummarizationPrompt generates a structured summary attached to the clinician record. Advocates see it in the internal ops surface alongside the raw history.	SummarizationPrompt	Async	Surfaces alongside the interaction record; makes the record useful at a glance.
Autocuration	When a new job lands from a vendor feed, JobAICurationPrompt fills missing fields including specialty inference, shift parsing, and certification requirements. Structured output maps directly onto the job record.	JobAICurationPrompt	Async	Output maps onto the job record without translation.
Professional summary generation	When a qualification completes, ProfessionalSummaryPrompt assembles the professional summary section of the submission packet from the clinician’s experience data. Certifications, charting systems, education, trauma experience returned in schema order.	ProfessionalSummaryPrompt	Async	Packet generator drops the output straight in.
Work history facility matching	The agentic loop in action. Clinician types a facility, the agent matches it against the verified facility database via the FindFacility tool, and the resolved facility ID ends up on the work history row.	Agentic (.agentic_call with FindFacility)	Sync	Multi-turn tool use; full trajectory replayable from one OperationLog row.

They share retry behavior, provider clients, logging, and audit records. A feature author doesn’t need to implement those pieces again.

A shared structure for feedback

A side effect of the shared interface is that human feedback infrastructure plugs in once. The QualificationAIExecution has a feedback record. The credential AI verification has an assessment record. The shape is consistent because the executions are consistent.

An Advocate’s qualification override and an operations reviewer’s credential correction have a consistent structure. That gives us production examples to use when evaluating models or revising prompts, without first reconciling several log formats.

We still need to evaluate prompt changes against the full feedback collection before release. Having one log format and one feedback structure makes that work easier to plan.

Why we waited to build the framework

Building a shared framework too early can lock in guesses about what future features will need.

We waited. The first three AI features at Trusted didn’t run on GLaDOS, because GLaDOS didn’t exist yet. Each one shipped as its own service-class integration, and each one taught us something specific about what the shared interface needed to handle: structured output validation, the retry-on-transient-failure pattern, the async path for long-running calls, the multi-step chain. By the time we extracted the framework, we had three production codebases worth of opinions about what it should look like.

The compounding shows up afterward. The first prompt class in GLaDOS gets you, with no extra work: logging, retries, structured output validation, async execution via Sidekiq, multi-turn conversation support, tool calling, and the audit trail. The tenth prompt class gets all of that plus the ability to read the operation log of an earlier step in the same pipeline. Our credential verification pipeline --- extract, then match, then verify --- works because each step can see the previous step’s context without anyone passing it explicitly.

For the next feature, we can write a prompt class and schema using the execution and debugging tools we already have.