Job curation: turning vendor data into a listing

A vendor management system sends us a row that says something like TRAVEL---RN---ICU-MED-3W, a company code, a numeric rate, a start date, a position count, and a paragraph of free text glued to a PDF. That is not a job. Nothing in that payload could be shown to a clinician and called a posting. No canonical unit, no validated rate, no rule about who can apply. The vendor sent us source material. Someone has to turn it into a record the marketplace can use.

Human curators have done this in our Manage app for most of Trusted’s history. More recently, an agent has started doing the same work through the same form and validations. Before we could automate it, we needed to define the decisions behind that form.

This is the background to Agents that fill out forms: the rules, lifecycle, and tasks we developed through years of manual curation.

Four tasks, not one

Curation involves four tasks that share an interface and validation rules.

A decision: does this job proceed to the marketplace, get rejected outright, or sit on a waitlist until something resolves? A duplicate of a job we already posted yesterday. A rate that’s implausible for the unit. A facility we don’t contract with. Each of these terminates the curation flow with a different exit code.

A mapping: which canonical clinical unit does the vendor string refer to? We maintain a taxonomy of clinical units: Medical-Surgical ICU, Step-Down, Cardiac Cath Lab, Labor & Delivery, hundreds of them. The vendor sends abbreviations, internal codes, free-text fragments. A clinician searching for “Med-Surg ICU” finds this job only if the mapping is right.

Job details: the structured record. Shift type. Hours per week. Number of positions. Bill rate. Start and end date. Location. The vendor payload contains all of these somewhere---sometimes in a structured field, sometimes glued into a description, sometimes implicit in an attached PDF.

Job rules: the constraints. Required certifications, years of experience, EMR proficiency, charge-nurse status, floating policies, holiday coverage, license types. Every gate that determines who is eligible to accept the shift. These come partly from the vendor payload and partly from standing program rules at the client.

Each task has its own failure mode. A bad mapping means the right clinician can’t find the job. A wrong rate produces a wrong pay package. A missing rule lets unqualified clinicians apply and get bounced later in the funnel, which costs both sides. A wrong decision puts a duplicate or a bad job in front of the market and erodes trust in the catalog. Lumping these together as “curation quality” obscures which kind of error is happening and where to invest.

The four tasks also need different inputs and different validation. Decision compares against existing postings and client policy. Mapping needs the unit taxonomy. Details needs schema-level field validation. Rules needs the rule data model plus a way to merge job-specific rules with program and global defaults. They share a final commit; the work to get there has four different shapes.

The curation data model

Two kinds of records represent the work.

The first is Curation, a single record attached to each job. It holds the decision and its reasoning. The decision is one of a small set, and each decision class has its own enumerated reasons. A few from the production model:

• approve: the standard path
• reject with reasons like duplicate, rate_implausible, facility_not_under_contract, closed_to_travelers, posting_appears_invalid
• waitlist with reasons like awaiting_program_confirmation, awaiting_rule_clarification, awaiting_facility_setup

The reason is not optional. Every terminal state has to identify why it ended up there, in machine-readable form, because downstream systems behave differently depending on the cause. A rejected duplicate is recycled silently. A facility_not_under_contract raises a flag to the contracts team. A waitlisted job blocks marketplace visibility but stays available for re-curation when its blocker clears.

The second shape is JobRules, and this is the part that gets misunderstood. It’s not a table. It’s a polymorphic set of rule models, each owning one rule type with its own attributes and validations. References, SkillsChecklist, Certifications, AcuteCareExperience, LocationRestriction, FloatingPolicy, and more. A certifications rule has a list of credential codes. A floating policy has a yes/no plus an allowlist of acceptable float units. A reference rule has a count, a relationship type, and a recency window. Each constraint has its own shape, so each lives in its own model with its own validation logic.

Rules also live at three levels: the job itself, the program (the client’s standing configuration), and a global default. The effective rule set for a specific job is resolved by walking up the hierarchy, with job-level rules overriding program-level rules overriding the global default. It’s the same shape we use elsewhere on the platform, and it’s the reason we can onboard a new program by writing rule data instead of writing code.

For both a person and an agent, submitting the curation form writes to typed rule models, runs their validations, and saves the changes together in one transaction.

One job, five states

Curation is a state machine, and the states tell you who’s responsible for the next move.

The states are defined in a single enumeration.

module JobAutoCurationStatus
  INGESTED         = "ingested"
  PENDING          = "pending"
  APPROVED         = "approved"
  REJECTED         = "rejected"
  AUTO_TRANSITIONED = "auto_transitioned"
end

ingested is the raw state, fresh from the ingestion pipeline. Vendor data has been normalized into a Job record but no curation has run. Nothing is shown to anyone.

pending means curation is required and hasn’t completed. Either the autocuration agent hasn’t run on this job, or it ran and stopped short: low confidence, an unresolved validation, an ambiguity that needs a human eye. Pending jobs are the human queue.

approved and rejected are terminal. The job is either in the marketplace or it isn’t, with a Curation record explaining why.

auto_transitioned is the state we added when the agent shipped. It’s approved plus a flag that the approval came from the autocuration pipeline rather than a human action. We keep the distinction because we audit the two populations separately, and because auditor feedback on auto-transitioned jobs is the signal we use to keep the agent honest in production. We want that seam visible in the data, not buried in a boolean on a metadata column.

Each job has a current state and a record of its transitions. The human queue is a query for status = pending, and agents use the same state machine.

What makes curation hard at scale

Volume is the obvious one. A new vendor relationship can dump hundreds of jobs into the queue in a single ingestion run. Multiplied by the count of our vendor relationships and the cadence at which they refresh, the inbound rate dwarfs anything a static curation team can keep up with. Queues that grow faster than they drain become first-in-never-out.

Consistency is harder than volume. The same position title means different things at different health systems. ICU-MED-3W at one facility might be a Medical-Surgical ICU on the third floor; at another it’s an outpatient infusion suite renamed in some old reorg and never updated in the VMS. Mapping that correctly takes institutional knowledge, or a feedback loop that captures it.

Rules evolve. A certification list changes. A new EMR rolls out. A floating policy gets renegotiated. When rules change, previously curated jobs at that program may or may not need re-curation, depending on which rules changed. The domain model has to support “re-curate this slice” as a first-class operation, not a manual cleanup.

Ground truth is delayed. The only honest signal that a curation was right is whether the right clinician ended up submitted, qualified, and placed. That feedback arrives days or weeks later. The audit signals we use day-to-day---auditor thumbs up, auditor thumbs down with a note---are proxies, calibrated against the placement signal over longer windows.

Automation still has to handle these cases. Defining them in the data model gave us something consistent for both people and agents to work with.

Automate the actor, not the model

The autocuration work showed why that order matters.

When we built v1 of the autocuration agent, we treated the LLM as the unit of work and the curation result as something the model produced. We sent the raw job in, asked for a fully curated record out, and hoped the schema would hold. It worked often enough to be misleading. When it failed, the failures weren’t localizable: the agent and the domain model were entangled inside one prompt.

When we rebuilt for v2, we inverted the relationship. The domain model came first---the curation form, the rule models, the validation contract, the state machine, sized and structured around the work itself, independent of who or what was doing it. The agent got bolted onto that model by exposing the form as a set of tools. The agent operates the form; the form enforces the model. The model exists whether or not an agent is in the room.

That gave us three practical advantages.

One validation contract covers both actors. The error a human sees when they save an invalid combination is the same error the agent sees when it submits. There is no “what AI is allowed to do” rulebook that drifts from “what the curator UI lets a human do.” Constraints live in one place, owned by the engineers who own curation, not by the engineers who own the agent.

Auditing is uniform. A curation produced by an agent and a curation produced by a human are the same kind of artifact in the database. The audit tooling, the feedback queue, the eval harness, the metrics dashboard---all of them treat both populations as members of the same set, with an auto_transitioned flag to slice when needed. We aren’t maintaining two pipelines of analytics for the same domain.

New agents can use the existing form, validations, and state machine. We can add automation without building a separate set of curation rules, which is why we’re reusing the form pattern elsewhere.