Scoring an attribute

Scoring reads one attribute of every entity in a collection and matches it to one of a set of levels or categories that you describe. For example, it can read a free-text “Education” attribute and match each entity to a “Highest degree” level: bachelor’s, master’s or doctorate. Use it when the answer is a judgment rather than a value to look up, for example:

  • which suppliers are far enough along to shortlist
  • how mature each company’s disclosure is
  • whether a program is early, established or winding down

Every entity is judged against the same levels, so the results are consistent across the collection.

Asking for a score

Ask your assistant in your conversation. There’s no form for this. Tell it:

  • which collection to score
  • which attribute to read
  • what the levels or categories are, in your own words (levels in order)

Your assistant turns this into a rubric, like the one below. Behind the scenes, each entity gets a score along your levels, in the order you listed them, and you’re shown the level it lands closest to. That’s why the order matters, and why the result also says how confident the judge, a model separate from your assistant, was: a clear fit is confident, and an entity that sits between two levels isn’t.

Check the levels before it runs: every entity will be judged against them, one at a time, based on what the attribute holds about it. A level you meant differently is easier to fix now than to spot later in the results. 

YOU ASK THE ASSISTANT

“score these companies on Disclosure: no public figures, figures
 published but not audited, audited figures for last year, or
 audited figures for this year.”

THE ASSISTANT DRAFTS

attribute   Disclosure
levels      No public figures
            Figures published, not audited
            Audited figures, prior year
            Audited figures, current year

AND GETS THIS RESULT FOR YOU

entity      what it read               level                           confidence
------      ------------               -----                           ----------
Northwind   audited 2026 statement     Audited figures, current year   0.93
Meridian    annual review, no auditor  Figures published, not audited  0.88
Brightline  audited 2025 only          Audited figures, prior year     0.81
Kestrel     nothing recorded           not scored

For each entity you get the level it matched, given as that level’s description, along with what the judge read to decide and how confident it was. You also see how the collection is spread across the levels, which is a good first thing to check. If nearly everything lands on one level, the levels probably need different cut-off points: rewrite the rubric and ask again.

Writing a good rubric

A few things make scoring more reliable:

  • Keep to one dimension. The levels should all describe the same thing, in order, like the four stages of disclosure above. To judge two things, score twice.
  • Describe situations, not degrees. Each level is judged on its own, so it has to make sense by itself. “Discloses audited figures for the current year” is a level. “Good” or “moderately transparent” is not, because it doesn’t say what the entity actually does.
  • Say where each level ends. Use 2 to 10 levels; 3 to 5 is usually plenty. A gap between two levels is a place where similar entities can land on different sides for no clear reason.
  • Describe the attribute. If the attribute has guidance saying what belongs in it, the judge uses that too. If it doesn’t, scoring still runs but tells you the attribute has no guidance.

What it reads

Scoring reads one attribute, and only what’s recorded in it. It doesn’t do any research or look at other attributes. An entity with nothing recorded in that attribute is marked as not scored rather than guessed at.

This means results are repeatable, and you can open the cells behind any placement to see exactly what the judge read. Large collections are scored a page at a time, so you can stop after the first page if the rubric isn’t working.

Use the confidence figure: review the least confident placements first. They usually point to a vague level in your rubric or a thin cell in your data.

Keeping the results

By default, scoring doesn’t save anything. The placements are shown in your conversation only, and running the same rubric again judges afresh from whatever the attribute holds at that time.

To keep them, ask your assistant to record the levels as a new attribute on the collection, and give it a name: “keep those as Disclosure level.” They are stored as claims like any other, showing which assistant judged them and when. A later run adds a second claim beside the first instead of replacing it, so you can see what changed and when. 

attribute   Disclosure level

entity      what the cell holds
------      -------------------
Northwind   Audited figures, current year
Meridian    Figures published, not audited
Brightline  Audited figures, prior year
Kestrel     not scored, so nothing recorded

Once saved, a level can be used when filling a collection: “suppliers whose Disclosure level is audited figures” becomes a simple rule that checks what you already decided, instead of a new judgment.