01 / The problem
Data is only useful
if people can find and understand it.
People search for data using business concepts. The tables and columns they need often carry technical names that do not explain what the information means. Curators bridge that gap by organizing and enriching the catalog so people can find, trust, and use its contents.
Glossary terms provide shared business definitions and connect those concepts to the underlying data. Adding a term to an asset or column is straightforward. Repeating that work across a large enterprise catalog, and keeping it current, is much harder.
The Data Curation Agent was designed to review assets and columns and suggest relevant glossary terms. Glossary curation was the starting point for an MVP, with additional curation capabilities planned to follow. We used a Build, Measure, Learn approach to get feedback early and refine the experience.
02 / Defining scope
A tool for scale
needed to start at scale.
The initial configuration flow asked users to select individual assets for the agent to review. That put a manual task at the beginning of an experience meant to reduce manual work. Curating a sales data source could mean selecting every asset inside it, then updating that selection as its contents changed.
The main change through our iteration process was to configure a scope instead. Users could target the data source itself, allowing subsequent runs to operate on the assets within that container without rebuilding an item-by-item selection.
Select the assets.
Assemble individual items for review and maintain that selection as the catalog changes.
Define the scope.
Target a container so later runs can include its current contents.
Broader scope introduced a tradeoff: users lost some of the context they had when choosing individual assets. I added a summary to the configuration panel to make the selected scope understandable before starting the agent. Asset totals and breakdowns by domain, owner, and asset type showed what was included.
03 / Review at scale
Generating suggestions
was only half the task.
An agent could produce more suggestions than a steward could practically review one at a time. Confidence thresholds and bulk approval were part of the design from the beginning, helping users work through recommendations at a useful scale.
At the individual asset level, the review view separates asset and column suggestions. Each recommendation pairs a glossary term with confidence, reasoning, and review status, with controls to approve or reject it. Filters help narrow the list by confidence and status.
Bulk approval lets users choose a confidence threshold and see how many suggestions it would approve before confirming. It offers a faster path through the review while leaving that decision with the steward.
04 / Applying changes
Review over time.
Apply with intent.
Approval and application were deliberately separate. Reviewing a large run could take time. Users needed to work through its suggestions without every approval immediately modifying the catalog.
Once review was complete, they could apply the approved changes together. This kept the changes associated with a single agent run for auditing and made the catalog update an explicit confirmation, with another opportunity to review before committing.
Decide what belongs.
Review suggestions over time and mark the recommendations that should be accepted.
Commit the changes.
Deliberately update the catalog with approved changes associated with the run.
The run overview brings those stages together: suggestions are organized by asset, progress and status show where review stands, and a distinct Apply changes action controls the final update.
05 / The MVP direction
Scale the work.
Keep the decision deliberate.
The project was in design and prototyping and had not entered development when I left. The MVP direction was user-initiated runs, human review, and explicit application of approved glossary suggestions. Scheduled runs and automatic curation were considered follow-on capabilities after an initial release.
The design addressed scale at each stage: scope defined the work, bulk actions supported review, and separate application preserved an intentional catalog update. The next questions were whether this reduced the review burden in practice and how well confidence and reasoning supported stewards' decisions.