Most MSPs that turn on AI triage point it at a ticket history they’d never hand to a new technician. Years of category drift live in that history. “Password Reset”, “PW reset” and “Login issue” sit as three separate types. One board belongs to a client who left long ago, and close notes read “fixed” and nothing more.
With each new client and PSA upgrade, successive service managers added labels, and you never had enough time away from the queue to retire the old ones.
That drift used to cost you a messy report. With AI in the queue it costs more, because a triage model classifies new tickets by what it learned from old ones. Feeding it your history is like training a new hire from a filing cabinet that four office managers relabeled and nobody purged: they file exactly the way the cabinet taught them.
The AI won’t ask what “Misc 2” means. It will get very confident about it. Your technicians correct its suggestions a few times, stop reading them, and you quietly shelve the rollout.
In a Gartner survey of 782 infrastructure and operations leaders, run in November and December 2025, 38% named poor data quality or limited data availability as a direct cause of AI project failure.
Clean PSA data before using AI by auditing what you have, consolidating ticket types and boards, fixing close notes, and validating before the AI goes live.
Step 1: Audit your PSA ticket data
Export the originals, untouched, before you change a single record, so every later fix can be checked against what was there.
Then decide how far back to look. A recent window that reflects your current clients, tools and staff teaches an AI more than the full archive, because older tickets describe retired products, departed clients and category lists you already abandoned.
Before any AI triage goes live, the quality of its decisions tracks the quality of the fields it keys off, so audit those first:
- Ticket type and subtype (or issue and sub-issue type)
- Board or queue
- Status, including every “closed” variant
- Close or resolution notes
- Contacts and companies on each ticket
Duplicates come next. Monitoring alerts that auto-create tickets, emails sent twice, and one client living as “ABC Technologies” and “ABC Technologies Inc.” all inflate counts the AI will read as patterns. Native merge tools handle most of it, though they differ on what carries over, so test one merge before running a batch.
Handle missing values one field at a time. Fill a blank when the ticket text makes the answer obvious, flag it when a technician who knows the client should decide, and exclude the ticket from what the model reads when no one can tell. Leave outliers alone until you understand them, because a week of lockouts at one client after an MFA rollout is a real signal.
Three measurements tell you the data isn’t ready yet:
You set the bar for each, based on what your senior techs would call usable. Write those numbers down, because you will measure against them after cleanup and every quarter after that.
Step 2: Consolidate ticket types, subtypes and boards
Collapse every variant label into one controlled list, with each real kind of work appearing exactly once. Drift mostly produces near-duplicates, so most of the job is merging.
Type, subtype and item lists pile up redundancies, so keep issue types broad and push the specifics to tags. Broad types are the ones every technician picks the same way.
Here is a typical before and after for account access work:
Boards follow the same logic. Keep a board when its work routes, bills or reports differently, and fold back the one that exists because a client asked for it in 2019.
How to change categories without breaking reports
Build an old-to-new mapping table before you touch the PSA: every existing value, its new home, and the person who signed off. That table is your rollback plan and the key anyone needs to read last year’s reports.
Then remap one well-understood board, let it run for a couple of weeks, and check two things:
- Workflow rules: List every rule that triggers on a value you are retiring. Rules keyed to a status or priority you inactivate become orphaned: they stop firing and can still count toward your PSA’s workflow-rule limit.
- SLA and historical reporting: Confirm SLA assignments and dashboards still group the right tickets, and that old reports still run against old labels.
Inactivate retired values instead of deleting them, so historical tickets keep the labels they closed under.
Step 3: Fix your close notes before the AI learns from them
Close notes record how your team resolves work: triage uses the summary to classify a ticket, while tools that suggest fixes or build a playbook use the resolution. With a history full of “done”, you have no resolution steps to supply; document the fix and scrub credentials and client PII before ingestion.
Thin notes are a process problem. When you leave the close screen without required fields, a technician with nine tickets waiting writes nothing; you can fix that faster with a required field and a short template than with another reminder email.
What a usable close note contains
A usable close note says what was wrong, what was done, and how the tech knew it worked. Compare two notes on the same lockout:
- Bad: “Reset done, closed.”
- Good: “User locked out of Microsoft 365 after repeated failed sign-ins from a replaced phone. Verified identity by callback to the number on file, per client policy. Unlocked the account, cleared the old device session, user signed in and confirmed. Cause: saved password on old phone.”
Score historical notes with one simple rule: a point each for a stated symptom, the steps taken, and confirmation that it worked. Notes scoring 2 or 3 go into what the AI reads. Notes scoring 0 or 1 stay in the PSA for audit and reporting, outside the AI’s view.
For new tickets, add a close template borrowed from the Knowledge-Centered Service article structure (issue, environment, resolution, cause if known) and make resolution required at close.
How to remove credentials and client PII from ticket notes
Ticket notes collect secrets because troubleshooting is urgent and the note field is right there. Temporary passwords, MFA backup codes, Wi-Fi keys and a user’s date of birth from an HR request all end up in it, and everything in the notes travels with the export. Scrub before any AI tool ingests them.
Run two passes on the export. A secret scanner such as Gitleaks or TruffleHog finds passwords, tokens and keys, and a PII detector such as Microsoft Presidio finds names, phone numbers and government IDs. Neither catches everything, so someone reviews the hits and spot-checks the rest. Some PSAs work in place: HaloPSA’s PII data flagging lets permitted agents scan tickets, review flagged data and redact it. Rotate any live credential you find as well as redacting it, and if a client falls under HIPAA or GDPR, check what your agreement with them allows before their ticket text reaches a third-party AI tool.
Step 4: Validate the cleaned data before the AI goes live
Check the cleaned data with four plain tests before the AI reads a single record:
- Record counts: Total tickets equal the original export minus the merges and exclusions you logged.
- Category totals: The old labels that mapped to each new type add up to that type’s new count; a shortfall means a label fell through the mapping table.
- Required fields: Type, subtype, board and close note are filled on every in-scope ticket, or the ticket is deliberately excluded.
- Sample review: Cleaned tickets from every client and board get read by the technicians who work those clients.
That last check catches what no count will, like the client whose “onboarding” tickets were offboarding all along. Get those technicians to sign off before go-live, since they have to trust the output.
How do you keep PSA data clean after the AI goes live?
Keep PSA data clean by making the right entry the easy one and checking on a schedule. Use three controls to catch drift before it builds up again:
- Required fields at close: Require type, subtype and a resolution note before closing a ticket, including tickets created through integrations, whose API path can skip rules the ticket form enforces.
- A locked type list with one owner: One named person adds, merges or retires categories through the mapping table, and everyone else submits requests.
- Scheduled audits: Re-run the Step 1 measurements quarterly against the thresholds you wrote down.
Treat a rising “Other” share as a request for a new category, and have the list owner answer it within the month.
How does managed automation use your cleaned ticket history?
Managed automation reads your ticket history to decide how each client’s routine requests get handled, so your cleanup carries straight into the work. Fixify is a managed AI automation solution for MSPs: it builds natural-language playbooks for each client from your existing documentation and historical tickets, and Fixify-employed human IT analysts supervise the automation 24/7. A request lands in your PSA, automation works the routine steps, an analyst steps in when the automation hits a roadblock or something requires a judgment call, and each step is updated in the PSA.
Cleanup still matters. Consistent ticket types and readable close notes give each playbook an accurate record of how your team resolves that client’s work, including the approval quirks that never made it into documentation.
Start with a 90-day audit of your own tickets
Find out where your gaps sit before you remap a single board. Use your category mix and close-note quality to decide whether Step 2 or Step 3 deserves your first week, and compare your queue with a benchmark to see how far off normal it runs.
Run the free IT Help Desk Ticket Snapshot on your own data: upload 90 days of CSV ticket data and receive a customized benchmarking report in 48 hours. The 90 days is what the report reads; your cleanup window stays your call.






