Text-only companion

Customer Experience Research for Government Leaders

Mirrors the course slide deck, chapter by chapter, including the WOSC case you work all day.

This page holds the same content as the slide presentation, as plain, structured text. It works with screen readers and keyboard navigation, reflows to any text size or zoom, and follows your device’s light or dark setting.

Getting around: use your screen reader’s heading commands, or the list below, to jump between sections. Every section is a heading. Under each section title, a short line names the slides it mirrors, so you can follow along in class. Where a slide runs a class poll, this page says “share.” Where the speaker notes carry teaching worth keeping, this page includes them, marked as notes.

Observing your data landscape

Slides 1–20.

Welcome! Bring your experience to the table. CX research will draw on your experience and extend it to help you better understand the customer experience. It extends and pairs with the crafts and disciplines you practice every day. Tell us about your work—what brings you here today?

Today, we’ll learn to hear what our customers experience and extend what our systems report. You’ll run a real analysis start to finish:

Our built case

A fictional agency service, built to be as messy as the real thing. The deeper you look, the better your questions get. Treat your first read and first understanding of the data as a draft. Themes evolve as you look deeper.

From the speaker notes: you may be surprised by what you find in it. When better questions change your findings, that’s the work going right.

The question we center the day on

Who gives up on our service, and how would we know?

In our work, people can’t take their business elsewhere. When folks give up, they disappear from our data. Satisfaction scores and completion rates are often made of the people who made it through and kept talking to us. Washington gives people real doors: ombuds offices, complaint lines, story portals. Yes, some of the people who cannot make it through our processes use them.

From the speaker notes: those conversations often become casework, and they don’t show up in reports that capture satisfaction.

In the numbers, silence can read as success. How many of the quietly defeated, very busy, and overwhelmed do not walk through those open doors? They’re people who disappear from our data.

Our customer landscapes

We begin our research by observing what we already have and noticing what we don’t. You took a first look at ways your agency hears from customers in your pre-work, your CX Research Working File.

Open your CX Research Working File. Go to Part 1. Prepare to share two things with us:

Share: a first look at what we hold.

From the speaker notes: if your honest answer is “we don’t really measure anything”—that’s ok! That’s your first finding.

Let’s explore a dashboard

In class, we explore a permit-program performance dashboard together.

From the speaker notes: this dashboard is good. It does its job. It gives us process information, and process directly impacts customer experience. But it differs from something that captures the customer’s experience. It can’t tell us what 14 days of silence feels like. No row in this dashboard can capture the person who read the requirements and never applied at all.

Share: sorting what we hold—CX or performance data?

Sorting what we hold

Engagement output is rich CX qualitative data

Ethical use of engagement output

Three types of conversations don’t enter data directly. Pause on things said:

Relationships are outcomes, not instruments.

From the speaker notes: judgment about what to convert is itself a research skill. Building trust, fostering community advocacy, and creating a safe space for marginalized voices is the reason we engage. Forcing these complex human realities into a dashboard column misrepresents what was shared. Your Working File has suggestions for elevating these stories outside the data set: you may return to a participant with a specific proposal—a heavily scrubbed, anonymized version of the story, or permission to build a composite story from a group. You can invite community advocates into your process as co-designers.

Share: the gap list opens. The framing question: what do we need to know about our customers’ experience that our data does not capture?

Back to top

Mapping the customer landscape

Slides 21–31.

You’ve named what you have and what you think you’re missing. Our next question is who. Who are we missing?

Four kinds of customers

We’ll use the customer categories named in Executive Order 25-06.

The four customer types, with one example of each
Customer typeExampleWhy it fits
Direct service recipientA parent applying to ESD for Paid Family and Medical LeaveThey receive the benefit themselves.
Regulated entityA city wastewater treatment plant renewing its NPDES discharge permit with EcologyOversight authority defines the relationship. They cannot opt out.
IntermediaryAn attorney contacting L&I on behalf of an injured workerThey act for someone else, and they navigate the process far more often than the person they represent.
Internal customerAn agency training manager coordinating staff enrollment in DES trainingA state employee who depends on another agency’s service to do their own job.

From the speaker notes: the same person can move between these categories depending on which transaction you are mapping. Watch the intermediary row in particular—that attorney may handle two hundred claims a year while their client handles one. They see friction the worker will never notice, and they have built workarounds you have never heard of. Intermediaries can be a rich source of process insight, and the easiest group to leave out of a research plan. Always name which of the four you are mapping before you start.

Who is your service for? Name groups specifically enough that you could go find them. In the CX Research Working File, Part 3, read one example in each category, then name your own.

Share: your customers.

Who’s missing from the map?

Search for them

Share: who joined your map?

From the speaker notes: your map just improved. You ran a failure-mode prompt—you asked your own draft how it could be wrong. That’s an important rigor technique; use it on any draft. Those doing engagement work often hold the best knowledge of who’s missing—that knowledge is data.

Back to top

Framing the research

Slides 32–45. A break falls at slide 38, and another at slide 45.

You have an idea of what you have, what you might be missing, who your service is for, and who you might have to search harder to find. The next question is what you would ask to find out what would improve your customers’ experience—and that’s research framing.

The case: a credential you can’t work without

Washington Office of Service Credentialing (WOSC).

WOSC wants to improve how people get and keep the CFST credential. Where should they focus first?

Share: what research question would you frame to find out where to focus?

Two tests of any research question

The equity test has two halves:

From the speaker notes, examples of running the tests: “Overall, how satisfied are people with the CFST credentialing process?” is aggregate—the answer gives us nothing to act on directly, and it cannot tell the people within our reach apart. “Where in the CFST application process do delays and rework happen most?” is a process question—there’s no person in it to reach or resolve, which is what makes it right for a process decision. “What should we ask CFST applicants on a survey?” names the instrument before naming what we’re trying to learn, and narrows who we can hear from before we begin. “How many people are we reaching with CFST outreach?” can only count people the outreach already connected with—its unit is agency activity, not experience.

Share: let’s run a few candidates through that test. Then open the reveal.

Reveal the question we’ll carry
Where do people get stuck in getting and keeping this credential, and are those places the same for everyone?

From the speaker notes: reach is carried by “get stuck” and “keeping.” Someone stuck hasn’t finished, so people mid-process and people who stopped are inside the question—and so are people who had the credential and lost it. Resolution is carried by “the same for everyone?” That clause forces the answer to divide, so we can see differences inside groups.

Next-level framing

We have our organizing question at whole-service altitude. Now we’ll drop one level. You will each draft your own research question on the case and choose a method WOSC resources can carry.

Where does WOSC stand right now? WOSC has six people who can work on research, no budget line to fund research, and unread feedback in several channels. People apply for the CFST credential, and their feedback arrives through several channels: survey answers, emails to the program inbox, phone-line notes, submissions from community organizations. None of it has been analyzed yet.

Breakout: Research Question Spotlight

You’ve received an invitation in your direct chat. Open the link, type your first name, and claim a role. The activity runs write · read · return · reveal · talk:

Share: what research question did your group choose to advance?

From the speaker notes: when the question needs data nobody has, collecting something new is the right answer—and that’s not a call most people can make alone. It takes a decision at your agency. We’ll come back to how to name those decisions before the day ends.

Make it yours

CX Research Working File, Part 4: your research question. The key points will help you; the first block is a filled example. Draft one research question for a real decision your own agency faces: the service you named this morning. Self-test it with the two tests and name a method that fits your resources.

Back to top

The customer voice

Slides 46–52. A break falls at slide 52.

We have decided what to look for; now we need a tool that helps us find out where to look. A journey map is one place: it’s the customer’s side of the process, and it gives our question a place to land.

Are we following the paperwork or the person?

From the speaker notes: you might have existing customer-voice tools—Voice of the Customer, Critical To Quality criteria—that translate customer input into process requirements. That is the job of those tools, and they do it well. CX research holds the experience in the customer’s own words and sequence, before and beyond those translations. We are not workflow mapping. Along the way, we might discover that efficient workflows produce poor experiences.

CX research follows the customer.

We always note: evidence or inference

Both belong on the map. An “I” is not a weak entry; it reveals a question.

From the speaker notes: the question tells us where to dig. But that’s not all we map. The map holds the whole experience—what works belongs on it too, or we won’t know what to protect.

The seven voices and the co-built map

In class, seven customer voices are read aloud—seven customers along seven stages of a journey. The voices are already on the shared journey map at the stages we found them, word for word. Two stages have no voice. Together, on screen, we add what we heard, mark each entry evidence or inference, and build the map page.

You’ve put together the instrument

From the speaker notes: next, WOSC will hand us what customers actually said. You’ll read it fresh first—what people are saying, before anything else. Our question and our map both come back before the day is done.

Back to top

Affinity tagging: thematic mapping

Slides 53–61. A break falls at slide 61.

Breakouts ahead: 29 voices become a thematic map. You’ll run a real analysis, start to finish.

From the speaker notes: last year 1,847 people applied for the CFST credential. 72 pieces of feedback reached WOSC across five channels. Here are 29 of them—a reading cut so you can actually read them. Before we can ask “who,” we need to know what people are saying. Themes first—then we test them.

Thematic mapping

We’ll read raw customer feedback and organize what it says into named themes. Our coding and refining will feed a map of what people experience, built from their own words. One item walks the path:

Why create a thematic map? Numbers alone can’t tell you why. Unstructured reading can’t scale and can’t be shared. Thematic mapping is the discipline between. The solo pass is called coding the data. Making the themes together is called theme development.

Today’s run

1,847 people applied → 72 pieces of feedback → 29 you’ll read.

A good first pass:

Breakouts ahead: check your private message for your link and your reader role. Open it. Claim your role with your first name. Code your batch on your own; we work here, in silence, for 14 minutes. When your items are done, hold. Your group will refine themes together in a breakout room.

Let’s check out the affinity map

Back together, we read the wall of every team’s themes. Which theme has the biggest count? Notice the themes that emerge in every batch.

Good work! You just carried out the steps of affinity mapping at a draft level. The themes are what the feedback says so far. When we return, we’ll ask one question of every theme on the wall.

Back to top

Qualitative disaggregation

Slides 62–99. A break falls at slide 99.

Qualitative disaggregation: sorting what people told us by who they are, then reading each group’s words.

For the next forty minutes, we’ll disaggregate our map. We’ll start by asking one question: does this hold for everyone in it, and does it hold for the same reason?

The director asked: who gives up on our service, and how would we know? Let’s see if we can find the answer.

Is that a pattern?

Five people. All long-tenured. All frustrated with the system. Let’s hear their voices again. These voices were coded with similar themes: OS-07, OS-20, OS-30, EM-01. Did anyone name a screen, a button, an upload?

The “system” they find unacceptable is not the portal. It is the silence after submission.

Now two more with similar themes: OS-42, twenty-nine years in the trade, and OS-32, twenty-four years old. Both stuck on the phone upload.

Now our themes have produced findings. A finding is a claim about a pattern—something you are prepared to say is true of a group. We started this run with a theme. We now have two findings. Letting a theme go for two sharper findings is analysis working.

From the speaker notes: first finding—the phone upload fails for users across all ages and experience levels. Second—notification doesn’t happen at key stages.

Check the denominator

Who fills out the hours verification form? First-timers only. The voices within one finding were all renewals—they never touched the first-time process.

Who you count is a design decision. Make it before you ever open the data. When you state the people who had a problem in your numerator, make sure people who could ever have had the problem are the denominator.

Getting it right protects the picture:

From the speaker notes: the gap is about 20 points, and the mistake makes the hours form look like a coin-flip problem when it’s actually the dominant first-timer experience.

Read for reasons

Let’s examine this theme. Count supports: sole proprietors have more problems with the hours form than anyone else. Does that theme hold for everyone in it? Yes. For the same reason? Read and find out.

Five sole proprietor cards are read aloud: OS-06, OS-09, OS-17, OS-25, OS-41. As you listen, write a note—what is the actual reason each person is stuck? If two different problems could be fixed by the same change, write the reason at a slightly broader level. It’s one.

Share: what is the reason each person is stuck?

Would you add any other reasons? Are we finished with this finding? How do you know if you’re finished? We propose a test.

The hold-back test

There are three cases in your map that nobody has read. As you hear them, take notes: does this batch add a reason that is not on our list? Then open the reveal.

Reveal what the held-back batch said

This time we heard a reason things went well. All sole proprietors in this batch gained or kept the certification. One reason? All of them had reachable signatories: someone who could and would verify hours.

What does this say about our theme? Sole proprietors don’t get stuck trying to figure out or use the forms. People with unreachable or unwilling signatories get stuck trying to verify hours.

We answered one saturation question: when did the last new thing appear? Commit → hold out → read → test. You wrote your reasons down and committed to them. Then you read held-out material and asked what it added. If you find new reasons, you’ve not reached saturation. Keep reading until you get hold-out batches with no new reasons.

Four saturation questions:

  1. Saturated on what?
  2. When did the last new thing appear?
  3. If we read more, would we learn something new?
  4. What could we never have learned from this data?

On your own data: today, I chose those three. On your data, you choose at random: about one in five, never fewer than five. Choose before you start reading. You have a job aid outlining the method in detail—the qualitative disaggregation job aid at method.sentient-learning.com (link leaves this site).

The language fold

Sometimes, people converge on a theme around language barrier with this corpus. OS-22 wrote in Spanish, and she withdrew. Her barrier was not language. The five severe cases we read were all in English. Reading for reasons, we find that this Spanish-speaking applicant had an unreachable signatory. She is one more voice of evidence that unreachable signatories prevent qualified people from getting and keeping this credential.

From the speaker notes: what put those people in trouble is how they were employed, whether they were paid in cash, and whether anyone kept records.

We apply reading discipline for this reason: we must name true causes so we can pursue true solutions. Naming the wrong cause wastes resources. It can confirm a story about a community that was never true. It leaves the real barrier in place.

From the speaker notes: don’t rush the reading. First reads are a draft. Second reads get closer. Third reads refine our themes into findings we can prove.

Write the finding as a statement

Write the signatory finding as a three-part statement of your own: the finding, what it rests on, what it does not tell us. Try it, then open the reveal.

Share: the finding as a statement.

Reveal what it could look like

Applicants are blocked when their signatory cannot or will not verify hours.

It rests on five severe cases, tested against three held-back cases.

It does not tell us how many people are affected.

From the speaker notes: “it doesn’t tell us how many people” is the half that makes everything else believable. If we can state the finding, say what it rests on, and say what it does not tell us, we are ready to put it in front of a director.

EM-05: not a finding. Escalate anyway.

EM-05 was placed on the map at 1:30 and is still there. Is it a finding?

Reading discipline and disaggregation give us: themes, findings, limitations, outliers. We now know what to do with each.

What the data could never tell you

What would this dataset never allow you to learn? PH-04 is here only because he picked up a phone. Everyone like him is nowhere. Nothing in the data would tell you—WOSC doesn’t capture data from customer calls. This was a story told by the person who answered the call.

CO-01: not a customer. It corroborates the sole-proprietor finding from a second channel. Triangulation: a stronger finding. But it is one advocacy-written submission. You cannot tally it alongside surveys. It is not one customer voice.

A move to consider: member checking. Take the finding back to the people it describes, or to an organization that works with them. Does it ring true? We cannot do it in a workshop. When you cannot either, say so in your write-up.

Saturation tells you what your data has run out of. It tells you nothing about who never entered into it.

What this method gives us, and what it does not

It gives us: defensible descriptions of how different people experience our service. Enough to inform a Customer Experience Improvement Plan. Enough to justify a fuller study.

It does not give us: statistics, causes, prevalence estimates, or claims about anyone we did not hear from.

Those limits are features, not weaknesses. The honest analyst names them out loud. Naming them is what makes everything else we said believable.

Back to top

How much of the world is on our wall?

Slides 100–109.

We’ll read metrics the agency already keeps to see how they can inform our findings or reveal missing customer voices. First, we zoom out:

Under four percent. And behind the 1,847: the people in no denominator at all.

Who can’t each number see?

Definitions tell us how each number is calculated. Answering this question about metrics we already have can tell us where to look for customer voices we haven’t heard. In class, we read the agency’s latest performance report definitions aloud, holding one question: who can’t this number see?

Of the numbers that tell us who the metrics can’t hear from, what might be the most important number? 12%—the only number with no explanation. Eligible credential holders who did not renew before expiration.

A trap. Walk into it on purpose.

Of the 72 who gave feedback, a bit over half were approved. Of all applicants, eighty percent were approved.

“Angry people complete surveys more often than happy people. That data overstates the problem.”

Among the people who reached us, we hear disproportionately from the unhappy. This is true. We’re not hearing enough about what makes the experience work well. Who else is missing from the data?

The people who never applied. The man with no email who said he’d think about it. The caller who asked if there was any other way. Not in the 72. Not in the 1,847. In no denominator. The second bias is bigger, and invisible: people who never became a row in the report.

The honest statement, both halves:

Among the people who reached us, we hear disproportionately from the unhappy.

We do not hear about the customer experience from the people who never applied.

Who gives up on our service, and how would we know? Every measure met target. From inside the agency, this looks like a year that went well. The director asked us to answer a big question. We can look to the data you coded for themes and patterns. Reading the definitions in our metrics helps us state what we don’t know more fully.

Back to top

Rank the findings

Slides 110–116.

We’re ready to take the next step: we need to rank our findings.

  1. Notification failure.
  2. Phone-upload failure.
  3. Unreachable signatory.
  4. Renewal contact.

For this course, we’ll use a two-step method. We’ll place our findings within an impact/feasibility matrix. Then we’ll rank in a way that supports the practice of targeted universalism. The universal goal: every qualified person can get and keep this credential. The equity pass asks if a finding leaves a group entirely unable to reach that goal, and gives it higher priority.

Impact and feasibility

On the matrix: notification and renewal contact sit at high impact and high feasibility. Phone upload is high impact but low feasibility. Signatory sits at low impact and low feasibility.

The four findings on the impact/feasibility matrix, and the initial rank
FindingImpactFeasibilityInitial rank
NotificationHighHigh1
Renewal contactHighHigh2
Phone uploadHighLow3
SignatoryLowLow4

From the speaker notes: high impact means it affects a lot of people or harms a smaller group in serious ways. The highest rank combines high impact and high feasibility—something we can definitely do that will make a big improvement. Why do you think signatory lands low/low?

The equity pass

Do any of our findings reveal a group shut out entirely from reaching the universal goal? The signatory. A structurally-situated group, excluded from the goal entirely: those with unreachable, unwilling, or disqualified signatories have no path forward. A complete barrier outranks a widespread inconvenience or a barrier it is possible to overcome.

Watch the order change:

The rank before and after the equity pass
RankImpact and feasibilityThen equity
1NotificationSignatory
2Renewal contactNotification
3Phone uploadRenewal contact
4SignatoryPhone upload

Impact and feasibility ask what does the most good in the easiest way. Equity asks good for whom? And who is left out of the goal entirely? Washington’s framework asks both, in that order.

From the speaker notes: under the state’s own equity principles, equity is not equality, and a complete systemic barrier outranks a widespread inconvenience. So it rises. Signatory first.

Back to top

The decision brief

Slides 117–126.

We’re ready to provide research input for a CX Improvement Plan. These ranked findings feed one. Next, we’ll write it up for a decision brief. We ranked findings against impact, effort, and equity; a decision brief tells a leader what we found and what to decide.

Three minutes. One sentence.

Ask leadership to approve a request based on our top-ranked finding: the unreachable signatory. What are you asking the leader to decide? Write a statement that states that clearly and concisely.

Check yourself: does your request require a yes or no answer? A goal is not a decision to make. What would leadership need to decide to scope, fund, or change?

Share: drop your request statement here. Then open the reveal.

Reveal a form the request could take
We ask you to decide whether WOSC will scope an alternative way for applicants to prove supervised hours when the original signatory is unreachable, unwilling, or unable to verify them.

Findings and recommendations stay apart

A decision brief moves from what we found to what we recommend. Found = evidence. Recommend = judgment. Keep them apart. A leader can disagree with your fix and keep your evidence.

Watch out for: a recommendation wearing a finding’s clothes.

“WOSC should replace its email notification vendor.”

Nothing in the feedback says a vendor is the cause. This is an inference presented as a recommendation.

Job aids to take home

You have a job aid to take home: a findings brief, annotated, and a blank template with the full five sections—the findings brief and template at brief.sentient-learning.com (link leaves this site).

If you want more on this kind of writing:

  1. Writing for Busy Decision Makers.
  2. Plain Language Writing and Design.
  3. Technical Writing for Public Sector Professionals.

Back to top

Keep the loop alive, and take it home

Slides 127–130.

One study answers one question once. What you need back at work is a way to keep answering it. A reporting cycle is not a feedback loop. A loop closes back to the customers it heard from.

Three things kill a feedback loop:

  1. No owner—it drops as soon as we get busy.
  2. Collection without closing the loop—asked, never answered.
  3. Channel blindness—every channel misses someone.

Bring this back to your own service. In your CX Research Working File, Part 5, complete the last block: name one decision, and who makes it. Come back and share with us:

Share: what question are you taking back to work? What will you stop treating as a finding?

Back to top

The close

Slides 131–138.

Did we discover some answers we can act on? Who gives up on our service, and how would we know?

  1. The sole proprietor stuck at a signature the process can’t reach.
  2. The renewer whose notice went to an inbox she never checks.
  3. The caller who asked if there was any other way, and was told no.

And you know why WOSC didn’t know: every one of them is invisible in numbers that all met target.

This morning, we couldn’t answer that. You ran a research instrument that can.

The instrument:

  1. A question you’ve tested.
  2. A way of reading what customers already tell you.
  3. A map that shows where you’re not hearing anyone at all.

You leave with a draft and a method, not a finished plan. You’re leaving with resources that can help you carry out CX research that supports your agency’s universal goals.

From the speaker notes: you leave with a draft and a method, not a finished plan—and that is the correct thing to leave with.

Take it to work!

  1. CX Research Working File—all five parts started.
  2. Your team’s affinity map.
  3. The disaggregation method job aid—the method job aid at method.sentient-learning.com (link leaves this site).
  4. The brief and the template—the findings brief at brief.sentient-learning.com (link leaves this site).

You did it!

Back to top