A User-Centered Approach to Developing Metadata

16 Apr 2012

A few weeks ago I was lucky enough to volunteer at Information Architecture Summit, where I saw two fantastic presentations: Karen McGrane's Adapting Ourselves to Adaptive Content, and Adam Ungstad's Better Cross-Channel Experiences with Metadata.

The takeaway for me from both presentations was the importance of metadata in creating viable cross-channel experiences. In her talk, Karen advocates for "demystifying metadata" in digital publishing so that you can "Create Once, Publish Everywhere" (that's NPR's motto in her examples). Adam's entire presentation hit similar notes: "metadata enables consistency, context, and interoperability in cross-channel experiences."

True dat. Metadata needs to have a central role in design and planning of user experiences. Which brings me to my question (and, conveniently for you, one possible answer): how can we apply user-centered design principles to metadata? Too often we get stuck with legacy metadata that's become clumsy over time, or is too focused on satisfying the business' needs rather than the customer's. That shouldn't be.

The Process

I think the answer is to combine understanding of user tasks and of the conceptual world they operate in (their personal ontologies, say). You need to capture the functional requirements of your schema through lots of interviews, contextual inquiry, and document analysis to ultimately build a mental model. You also need to figure out a way to understand the ontology in your users' minds, i.e. the entities/content-types and the relationships between them.

The remainder of this post will outline an approach I've come up with, and which I presented at IA Summit's poster session. You can also download the pdf of my poster.

Phase 1: Generative User Research

1.1 Understand the Tasks Your Metadata Will Be Used For. I may not need to say more than that... but I will! Your tools for understanding this are probably all familiar: interviewing, contextual inquiry, document analysis, survey, whatever you're comfortable with. Just make sure you understand what people are trying to do with your system. What you shouldn't do is get caught creating metadata elements to describe the items in your system in a vacuum, without context. It's easy to think of tons of ways to describe a given content type (or entity, or model, or whatever word makes sense in your world) in your system... but impose some constraints on yourself by doing your research. If you can, build a mental model.

1.2 Draw an Entity-Relationship Diagram or a Concept Map. You need an expert to tell you how they think about these things. My thinking here is based on Emma Tonkin's excellent Dublin Core paper called Multilayered Paper Prototyping for User Concept Modeling. You want to try to expose the entities and relationships your users hold in their own minds, however you can do that. In the paper Tonkin talks about having a user draw a base layer of entities on a piece of paper, then using additional layers of transparencies on top of that for different types of relationships, adding and subtracting from the map, etc. I've tried this in simplified ways on a whiteboard, but I am still looking to try the full version she describes. In any case the point is to get a topic map straight out of your user's head.

Phase 2: Draft a taxonomy

2.1 Merge the Above Two Steps in a Proposed Taxonomy. Now you're ready to make something. Ideally you'll have a mental model you can work with, and all you'll need to do is start proposing metadata elements suggested by your users' concept map, directly underneath the task "towers" in the mental model. Here's what I mean:

I included both metadata and possible site features. See how I lined up the Population metadata element under the "Prioritize Countries" task? That's because I know from interviewing that my users tend to prioritize their work according to country size. So here I'm trying to come up with specific metadata elements that I know will support specific tasks. That's the essence of the process I'm describing here.

Most of your work will happen in this step. Keep going until you feel your metadata supports everything the user is trying to do.

Phase 3: Evaluate the Taxonomy

With the user research complete and a draft taxonomy proposed, you need to validate your design, just as you would with any other piece of UX work. Here's how I do it.

3.1 Get An Expert's Opinion. I've done this kind of work once in the legal industry, and once in medicine. Both times I sometimes had to put terms into the taxonomy that I only had a very basic understanding of. However, I had plenty of eager co-conspirators at the organizations where I did the work, and I could enlist their help assessing the general accuracy of the metadata I had created. You can fix a lot of mistakes with a quick expert review.

3.2 Survey Many Users. You can put an interesting twist on expert review by surveying your users. During one project I asked people to think about various metadata elements I had proposed, and rate their usefulness. I got some great input that way. It's not necessarily bulletproof, empirically, but like most social science work it gives you something to go on.

Here's what's going on in the image below, which is a graph of the answers to one single survey question about one single metadata element. The question was "If you were looking for a document to help you with your case, how useful would it be to know the Case ID # that a document pertains to?" On the X axis is a Likert scale where on the left is "not useful" and the right side is "really useful". The two colors represent two user groups. You can see that one group really doesn't think it would be useful at all, where the other thinks it could at least be somewhat helpful. Nobody thinks it's really useful :) But having a survey lets me know that and focus on Case ID # as an administrative metadata element rather than one that the user sees.

Conclusion

I hope this has given you some new ideas. Naturally this process is iterative, creative, and not always possible in full. If you remember nothing else, remember that metadata has been, and will always be, a central part of user experience for large websites---and if you call yourself a UX pro, you need to understand how to make it user-centered. Don't be satisfied with legacy metadata and don't build your taxonomy in a vacuum! Tie it to user research of both qualitative and quantitative types. You'll be glad you did... and please tweet me at @aktaplin if you have any feedback on anything I've said here!

This post is based on a poster I presented at IA Summit 2012, which you can download here.