AI-powered Research Repository at LinkedIn (a 4+ year journey)

The UXR team at LinkedIn tried Airtable, EnjoyHQ, and, most recently, Glean when building a research repository. How did they tackle tool integration and introduce an AI-powered Research Repository to streamline UX research? Discover what worked, what didn’t, and how PMs & Designers fell in love with the solution.

“We turned scattered insights and endless documents into a research goldmine!”

Marieke McCloskey (UXR Director at LinkedIn) and Brandi Amm (ReOps manager at LinkedIn) shared their journey and where they are now during a UXinsight Think Tank session. They discussed the ongoing challenge of making it work even better for their UX researchers.

Watch the recording

Watch the recording of the session or read the session notes below.

Session notes (shortened)

The text displays portions of the session transcripts. For a complete conversation, please refer to the video.

Marieke – My name is Marieke McCloskey. At LinkedIn, I lead what we call our foundation’s research team. It’s one of our three research teams, and my team looks across our whole ecosystem of products. So everything LinkedIn does, making sure that it’s a better experience for more users. And there’s three pillars on my team:

• I have a horizontal research team: things like growth, trust, and equity.
• I also have a UX measurement team that we just established this year looking at how do we define quality?
• And the last pillar is our research programs: a lot of rapid research programs, but also things like our research repository which is why I am so excited to get to talk to you about what we’ve done here.

I’m joined by my colleague Brandi Amm, who has played and continues to play an instrumental part in this initiative. Thus, there is a true collaboration between our research team and our research ops team.

How does the LinkedIn Insights Hub work?

We call our research repository Insights Hub. [Demonstrating the Hub] As you can see here, the person searching it typed in a question. What research do we have? You can type in any topic or any question. It summarizes research from across any data source you tell it to. At the end, it’ll show you a summary and all the references.

So our repository is built on Glean. I cannot take credit for all the amazingness that is Glean. It’s a knowledge management tool we use across LinkedIn’s company. It really has replaced our intranet. It has a conversational UI interface. You can just search for it. It also pushes content. I can log in. It’s like my browser homepage. When I show up at work in the morning, it tells me things I might be interested in and want to know.

We’re a global company with a wide variety of work happening across different clients. We also have information in so many different places. Some information is in Google Drive and the Google products. We have information in SharePoint, and many teams use Confluence to sort information. One of the amazing things that Glean does is that it lets you search across all of it.

One of the things you can do in Glean is create a custom app, a no-code app. You can tell what you want to search for and how you want to present information. So our repository is basically a custom Glean app.

The box on the right contains the information that we have told this custom Glean app to include and how we want the information presented to people. We specifically tell it how to act: If a question is asked a certain way, interpret it this way and ask more questions. It’s amazing if users can get their answers from a repository like this, but we also want them to know where it came from, especially if they want to look up more.

“It’s also great to just gut-check what it’s telling you. I’m still a little skeptical about AI.”

Brandi spent a lot of time talking to folks across the company, trying to figure out how other people have built the Glean apps. One of the amazing things is that we [the UXR team] were set up well. Our reports are pretty structured and templatised, which really helps the Glean app know what we’re talking about and find the right information.

We make sure every research report has an executive summary. We know what questions we answered in the research, which helps it tie back to the questions that maybe a PM somewhere is asking. We also include the primary job-to-be-done, helping us understand or uncover more of what audience it’s related to. You can imagine it as metadata on the report.

How is the Insights Hub received by users?

We launched this version of our repository in April. Since then, we have over 1000 unique queries. So unique questions asked in just the first six months!

Everyone has access to our repository, which we have marketed and promoted primarily in our work. Most people who have used it are designers, product managers, and UX researchers. But we’ve also seen engineers, marketers, product marketing managers, and data scientists use it. (That one makes me super happy because I love working with them.)

And people love it. Someone even told us last month it changed their life, which is great feedback to hear. It’s also given researchers so much time back. We used to get pinged all the time with questions like, what do we know about job seekers? As you can imagine, LinkedIn knows a ton about job seekers.

So now, a PM can go to our Insights Hub, ask a question, get a summary of what we know, and find out where to dig in. They still will often go to a researcher but ask much more thoughtful, deeper questions, which is great. Whether or not to even consider conducting new research, we seem just one level deeper than I want this custom study done.

The journey

Brandi – I am a research program manager here at LinkedIn on the user experience research team. I’ve been with LinkedIn for about six years. I started as a research coordinator, and I’ve done everything from recruitment to data governance, tools and infrastructure. So really just lived my life in the operations world.

My charter currently covers high-priority initiatives, so I’m making sure that I’m enabling operational processes for high-priority initiatives across our UX organization. I’m working on our self-serve insights program, including our insights repository. I’ll go into that today and talk a little bit about our journey and where we’ve been with our insights repository.

It started with project-tracking

We’ll start way back in 2018. So even before I started here at LinkedIn we were just a small research organization. I think when I started, we had around 15 researchers. Maybe a little bit more and I joined as a research coordinator.

One of the main problems that we were facing at the time was that we had so much different work in so many different areas. We had been doing research for about three to four years at this point, and we were starting to collect quite a bit of assets and research data.

At the time, we were looking for a way to organize the work. We had work in various documents: videos over here, documents over there. So, we needed a way to track this across the organisation.

Our first use case overall wasn’t necessarily an insights repository, but more of a project-tracking solution and a research assets library. Our goal at this point was to build processes for streaming or UXR work and storing assets. Additionally, we needed a way to manage projects and see what we were doing across the organization.

Operationally, we needed to manage things like budget and logistics. But our UXR leaders were also starting to recognize that we needed to be able to see the work that we were doing. We did a full tool exploration just as any operations team would do.

We ultimately landed on Airtable, which if you haven’t used it, is a great project-tracking tool. We actually still use it to this day. The great thing about Airtable is it’s given us a way to organize a lot of this data. For example the logistical things that we face with research and budgeting how the life of a research project might go. It helps to inform things like prioritization and planning.

“Airtable is great for project tracking. But it doesn’t quite do very well at connecting the dots across different lines of business.”

The need to identify patterns and themes

So we started to recognize that we have all this amazing work. We wanted to influence our strategy and our vision, but it was really hard for us to understand some of those central themes and patterns that we were seeing across different populations and services.

At this point (around 2021), we started to explore new tools. We had identified the need for a tool outside of asset storage. Then, we wanted to identify patterns and themes across our work and start to look at them holistically.

Additionally, we, around 2022, wanted to make sure as well that we were able to share some of that work. As said previously Airtable is a tool that we use for project tracking, but not everybody has access to it. We wanted to find a tool that could allow for more discovery.

We recognized that our work is complex. There are so many different things that you have to consider. We started to look into hiring a research librarian to do that work for us.

So, at the same time that we started exploring new research repository tools, we also hired a librarian who helped us focus on some of those administrative workflows and templates and develop a taxonomy that would give us a foundation for what we have right now.

In 2021, we onboarded our tool in EnjoyHQ. Like Marieke had said previously, we have things in Google Docs, Slides, Confluence, SharePoint, all over the place. And we really needed to get that into one tool. So this wasn’t something that was done overnight. Fortunately, we had this amazing librarian who helped us to organize all of our work, organize our structure, and get us into EnjoyHQ.

How can we use AI in our work?

In 2023, AI started to become a big word. We’re all starting to think, okay, AI is coming. It can be really influential. How do we use AI in our products? We’re also beginning to recognize that, with their merging with UserZoom and Usertesting, EnjoyHQ’s product roadmap wasn’t quite aligned with our long-term vision of what we wanted our insights repository to look like.

That brings us to the end of 2023. Marieke and our colleague Kevin reached out to someone internally experiencing this tool called Glean. Our engineering team was testing out this tool. Marieke and Kevin made some connections, and we were able to get access to this tool. We started to explore Glean itself and what it could do for us.

It’s a bit of a tumultuous time for the industry, right? We noticed internally that we have a ton of work that we’re doing as a team, but we’re not able to handle all of that work. We have a small team. We want to make sure that the work that we’re doing is really impactful.

So we started having a lot of conversations around how to evangelize and democratize this research that we have. How do we make sure that the research that we have that we’ve done since 2015 is easily and readily available for the teams that want to access it?

And that’s where we came to a head with Glean. And it was a fast process too. I’ll say, I think we went from EnjoyHQ to Glean in three weeks. It was a gargantuan effort, but luckily, we were able to get over and move everything over into Glean. The reason that we were able to do that so quickly is because of the foundations that we’ve built over time. So we had all of our work in Airtable. We have been tracking our work since 2018. We had been backtracking as well. We had everything in Enjoy HQ that we could move quickly into Glean.

Towards a system of research

Our journey has been about creating a foundation that empowers our team and drives that organizational success. So this effort that we’ve put in has really gone beyond just data collection. It’s about building a solid foundation of project tracking and research knowledge that reflects the hard work that our team is doing. We started with some of these foundational building blocks. We engaged in a lot of collaboration and exploration, and we continue to explore.

We decided through all of this that we’re not actually just looking for a one-size-fits-all repository. We want to build a system of research. We have our project tracking, which gives us a robust amount of data. To understand what we’re doing here at LinkedIn, but it also helps us understand the resources we need, tooling and things of that nature.

Then we have our separate insights repository. Together, we’ve built this system, which operates as a pretty well-oiled system, where each component comes together to provide this holistic view of our business. Through this, we can gain a unique perspective on, how we want to drive our strategy and our vision with our data.

So what are our current use cases?

Starting point for discovery

Our top use case right now is that we really want this to be a starting point for discovery for our PMs and our designers. So say you are a new PM, or you can be a PM that’s been here for a long time, and you’re starting at that project – as mentioned before, ‘job seeker’ is a huge population that we focus on, so you want to know everything there is about ‘job seeker’. So you can inform some of those early-stage product decisions. All of our knowledge of an audience and a product area is easily accessible in this repository. All they have to go in and say, tell me what I need to know about ‘job seekers’ and it’ll pull out most of what we have.

Build on existing knowledge

Our second use case is to build on the existing knowledge. We have done nine years of research. And that’s just a lot. Our partners often request research from us very quickly. We want to ensure that their requests are built on previous research. Our goal is to add new insights to our knowledge base rather than doing some of that duplicative work. We want to make sure that, with the time and resources we invest in this knowledge, we can continue to use that over time.

A research knowledge management space

Our insights actually aren’t in Airtable. They’re in Google documents. They’re in, SharePoint sites, things of that nature. So they’re not visible there, but they’re not accessible to people outside of our organization. Airtable is just a tool that we use within design.

And so what this has done is we’re able to put all of our insights into one space and it stores all of our insights and it gives us that, or not it gives us that single source of truth. So that our people can access that and they can continue to access our insights.

Existing research repository challenges

Engagement – How can we get more people to use the Insights Hub more often?

Marieke – The first one is just increasing engagement. I mentioned the breakdown of the people who have used the Insights Hub. It has only been six months, but only about 25% of our design team and only about 5% of the product or product management organisation have used it.

Brandi and Emily, a researcher on my team, ran a series of feedback sessions this summer with both users and non-users of our Insights Hub. We learned that we want to do more to educate people on using it. This is true for any conversational UI: people struggling what kinds of questions can I ask? How do I ask it?

Incorporating even more team and company contacts, like making this app that we built a little smarter. For example tying the right research to the right sort of priorities. Imagine actually putting in, I’m working on this project, what do I need to know? It’s not a kind of thing that the Insights Hub can do now.

We want the app to be a little smarter about saying, “I don’t know.” It’s not very good at saying, “I don’t know the answer to that question.” And we’d rather be like, Oh, actually, that’s a gap in our knowledge, we should do more research. That’s ultimately what we want to do more of.

Exploration – How do you search when you don’t know what you don’t know?

The next one is that this version of our repository does not solve for exploration well. So imagine going into a library and only getting to use their online catalogue. So you can come in. You know there are shells with books, but you’re only allowed to sit at the computer and search. It’s hard to know what to search for. There’s tremendous value in just getting to browse and see. And Enjoy HQ did this really well.

UXR needs – Should we build a different system for researchers?

We have been very focused on our stakeholders, but researchers think differently and want different stuff. It feels silly to have a second bot that’s for researchers that structures the output differently, but sometimes all researchers want the list of the research, or they want a summary. Our current Insights Hub is not perfect for what our research team needs, and it’s something we want to solve in the next year.

Triangulation – Should we open up our repository to other insights functions?

The last one is the most future-looking: exciting and scary at the same time. This also came out of our feedback. We have so many different insights functions at the company that all contribute to a holistic view of what is going on in our product. What are our opportunities? How do we make LinkedIn better? I believe as a researcher that the best research is done together with other insights functions.

I love it when we can come together and present recommendations together to a product manager. So why shouldn’t our research repository do the same? There’s a little bit of a hesitation to give up on that control of what happens with our information. But I’m also so excited to see what could come from this.

We’re struggling, maybe with an identity crisis and we really want to make sure AI can represent our function well. And with that, we’d love to open it up to questions from all of you.

…

Q&A in recording only.

Photo by Paul Talbot on Unsplash