
Duck Tales: Why we built a ‘second opinion’ option in Duck.ai and how you can use it to increase confidence in responses. (Ep.48)
The “second opinion” option in Duck.ai acts almost like a shuffle button, automatically choosing a different AI model to take a second shot at your query. This lets you compare answers side by side, learn the strengths and weaknesses of different models, and avoid overconfidence in a given chatbot’s answers. In this episode, Tatum and Beah (product) discuss how the feature came about and how it addresses multiple challenges with AI search. If you have feedback on Duck Tales, or episode ideas, email us at podcast@duckduckgo.com.Disclaimers: (1) The audio, video (above), and transcript (below) are only lightly edited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy. Tatum: Hello and welcome to DuckTales, where we go behind the scenes at DuckDuckGo and talk about the people, stories, and technology that are building privacy tools for everyone. This week I’m talking to DuckDuckGo Chief Product Officer Beah about a feature in Duck.ai called Second Opinion that lets you compare answers from different AI models side by side. Beah, thank you so much for coming on DuckTales to tell me about second opinion. Beah: My pleasure. Thanks for having me. Tatum: Will you explain what this feature is? Beah: Sure. and in fact, let me show you, which I think will be even better. So we’re at Duck.ai. I’m gonna ask a question that’s at the forefront of everybody’s mind, which is best trick for removing a butternut squash soup stain from white t-shirt. and I’m getting a response here. The model that provided it was GPT-6 Luna and it says to blot off excess, flush the stain from the back with cold water, liquid laundry detergent, Tatum: So we’ve got about two short paragraphs of instructions from GPT-6 Luna. And then at the bottom of the answer, we see a couple buttons like a share button, a copy button, and a button that says second opinion. Beah: Yes, so I I’m like, all right, this answer seems okay, but I don’t want to get this wrong and forever have a yellow stain on my white t-shirt. So I’m gonna click the second opinion and see what another model’s advice is. So in this case, Duck.ai is using Gemma 4, which is an open weight model, asking the same question and we get a slightly different answer here. So we got to treat the grease first and the color second. So looks like there’s a lot of overlap with the first answer, but some additional detail and some additional steps that give me a little bit more confidence here. and so yeah the idea like roughly with this second opinion is that Sometimes you ask AI a question, you get an answer, and you’re like, maybe, okay, I don’t know. Like I would ask another friend if I was at a sitting at a table looking for advice. And so you get to roll the dice again and and ask another model and get another question, and then you can kind of figure out the the diff Tatum: Interesting. Okay, so you just used the image of like rolling a dice. Why that? Beah: Yeah, because answers from AI are non-deterministic, meaning, they are generated on the spot using a prediction basically of what you want to hear um based on some huge data training set and They can be wrong and they can be like totally wrong. They can hallucinate and be crazy, so that I think is one of the reasons that people are rightly cautious about using AI to answer their questions because they are not guaranteed right answers. And so Running the same question on different two different models gives you kind of a sense of the universe of possibilities and a little bit more data to make an informed decision. Tatum: Yeah, and it seems like DuckDuckGo is uniquely able to build a feature like this because an AI company would never encourage you and has no reason to show you an answer from a different company’s models. Whereas, you know, we have all these models in the same place with a privacy layer that is now letting us do this kind of fun and useful thing of seeing two answers side by side. How did this feature come to be? Like what kind of problem is it solving for people as they search? Beah: There’s a few answers to that. Anecdotally, when I’m using AI, which I use Duck.ai every day many times over, I often find myself kind of wanting to like second guess or fact check or get another opinion on. The answers that I’m getting. sometimes you’re like asking for some, like, you know, what’s Tom Hanks’ birthday? There’s one answer to that that’s correct. All the other answers are wrong. most of the things that I’m using AI for don’t have one right answer. And so getting kind of a a spread of perspectives is is part of my reasoning and decision making process. So that’s like the anecdotal answer. There’s like a more kind of meta-societal answer, which is people don’t trust AI answers for good reasons because they often, spit out an answer that is just one perspective and maybe wrong in varying degrees, but they present it with great confidence and don’t really necessarily signal the extent to which it could be wrong or how it might be wrong or what other answers might be. And so I think second opinion gives you a literally a one tap or one click way to mitigate that downside of using AI. Tatum: Yeah, so it seems like it solves, a problem for the user or it’s useful to that one person on their search of getting, you know, more information or a second, a second perspective. But it seems to also be useful as a critical thinking tool or useful to shape the way that people use AI search. It this made me think of some research that Microsoft did about how relying on AI help affects people’s cognitive engagement with the task that they’re doing. And a lot of people pointed at this study saying, like, look, using AI makes you stupider. when I went and read the study, what it seemed to be more related to was actually your confidence in the AI’s answers. The more kind of confident you become that it’s correct, then the less critical thinking you’re you’re doing. Whereas when you have a healthy amount of skepticism, you it doesn’t affect your cognitive engagement the same way. So so how can a feature like this kind of help shape the way that people use AI to make it kind of less risky, more useful? Beah: Yeah, I mean I think like an interesting analog is like if you were doing research on something, the old fashioned way without AI, you would probably read several sources. Like if you’re planning a trip, you might read like this guide book and this person’s blog post and this official website. And so, you know, AI is handy in that you can kick a button and get like one tidy answer. But if that tidy answer is misleading or incomplete, then that’s problematic. And so my hope with second opinion is that it gives you an equally, low friction way to round out those data points a bit and kind of require or accommodate a little bit more critical thinking where it’s appropriate, Tatum: Is it fair to say that the different models and different companies excel at different things? And as a lay person, is there any hope for me to like know what they are? Beah: There’s no hope for you, Tatum. Tatum: No! Beah: Yeah, no, I I it’s a good question. I I mean I do think the short answer is no. I think they’re like the the things are changing too fast and the nuances are too great. I think that there are some dimensions at which a consumer could make informed choices, like one that I think probably is reasonable is you might have an intuition of whether your question is like, a very quick answer is sufficient or whether you need it to do some like deeper thinking and research and challenging itself. And so on a given model, you can turn the reasoning up or down. but like in terms of the distinction, you know, when you’re trying to figure out how to like remove butternut squash soup from a t shirt, the distinction between GPT six Luna and GPT 5.6 Luna and Claude latest model, like I do not think that a normal human could or should need to like ho… This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit insideduckduckgo.substack.com
- Transcript


















