Google executive: "Bard is not intended to provide factual answers, just a companion for creative tasks"

Several months after the rushed release of AI service Bard, Google executives are still busy downplaying its capabilities. Google's UK boss Debbie Weinstein told the BBC last week that Bard was "not really the place that you go to search for specific information" and that the company is still "encouraging people to actually use Google as the search engine to actually reference information they found."

A similar sentiment was echoed by Adi Mayrav Gilady, a product manager in Google's research division in Israel, who has been at the forefront of developing generative artificial intelligence products at the company. In an interview with Calcalist, she delves into the intricacies of the innovative AI chatbot, the challenges and triumphs in training Bard in Hebrew, and sheds light on the efforts to make it as bias-free and creative as possible.

Adi Mayrav Gilady.

One of the known problems with large language models is that they sometimes generate invented information in response to questions. How do you deal with it?

"Behind a large language model, there is no database from which it extracts information. It's a statistical tool trained on a vast amount of text and predicts word combinations with statistical probability. That's why we may observe mistakes or inaccuracies. We are constantly working on improving this. For example, problematic or inaccurate answers are removed, and as part of the model's training, it is taught to be as close to reality as possible. However, it is important to understand that Bard is not intended to be a tool that provides specific or factual answers. For that purpose, Google search is the best tool. Bard is a companion for creative tasks, a place for brainstorming, and a tool for imagination. If a user asks Bard to write a story about a purple unicorn, we don't want it to say, 'There's no such thing as a unicorn.' We want it to be a tool to help users think of new things."

Let's expand on that. What are the intended uses for Bard, or what do you see users doing with it?

"Bard is excellent for a variety of creative tasks related to text. It is very effective for a variety of productive tasks, such as writing an email or a speech, or helping with coding. It is particularly useful for overcoming the blank page problem, providing a starting point for tasks when you don't know where to begin the research or writing. Some uses I've seen or done include feeding Bard a very long article and asking it to summarize it in 300 words, or presenting the article's strengths and weaknesses.

"Bard can help write all kinds of creative things that are highly personalized. For example, you can ask it to write a bedtime story for children that is adapted to your child. If you have a 4-year-old who needs help saying goodbye to the pacifier and loves football, you can ask Bard to create special stories for him. My daughter likes unicorns, ice cream, and the color purple, so I asked Bard to write her a bedtime story based on her preferences. You can ask it to create a personalized training program that fits your needs, considering personal parameters and the place where you want to train, such as in the park. You can seek ideas on how to teach children fractions or come up with ideas for a new book you want to write. You can even describe the book and ask Bard to suggest an ending.

"Bard is designed to facilitate idea generation. For instance, I'm celebrating my daughter's birthday this weekend, and Bard gave me ideas for places to celebrate a birthday in August and helped me make a list of things to buy and refreshments. With the new feature of Google Lens, which allows you to ask Bard questions based on an image, I've seen people on TikTok taking pictures of their meals and asking Bard to analyze how many calories they contain, or taking pictures of the contents of their refrigerator and asking for ideas on what to cook. You can even take a picture of a chart on paper and ask Bard to turn it into a text chart that can be exported to Google Docs."

And in these situations, it's important that it doesn't make things up. For example, you don't want it to incorrectly estimate the calories in your meal.

"Exactly. As a language model, Bard is not intended to provide accurate point-by-point information, and we strongly recommend that in everything you do with Bard, you verify and continue the search on Google. Even in these cases, we recommend checking to ensure that no inaccuracies are introduced along the way. In most cases, this approach provides a much better starting point, something to work with. Bard serves as a springboard for your creative process."

What about presenting the sources that Bard relied on in its answer? Microsoft's Bing browser integration of GPT does something similar.

"This is not a feature that exists in Bard currently. I cannot speak about future developments, but our product directly refers users to Google search. The goal is not for Bard to simply repeat texts verbatim from the Internet, but rather to be a tool for creativity and to help users understand and summarize large amounts of text."

Bard poses significant challenges to education and academia. Students are already using language models to write papers. Is this an issue that concerns you? Is it relevant to your work?

"This technology is powerful, exciting, and transformative. Like any other technology, it will impact and change the world. We hope and expect that the impact will be positive. It's a technology that can free up space and time for people to engage in more meaningful work and deeper thinking. It can help with tasks that consume time and create room for more creative thinking."

Are you collaborating with academia to address Bard's challenges?

"Google collaborates with academia on many levels. There are researchers here from various universities, and there are teams dedicated to working with all facets of the community, not just universities."

What is the importance of Google's R&D center in Israel for the development of Bard and generative artificial intelligence? Does it also focus on core areas or only adaptation to Hebrew?

"There are several research groups in the field of NLP (Natural Language Processing). The research here is a core part of the model's development, not just an adaptation to Hebrew. We develop training methods and techniques to improve the model's results. The team here is part of the global team. The research conducted in Israel contributes significantly to making Bard what it is and improves the quality of the answers it provides."

About two weeks ago, Google launched Bard in Hebrew, as part of the second wave of the launch, along with about 40 other languages. Previously, it was only available in three languages. It is safe to say that Hebrew is not one of the 50 most spoken languages in the world. Why the honor?

"It was a very exciting day, Bard's biggest moment since it hit the market. With the release in 40 languages, we invested a lot in usability for users all over the world, and we were very proud of the result. Additionally, many new features were released, such as support for Google Lens (the visual recognition application - a.k.a.) and the ability to have a conversation with Bard. Regarding the Hebrew support, we noticed the demand for Google products all over the world, and there was a process of prioritizing languages based on the needs of the users, with Hebrew being among them."

One of the challenges with training a large language model (LLM) in Hebrew is that, compared to other languages, there is a very limited supply of texts. How did you deal with this challenge?

"True, there are far fewer online texts in Hebrew to practice on, and we noticed that the initial quality of the model in Hebrew was not as good as in English. We went through several processes of tuning, fine-tuning, adjusting the answers, and improving the quality. In some of these processes, the model learns from examples. For instance, Bard is taught what an email looks like, or what a blog post looks like. For this purpose, we have a team of human testers and writers who help improve the model and perform tasks like writing content, checking answers, correcting them, assessing quality, and flagging instances when an answer does not meet our criteria, such as negative bias, which is something we actively address.

"Then there is another step called reinforcement learning. This is a method of training the model where it learns by trial and error. It is similar to the training procedure of a dog. When it successfully performs an exercise, it receives a reward, and when it doesn't, it receives feedback. Accordingly, the model receives feedback on the answers it provides and learns to focus on the answers we desire. The teams of human testers rate the answers, and make corrections if necessary. There are also models and methods to automatically flag issues, and feedback comes from actual users who can rate Bard's answers. Based on all these factors, a reward model is trained, which is used to fine-tune Bard. When the model is used by more people, it improves thanks to their feedback. The more the Hebrew model is used, the more it will improve."

What other challenges arose in the development process in Hebrew?

"Hebrew is a language that differentiates between male and female. In all the lateral work on Bard, especially in defined languages, we strive for the model to be as bias-free as possible and to answer in neutral language. This is something we actively test and flag instances where gender is used unnecessarily or in the wrong context. For example, initially, we noticed that the model would talk about certain professions in a masculine way and certain professions in a feminine way. This is certainly not something we want to happen, so we take care of it during the training phase to avoid such biases. Bard is trained on the texts that exist in the world, and any bias that exists in the model reflects biases that exist in society. We constantly check and try to find such cases to address them."

Finally, could you provide some tips on how to write a good prompt for Bard?

"Emphasizing what is important to you can be very helpful when interacting with Bard. For example, explicitly stating that you want the response to be tailored to a girl who likes purple. Additionally, it's a lot of trial and error. After receiving an answer, you can provide a prompt with adjustments or clarifications. For example, you could say, 'It was too long' or 'I want a less formal tone.' If you're looking for ideas for a new trip and Bard provided inappropriate suggestions, you can write, 'I don't like beaches at all,' or 'I've already been to Thailand, and I'm looking for something different, so please give me other suggestions.'"