I spent a few days in Paris last week and while I have some basic French, it is rudimentary at best and not great when it comes to food. I decided to turn to ChatGPT to bridge that gap. And it performed better than I could have expected.
I've been using GPT-4o since it was first announced after the OpenAI Spring Update and one of the stand-out features is its vision capabilities — the ability to understand and analyze a full image accurately every time, even across different languages.
While in France I used this to navigate the train network, buy food in a supermarket and even have a brief conversation in a café to select a dairy-free option from the menu. That was even before OpenAI rolled out the next-generation speech-to-speech version of ChatGPT Voice.
Not only was it able to directly translate for me — in text and speech — but it was also able to teach me how to say words in French with patience and no judgment. Sorry Duolingo, but it seems ChatGPT is the true king of language learning platforms now.
Navigating the city with ChatGPT
When I first arrived at Paris Charles de Gaulle Airport I was relieved to see every sign was in English and French. This is in part due to the upcoming Olympics, but also because it is a major international airport. I was able to find my way to the ticket office and get on a train.
Turns out I was already standing at the right platform, which was good as the train I needed had just pulled up.
Everything went well until I arrived at my first station. Here the English disappeared and it was all French. While some degree of logic helped, such as looking for signs mentioning my destination station — I found showing ChatGPT the screen was quicker.
Some words were fine (I listened at least some of the time at school in French class), but not enough to help me find the right platform. Turns out I was already standing at the right platform, which was good as the train I needed had just pulled up.
I used it again to find the correct exit at my destination station to get to my hotel and it even offered up some café suggestions surrounding the hotel drawn from its new search feature, giving it live access to information from Bing and other sources.
Finding something to eat
The next time I turned to ChatGPT was in a supermarket. I am lactose intolerant so wanted to check the ingredients for dairy, and get an idea of what I was about to eat.
I sent photos of a few products and asked for details. It was able to not only give me details of ingredients, but it could explain the food and even answer questions about dietary issues.
For my main meal, I went to a cafe to grab something to takeaway and eat in my hotel room. I had a lot of work to do so didn’t have time to sit down and eat (I know, not really going local).
The problem was, the signs were in French and the man behind the takeaway counter couldn’t speak any English. Makes sense, given we were in France.
I opened ChatGPT Voice and asked it to ask, in French, whether they use butter or oil to cook the burgers then listen for and translate the response. It worked perfectly, they used oil and I got a burger and fries to takeaway — all requested for in French by ChatGPT Voice.
I repeated this while at the VivaTech conference, at least the vision part. There was a food truck with a list of delicious-sounding wraps. I took a photo, sent it to ChatGPT and asked it to identify any that are dairy-free. It translated the full sign, highlighted the ones I could eat and made a suggestion for how to place my order. Fortunately, they spoke English.
Bye bye Duolingo
I felt guilty at handing over all responsibility for communication to the AI. Wanting to at least order a coffee with breakfast and check out in French I asked ChatGPT for the appropriate sentences as well as a guide to saying it correctly.
This final part was possibly the most useful and one about to get significantly better in the next few months when GPT-4o live voice launches.
As well as saying it phonetically, it was able to listen to my pronunciation and make corrections until I was able to communicate the sentence I wanted to say correctly.
In this week's Prompt Jitsu I offered up a prompt you could try with ChatGPT to turn it into a language tutor, checking your phrasing and going through it conversationally in real time. This is a very powerful, not well utilized feature of multilingual AI models.
Overall this was more of a taste of what's to come from tools like ChatGPT than a perfect experience using AI as my own universal translator, but we are getting close to that level of technology — we won't be waiting a few hundred years.
