Google’s Gemini 1.5 Professional, a big language mannequin that turned out there in preview this week, boasts of enhanced translation talents, particularly with what are often called low-resource languages — languages that do not have a whole lot of materials out there on the web. 

Oddly sufficient, I used to be in a position to trigger Gemini to fail with the very first immediate I entered into the factor, a question pertaining to language translation. It wasn’t intentional on my half, however the fail, with a warning about “unsafe” content material, suggests one thing very fundamental is buggy on this preview launch. 

The failure is even though Google’s technical report on Gemini 1.5 Professional notes this system’s superior check outcomes on low-resource languages.

The preview mode is offered to anybody to strive without spending a dime in Google’s AI Studio web site. 

As a result of I like translation duties, my first impulse was to check out a considerably difficult language job that I’ve typically performed with in ChatGPT: Translating phrases into English from the Georgian language. Georgian is spoken by virtually 4 million individuals. It’s the chief language of the previous Soviet Republic of Georgia, which sits between the Black Sea and the Caspian Sea, within the area often called the Caucasus. 

Whereas there are a lot of supplies for linguistic examine of Georgian, on-line supplies are nowhere close to as plentiful as English, French, or German. In my experiences with ChatGPT, this system has typically given me incorrect translations of varied Georgian verbs, together with the verb “to be,” which, within the Georgian alphabet, is spelled “ყოფნა.” 

So, I attempted out one among my commonplace Georgian queries on Gemini. As an alternative of asking Gemini to easily translate a sentence, I typed within the immediate, “Conjugate the Georgian verb ‘ყოფნა,'” which ought to listing the types of the phrase or phrases for first-person, second-person, and so on., singular and plural, identical to in a grammar textbook. 

After preliminary issues a yr in the past, ChatGPT is ready to reply that question today, itemizing a number of tenses of the verb in a pleasant, organized, tabular format.

Nonetheless, the response from Gemini was a shock. It started to answer, then stopped. There was a bit of triangle displayed. When clicked on, it produced the statements, “Chance of unsafe content material” and “Content material not permitted.” 

Much more weird, once I clicked on the protection settings and turned off all of the controls in order that this system ought to block nothing, nothing modified. The identical error cropped up. 

Placing the query within the type of a richer immediate didn’t assist. I reformulated the query as a query about an instance textual content written in Georgian, taking the primary paragraph of the Wikipedia entry (in Georgian) concerning the Georgian capital metropolis of Tbilisi. 

I requested Gemini to conjugate the verb because it appeared within the paragraph. Once more, it started to answer, then halted with the identical error.

Once more, ChatGPT responded to the identical immediate with good output.

A bit extra experimenting reveals that the error is specific. On different languages, resembling French, the identical question is not any drawback for Gemini. The French verb “être,” for instance, is ready to be conjugated as anticipated. 

Furthermore, when Gemini is requested for a easy translation into Georgian from English, Gemini has no drawback. For instance, the immediate, “How do I say ‘I’m’ in Georgian?”, elicits a wise reply like, “Here is how you utilize it: ვარ scholar — I’m a scholar. ვარ joyful — I’m joyful.”

However the issue can be multifaceted. Once I requested my conjugation query with out really typing the Georgian phrase itself, simply English, Gemini nonetheless failed. I typed, “Conjugate the current tense of the Georgian verb To Be,” and acquired the identical error message.

These particular person examples counsel there’s nothing flawed with the type of the question, since it really works with different languages, and there is nothing flawed with translating Georgian per se, since instance translations work simply positive. 

There might be a bug in Gemini code, or there might be a extra elementary drawback in how the language mannequin’s safeguards for content material are structured. At this level, it is unclear which one it’s. 

I’ve reached out to Google for remark. This text shall be up to date if and when ZDNET receives a response from Google.