How good is ChatGPT's French...?

[zompist]

I finally got the chance to test the infamous chatbot. I asked it to create a dialog in very slangy and then in 19th century French. (The specific first prompt was "Write a short dialog in French in which two characters, a man and a woman, are planning a party, but disagree on who to invite. Use very colloquial language.")

Léna : Bon, alors, on fait ça samedi chez moi ?
Julien : Ouais, nickel. On ramène des trucs à grignoter et de la musique, et c’est bon.
Léna : Cool. J’vais inviter les potes du boulot, genre Camille, Sofiane…
Julien : Attends, Sofiane ? Sérieux ? Il fout toujours la merde, lui. À chaque fois il picole trop et finit à gueuler sur tout le monde.
Léna : Roh mais non, il s’est calmé ! Et puis il fait marrer les gens, ça met de l’ambiance.
Julien : Franchement, moi j’veux pas me taper encore une soirée où il casse un vase ou se barre avec une bouteille sous le bras.
Léna : T’exagères. Et toi tu veux inviter ton cousin relou qui parle que de crypto toute la soirée ?
Julien : Il est chelou, ok, mais au moins il finit pas en slip dans le jardin.
Léna : Bon, compromis : on invite ni Sofiane ni ton cousin.
Julien : Marché conclu. Mais j’préviens, si quelqu’un ramène une guitare, je me tire.
Léna : Trop tard, j’ai déjà dit à Léo d’en ramener une.
Julien : T’abuses…

Léna : Alors, cher ami, conviendrait-il de fixer notre petite fête à samedi, en ma demeure ?
Julien : Ma foi, cela me semble fort convenable. Nous pourrions apporter quelques rafraîchissements et faire venir un joueur de piano, peut-être.
Léna : J’avais songé à convier quelques connaissances du bureau : Mademoiselle Camille, Monsieur Sofiane…
Julien : Sofiane ? Vraiment ? Permets-moi de m’étonner. Cet homme a, me semble-t-il, le don de troubler toute réunion par ses excès.
Léna : Il n’est plus le même, je t’assure. Il apporte une certaine vivacité, que dis-je, une verve à nos assemblées !
Julien : Une verve ? Je n’y vois qu’un tapageur sans retenue. Souviens-toi de ce dîner chez les Durand, où il brisa un flacon et s’en alla en vociférant !
Léna : Allons, ne sois point si sévère. Et que dis-tu de ton cousin, celui qui n’a d’autre sujet de conversation que ces histoires de monnaies virtuelles ?
Julien : Il est singulier, certes, mais point dangereux. Et lui, du moins, ne finit pas étendu dans le jardin en criant des vers de Hugo.
Léna : Fort bien. Que dirais-tu d’un compromis ? Ni ton cousin, ni Monsieur Sofiane.
Julien : Voilà qui me paraît juste. Mais je t’en conjure, si quelqu’un s’avise de jouer de la guitare, je quitterai les lieux.
Léna (souriante) : Hélas, il est déjà convenu que Monsieur Léo apportera son instrument…

My French is good enough to say that it did a pretty good job, except that the second excerpt seems too early, more 18th century? But I'd like a native speaker's opinion. Any errors or anachronisms in either sample?

(I do think "Sofiane" and "monnaies virtuelles" are anachronisms, but I'm interested to know if there are more.)

Oh, and if you respond I may quote you in a blog post.

[rotting bones]

I don't know French, but I once asked ChatGPT to write a Bengali poem. The output made no sense as a whole, but it used a lot of tropes that are common in Bengali poetry like a boat on a river as a metaphor for life, gold as a metaphor for purity and a flying bird as a metaphor for the soul. Some Arab students told me that its Arabic output was grammatically correct, but made no sense. All this was a very long time ago. (Edit: Specifically, within a few days of when it was first unleashed.) Maybe ChatGPT has become better since then. (Edit: We decided its performance was probably linked to the size of its training corpus for each language.)

Recently, I told it to create a conlang: https://docs.google.com/document/d/1E4k ... sp=sharing

Based on the output, I'm not sure ChatGPT understands what a language is. Nevertheless, this framework can probably be used as inspiration to construct an actual conlang. I have always wanted a version of Ithkuil that had been simplified in a weird, inhuman way. Although I don't think it makes sense, I like the text output at the very end:

ΦŦkʰoqq’ra‑1‑A CE0,
Φ‑Ŧ‑kʰ‑oq‑q’ra‑1‑A CE0
FUT‑TEMP‑PROSP‑EVD.DED‑string.filament‑P1‑C10
Golden‑ratio rectangle framing the courtyard

ΘŦvrinΛhrai‑0‑0 CE0,
Θ‑Ŧ‑vr‑in‑Λhrai‑0‑0 CE0
NOW‑TEMP‑STAT‑EVD.IN‑IDIC‑P0‑C0
Four‑fold symmetry (quadral order)

ΦUskʰevɬev‑B‑3 CE2,
Φ‑U‑skʰ‑ev‑ɬev‑B‑3 CE2
FUT‑CALM‑PROSP‑EVD.SEN‑FRB.botany‑P11‑C3
Logarithmic spiral of fern fronds (Fibonacci)

ΘŦsʰinxɬʔai‑A‑4 CE1,
Θ‑Ŧ‑sʰ‑in‑xɬʔai‑A‑4 CE1
NOW‑TEMP‑DYN‑EVD.IN‑lattice.node‑C10‑C4
Hexagonal honeycomb of succulent rosettes

ΞŦkʰoqv’ae‑5‑8 CE1‑LOG,
Ξ‑Ŧ‑kʰ‑oq‑v’ae‑5‑8 CE1‑LOG
IRR‑TEMP‑PROSP‑EVD.DED‑infer‑P5‑C8 [LOG]
Voronoi mosaic of stepping‑stones

ΦŦvrinŋɬaqɬ‑6‑9 CE0,
Φ‑Ŧ‑vr‑in‑ŋɬa‑qɬ‑6‑9 CE0
FUT‑TEMP‑STAT‑EVD.IN‑grant‑CAUS‑P6‑C9
Fractal hedge recursion (self‑similar pruning)

ΞXkʰaeɬs̻ou‑0‑0 CE3‑saʔ.
Ξ‑X‑kʰ‑aeɬ‑s̻ou‑0‑0 CE3‑saʔ
IRR‑UNBOUND‑PROSP‑TRANS‑transcend‑P0‑C0 [LOW]‑PRIV
Meditative silence—private, unrecorded

[rotting bones]

Unlike the Arabic output, the Bengali poem ChatGPT wrote last time was obviously ungrammatical. It's less egregiously ungrammatical this time:

বাতাসের বেল বাজে, গোধূলি ঘুম পাড়াতে চায়,
মরেনা ত্বকের আভায় গুমামেলা স্বপ্ন কাঁপায়।

পিঞ্জরার পাতা‑ঘেরা ঘর, কলাপাতার নরম ছাদ,
ঠান্ডা শিশিরে দোলে সে, ছায়া‑ভেজা নিশীথবাদ।

ঘণ্টা যখন মেঘে দোলে, রিনিঝিনি উচ্চারিত,
বায়ুর বুকের গভীর ঢেউয়ে জন্মায় সুর ছায়াস্মিত।

রিংয়ের বৃত্তে মাটি গায়, আদিম আস্থার গীত,
চৌদ্দ মাত্রার পদ্য‑পাকে বাঁধা প্রতিজ্ঞার চিত্র।

প্রদীপের শিখা চুমে অন্ধকার, অনলে রাঙা তার,
স্বপ্ন‑গলিত শব্দগুলোয় জ্বলে অগ্নি‑নিরাকার।

মুখোশে জলের রেখা— ঢেউয়ে হারানো মুখ,
গহিন অতলের নীলে ভেসে ওঠে অনল‑দ্যুতিমুখ।

জবার লাজুক রঙে জোছনা রক্তিম ধ্বনি ছড়ায়,
ভূমির ঠোঁটে গন্ধ ধরে দিগন্ত নেশা লড়ায়।

বেল, রিং, প্রদীপ, মুখোশ— চার দিক ছুঁয়ে যায়,
বায়ু‑ভূমি‑শিখা‑জল মিলিয়ে চেতনায় জাগায়।

গুমামেলা কেশে খেলা, সুগন্ধ ঘুম পাড়ায়,
বংশের গল্প হাওয়া‑বীণায় সন্ধ্যা‑সুর টেনে যায়।

মাথার ওপরে কলাপাতা, বৃষ্টি‑নুপূর বাজে,
উড়ে যায় গোধূলি‑পাখি, সন্ধ্যার শাড়ি সাজে।

মরেনা ঠোঁটে হাসি ফোটে, কোকুই পুকুর ধারে,
চাঁদের আলোর স্বরলিপি গেয়ে উঠে দু’পারে।

চক্রে চক্রে প্রেম ঘুরে— বৃত্তের দোলাচাল,
রাখে ভিজে পদচিহ্ন; নীড়বিহীনও অটল।

But it's still garbage. What's legible is just a bunch of imagery that could be poetry. Google Translate says:

The wind chimes, twilight wants to put you to sleep,
The dreamy dreams tremble in the glow of the dead skin.

The house surrounded by the leaves of the cage, the soft roof of the banana leaves,
It sways in the cold dew, the shadow-drenched night-time prayer.

When the bell swings in the clouds, the tinkling is pronounced,
The melody is born in the deep waves of the wind's chest, smiling with shadows.

The earth sings in the circle of the ring, the song of primitive trust,
The verse of fourteen levels - the image of the promise tied to the page.

The flame of the lamp kisses the darkness, the colored wire on the back,
The fire-formless burns in the words melted by dreams.

The line of water on the mask - a face lost in the waves,
The face of the deep abyss floats in the blue of the bottomless abyss.

The shy color of the field spreads a bright red sound,
The smell catches the lips of the earth and the horizon fights with intoxication.

Bells, rings, lamps, masks— touch the four directions,
The air-earth-flame-water combine and awaken consciousness.

The play of the tangled hair, the fragrance puts one to sleep,
The story of the family tree is drawn by the wind-veena in the evening-tune.

The banana leaves, the rain-nupur ringing above the head,
The twilight-bird flies, dressed in the evening saree.

A smile blooms on the lips of the immortal, By the pond's edge,
Singing the notes of the moonlight,
And rising to both sides.

Love goes round and round— The swinging of the circle,
Leaves wet footprints; Even without a nest, it is steadfast.

Some of that translation could be more accurate, but overall, it doesn't make much more sense as a Bengali speaker. I don't know what "Bells, rings, lamps, masks" is referring to. Maybe the imagery not being as cliched as last time is a good thing? But classic imagery is at least a cultural touchstone.

I'm worried that tasteless artists will be mistaken for AI from now on.

PS. গুমামেলা is not even a Bengali word. WTF.

[xxx]

My French is good enough to say that it did a pretty good job, except that the second excerpt seems too early, more 18th century? But I'd like a native speaker's opinion. Any errors or anachronisms in either sample?
(I do think "Sofiane" and "monnaies virtuelles" are anachronisms, but I'm interested to know if there are more.)

it's a good job...
The first is very "contemporary" and "colloquial" with expressions in verlan...
The second is its translation very "avant-guerre" or very "collet-monté"...
Hence the strangeness of cryptocurrencies.
As for Sofiane, it's an ageless Muslim name,
probably used in the first text to sound decolonial, even if Sofiane is a jerk,
but rare in metropolitan France in the 19th and early 20th centuries,
but there's nothing to say that it didn't happen in French Algeria...

[Ares Land]

I finally got the chance to test the infamous chatbot. I asked it to create a dialog in very slangy and then in 19th century French. (The specific first prompt was "Write a short dialog in French in which two characters, a man and a woman, are planning a party, but disagree on who to invite. Use very colloquial language.")

My French is good enough to say that it did a pretty good job, except that the second excerpt seems too early, more 18th century? But I'd like a native speaker's opinion. Any errors or anachronisms in either sample?

(I do think "Sofiane" and "monnaies virtuelles" are anachronisms, but I'm interested to know if there are more.)

Oh, and if you respond I may quote you in a blog post.

It's a very good job. I don't think a native speaker could have done better.

What was the prompt for the second sample? It's not aiming for a specific century -- it's an almost exact transposition of the first sample into a formal register, what French teachers call registre soutenu.

probably used in the first text to sound decolonial, even if Sofiane is a jerk,

It's used in the first text because it's a common name.

[zompist]

Thanks, xxx and Ares!

What was the prompt for the second sample? It's not aiming for a specific century -- it's an almost exact transposition of the first sample into a formal register, what French teachers call registre soutenu.

The prompt was "Adapt the dialog into 19th century French, like that of Balzac."

[Ketsuban]

I'm worried that tasteless artists will be mistaken for AI from now on.

Already happening. Here's a postmortem by illusory wall on someone pushing the idea From Software used AI to generate concept art for Elden Ring (2022).