Building upon prior word knowledge

A major difficulty of the Japanese language is the alternate reading of kanji as they are combined into words.
When I learn new vocabularies in Japanese, I always start by looking up what other words each kanji is used in, so that I can link the new word to other words I already know.
This is because I often already know words that include the kanji, without knowing how the word is written.
E.g., everyone knows the word 「間違い」, but not how to write it until JLPT level N2.
Here is a multi-step instruction to help in this process:

Notice that I first instruct ChatGPT to generate multiple example words, before refining the selection to the most common usage.
For the term 「貯蔵」, here is an example output that illustrates this approach:

Common usages of the kanji inside「貯蔵」

While I chose to format the final answer as a table for readability, you can also have the output as a sentence enclosed in quotes for simple parsing, as suggested in the OpenAI Prompting Guide.
I employed this method inside a utility that is invoked via a keyboard shortcut for the currently marked word (e.g. in the browser), and generates and reads this information aloud using Google Cloud Text to Speech.
It is also worth noting that GPT-4 is much better at following these complex instructions than GPT-3.5:

GPT-3.5’s answer to the exact same prompt.

Limitations of ChatGPT

When learning new kanji in the technical terms class, the teacher would remind us of similar-looking kanji and their difference.
In Japanese, it is extremely important to closely look at kanji, as they can be easily confused.
Having ChatGPT point out such things would be helpful in many cases.
Let’s try to get some kanji with similar appearance.

Output for similar kanji.

You can see that ChatGPT does quite well for this commonly used kanji, at least when it verbosely writes down its chain of thought.
Now let’s try a rarer example with higher level of difficulty.

Output for similar kanji, this time for a rarer kanji.

As you can see, the answer is completely wrong.
This shows again that LLMs have no knowledge of the visual appearance of the actual characters, as they only interpret characters through their vector representation.

Let’s try a simple test that checks if ChatGPT can distinguish between hiragana and kanji.

ChatGPT’s output for distinguishing between Kanji and Hiragana.

This is quite disappointing.
Let’s try again using a chain of thought:

ChatGPT’s output for distinguishing between Kanji and Hiragana, now with chain of thought.

Now the output is correct.
ChatGPT seems to be adept with furigana.
However, this makes the answer significantly longer, so that for many real world texts, this procedure might be prohibitive because the character limit is reached.
This is especially a problem for Japanese, since Japanese kanji require more embedding tokens per character than alphabetic letters.
One possible solution could be to finetune GPT-3.5 or Llama-2 on this output.
However, in the long term, this is definitely something that needs to be fixed, maybe by adapting the Japanese character embeddings.

Conclusion

Even though I would never want to miss the experiences I had learning Japanese with fellow exchange students and the helpful and friendly teachers at the language school, the possibilities that ChatGPT provides allow me to continue studying Japanese back in Germany.
I can create customized learning materials for myself, from vocabulary tables to practice texts and kanji tests.
However, ChatGPT also has limitations in fully grasping Japanese characters, particularly rare kanji.
In the next article, we take a look at a possible solution to infuse ChatGPT with additional knowledge about kanji.

[date] 2024-02-03
[upload] 2024-04-05

東京大学　羽室開（Kai Hamuro Eberl）

Enhancing Japanese Study with ChatGPT – Part I
Visualizing All Kanji in a Graph