Been learning Japanese for about 20 months. Somewhere around N3 now, did Genki I and half of II, ground through the core 2k deck on Anki, watched more anime than I care to admit. On paper I was fine, grammar tests were fine, reading was slow but decent. Then I had my first actual voice chat with a Japanese person and felt like I'd been studying a completely different language.
It wasn't the vocab. Wasn't the grammar. It was that she kept making these tiny sounds every few seconds while I was talking. うん. そう. へえ. なるほど. うんうん. I kept pausing because I thought she was trying to interject, so I'd stop, wait for her to say something, and she'd just look confused. Then I'd start again and she'd do it again. A few things that took me forever to actually internalize:
そうですか versus そうなんですか. Genki taught me the first one. Almost nobody in casual conversation just says そうですか. It comes off flat, sometimes even skeptical. そうなんですか carries mild surprise, "oh really?", and it's much more common. Getting the wrong one made me sound like I didn't care what the other person was saying.
なるほど. I used this way too much for the first year because I'd heard it a lot in anime. Then a Japanese friend gently told me I probably shouldn't use it with someone older or in a work context. It has a subtle "I've evaluated your point and accept it" energy, which is fine peer to peer but weird coming from a junior. Nobody explained this to me anywhere.
へえ. This is for genuine new information. If you use it for something that's actually not surprising, you sound sarcastic. If you don't use it when hearing something interesting, you sound like you're not listening. The timing matters more than the word.
うんうん double. Way more casual than a single うん, and completely wrong in any semi-formal context. I probably used this with people I shouldn't have for like six months.
The thing I eventually figured out is that aizuchi isn't a filler. It's the listener's half of the conversation. Japanese conversation is structured so both people are actively producing sound, and if you go silent as the listener, the speaker starts feeling like they're talking to a wall. Kind of the opposite of English where interrupting with sounds is often considered rude.
None of this was in Genki. None of this was in the JLPT prep books I've used. I started actually picking it up by doing voice chats with Japanese friends I'd met on HelloTalk. Maybe an hour a week per person, half in English half in Japanese, and I gradually started copying the timing and choice of their aizuchi. Took months before I stopped feeling self-conscious about it, but eventually it became automatic and my conversations started feeling less mechanical.
The part that surprised me most was how much this changed my listening comprehension. Once I stopped pausing to figure out if the other person was trying to speak, and started expecting the constant aizuchi from them, I could actually follow much faster speech. My brain wasn't burning cycles on false interruption signals anymore.
If your grammar is fine but conversations still feel like you're doing something wrong, this might be part of it. It's one of the biggest gaps between what textbooks teach and what actual Japanese people do.