Audio underruns - Sending signals over audio



I’m getting frequent audio underruns and I can’t figure out why.

I’m trying to transmit data over audio signals based on [1] in an
uncontinuous way, as i’m using packets.
The problem is that after a while I get consecutive audio underruns
(like, for several seconds straight). The “TunSource” block you can see
in the picture just outputs data from a tun device, though it also
happens with the other blocks like “Socket PDU”.
I already tried using a slightly different sample rate than the audio
sink has but I still got underruns. I’m quite new to this field so maybe
I’m just missing a major point.
I hope you can help me.

Edit: To clarify: This also happens, when I use a file source.