Massive buffering occurring

I’ve got a couple proprietary transmitter and receiver blocks here with
some other custom blocks in the flowgraph, and the problem I’m seeing is
that I seem to be building up a really large buffer of data that hasn’t
been transmitted out of the USRP yet. I’m guessing I’ve missed something
with regards to the set_relative_rate? Can someone explain or point me
to documentation that does, how to minimize the buffering to a sane rate
relative to the consumption?