Help with many to many in fixtures

I have three tables,

Courses (like finance 101)
Courses_modules (many to many link table)
Modules ( a part of a course like a module on supply and demand)

In my Module test when I have the line

class ModuleTest < Test::Unit::TestCase
fixtures :course_modules , :modules

The order that is happening is course_modules are deleted and then
course_modules.yml file is called and inserts values. Then modules is
called and gets a foreign key violation on delete. How do I avoid this
when testing a many to many relationship?