How do I get a faster mile?

I know speed workouts, but I'm a fast sprinter and it hasn't helped much. I can't seem to break that 6:06 I got last year and it's killing me. My times are all 6:06, 6:09, 6:12, 6:08, like that. I want to be a fast miler and give up the 400 and race the mile, any ideas?