I had this problem on a large amethyst emerald cut I did a few years ago. I tried everything to get that stupid thing polished. I spent hours over several days.
Like Mr. Balmer said, I thought my facet wasn’t flat. That wasn’t my problem though.
In the end I actually asked Bob Keller for advice. He suggested my prepolish was bad. And he was right, I had to go back a step to prepolish.
Turned out I couldn’t go from 1200 to polish like I usually do with quartz for that particular stone. I am not sure why it was more difficult than other quartzes. Try backing off to a 3k or even 8k, then go back to polish. See if that helps if what Alan suggested doesn’t work.