My first guess is that the polish lap is uncovering subsurface damage from the 350 plated lap. Avoid using the 350 at all, unless you must remove a large amount of material. If you do have to use it, stop at least a couple millimeters from final dimensions.
Ceramic polishing laps are notoriously tricky. I’ve never used one. Perhaps someone else can give you advice on that.