My lowers are the same way.The imperfections in the cast are terrible on mine. To answer your query on the "plug" issue. Yes they are plugs that are welded in. I had on come out this winter. OOPS.
I also tried to polish them & found that it was a ton of work & I am not interested in doing that much work.
Good job though. More of a man than I want to be.

Thanks for the tip Mark. I hope they don't dull too fast. We only boat in fresh water.

Nice job! Next time, try a 3m deburring wheel. These wheels are just like a grinding wheel, but they are built out of Scotch Brite pads. They rock for polishing!