Yesterday the brass washers showed up and today the 6mm heat set brass inserts arrived.
All I'm waiting on now are the 6mm nuts and hopefully they will arrive tomorrow otherwise it's Monday.
It looks like I'll have to further widen the holes with a drill, but I'll wait and see.
I'll use the bolts fed through from one side to help pull the insert square into it's final location while applying heat to the insert. Well that's in theory and works with the 3mm ones I have, I think I need need more hands.