You’ve probably already seen this from the NRC:
“Any dose from these gems would be extremely small. The NRC did a study to estimate the dose to a person wearing a large blue topaz stone (six carats). Assuming the stone had the highest level of radioactivity allowed under NRC regulations, the dose during the first year could be 0.03 millirem (NUREG 1717, page 2-21). Having a porcelain crown or false teeth would give twice that dose (0.07 millirem), while a chest X-ray gives about 60 millirem. But this estimate is conservative. The radiation continues to decay, so the dose rate would go down over the course of the year.”
As for irradiating a steel lap, that would take a lot more than a few pounds of blue topaz. It’s good to be careful about inhaling microscopic particles of anything, but topaz is a lot safer than stuff you encounter every day. You’ll get about 30 millirems from a year’s consumption of food and water, and 1 millirem from watching TV in a year.