I get were you are coming from with the asbestos comparison. With asbestos though, we used it to manufacture everything. Drywall, floor covering, adhesives, insulation, drywall joint compound, popcorn ceiling spray, pipe and duct wrap, gaskets and seals just to name a few. Heck it was even in some cigarette filters, WTF?!?
In addition, asbestos is friable, so it can become airborne and inhaled.
I don't think bullets are quite that widespread. I mean, maybe in downtown Chicago or something but not most places.

I would think if you are having lead poisoning issues from a bullet it is more serious than simple exposure.