I do believe that the different alloys used for different bullets can and often do mess with accuracy and grouping. Driepootz of Gunsite fame (Tripodmvr on this forum) once wrote an very informative piece regarding this matter. I for one believe this is the cause of good loads being called out as suddenly going inaccurate.
Some folk use, lets say Sierra Game Kings during the year for springbuck, warthog, practice and then revert to their costly Nosler AccuBonds for hunting season and kudu. When they go back to the Sierra's they find the load is no longer as fantastic as it was, there have been a few posts aking how? What? Happened.
I used to be a fanatical clean my rifle to bare metal after every use kinda guy, I never had this problem. Later in my shooting life I got lazy and started leaving the rifle till 30 -50 shots later before decoppering - deep clean. It was about this time that I started noticing some bullet, load combo's did perfectly well in a clean rifle, but behaved badly when sighting in after changing back to the particular brand.
If you wish to see this more clearly, just look at the first few shots after changing brand of ammo when using a .22
A seasoned Bisley competitor once told me to check the hardness of the bullets or their jackets. When going to a harder bullet, decoppering is not crucial, as the harder bullet will scrape out the softer fouling. The other way round is problematic, as a softer bullet can not do much to some harder fouling sitting in the barrel, except to leave some more fouling on top of the hard fouling already in there.
Turned copper bullets are quite soft. Most imported copper jackets are harder. PMP jackets are harder than most others. Then we have the brass bullets that are harder than copper.
.22LR lead bullets have a similar story w.r.t. lead fouling, but the specific lube also plays a role.