"Let's talk clutches"....LOL...I love to talk...or type.
Your Drifters came with ArcticCat "Hex" clutches..so called because of the hexagonal shape of the center bushing. Bombardier (SkiDoo) called their clutch during the same time frame a "square shaft" clutch....same deal...their center bushing was square shaped.
Both clutches worked great when they were new. Trouble was, they transmitted all of the engine torque right near the crankshaft centerline, at those center bushings, which led to rapid wear, and binding of the shifting as those center bushings wore.
Polaris came out with a better idea, which Comet, Kawasaki (with their own in house clutch) and everyone else adapted, that moved the torque transfer points out to the outer perimeter of the drive clutch, with plastic buttons or shoes making contact with an aluminum surface. This led to much more efficient torque transfer, longer life, and cooler operation. They still have center bushings, but their only function was a sliding surface for the moveable sheeve to slide in and out.
Back to your Hex clutch....grab the inner sheave of your drive clutch with one hand, and try to rock the outer sheave back and forth. If you have much more than 1/8" play, it is suggested the center bushing needs replaced....yours probably has a lot more than that.
Clutches suck....you have to get the proper puller, hunt down replacement parts, etc. etc.
You probably need a new center bushing at the very least. You need to check the rollers and ramps to make sure nothing is too badly worn out, and the rollers are clean and spin freely. Your spring is probably OK, unless it's broken.
You might look and clean the secondary clutch. They are pretty simple critters compared to the drive clutch. Just make sure the spring isn't broken, and the 3 little plastic shoes aren't worn down to metal.
Kawasaki did start using their own clutch on the '81 440 Drifter. If you find a good used Kawi clutch, we could get the right ramps and weight figured out for your 340.....