Here are the reasons why a cat might not want to be held but always follows its owner:

Cats are slow to warm up and have relatively reserved emotions. Even if they love their owners very much, they express their feelings in a rather subtle way.
Cats don't like being restrained. When a cat follows its owner, it's very likely that it's apologizing to its owner, fearing punishment if the owner picks it up.
Cats don't like physical contact. Every cat has a different personality. Some cats like to follow their owners but don't like to be held. This is also a way of showing their love for their owners, but they tend to protect them more quietly.