If you swing a sword from horseback, you're not charging. And using a sword, you use your own damage score. When using a lance or a spear, you use your horse's damage score.
In this situation, for the first round, you'll have a +10 to your effective skill. Subsequent rounds you will have a +5.
Also note that, to anyone with an effective skill over 20, you take the excess above 20 and apply it as a bonus to the actual attack roll. Any result you get 20+ is a critical.