The +5 circumstance modifier is just added to your skill, not the actual "to hit" roll.
If your Sword skill is normally 19 and you have a +5 bonus, then your effective Sword skill is 24. That being said, your Sword skill for "to hit" purposes is 20, but with a +4 bonus to the skill. So even if you rolled an unmodified 20, there's no way you'd fumble your weapon, and in a competing sword check against your opponent, you'd get a +4 bonus. On top of all of that, you'd get a critical success on a roll of 16-20.
Actually, there's a situation exactly like the one you're describing on p. 95 of the core rulebook (under Values greater than 20).