Ranks in a weapon skill represent intense training your character has devoted to a particular category of weapon.
It is important to note that a character does not need to have ranks in a weapon to use it in combat, nor does the character take minuses for trying to use an unfamiliar weapon. If a character is unarmed and in danger, it makes perfect sense to pick up and wield whatever is nearby that could be considered painful to an attacker. Obviously, Attack Rolls with a weapon in which the character has no ranks will be fairly low.
A character's ranks in the weapon figure into combat maneuvers, such as attacking, parrying, disarming, and the like. See the Combat chapter for more details on the use of Weapon skill in combative rolls.