A muzzle loading firearm has a barrel with a bore on a bore axis, and the barrel has a muzzle end and a breech end. A frame is connected to the barrel, and has a breech face. The frame moves between an open position in which the breech face is away from the breech end of the barrel, and a closed position in which the breech face abuts the breech end of the barrel. A breech plug is removably attached to the barrel. The breech plug including a seal element closely received by the bore. The seal element may be a set of piston rings that are received in a circumferential groove about a forward end of the plug, or may be a cup at the forward end, with a forward rim that flares under pressure to provide a gas seal.