A portable building is disclosed, having a wall section including a first upwardly extending support member, a second upwardly extending support member and a header supported by the first upwardly extending support member and the second upwardly extending support member, the header having a header channel. The wall section also includes a floor beam secured to the first upwardly extending support member and the second upwardly extending support member, the floor beam having a floor beam channel. Further, the wall section includes a plurality of wall panels, where each of the plurality of wall panels are removably retained within the header channel and the floor beam channel, such that each of the plurality of wall panels are slidable along the floor beam and the header while retained within the header channel and the floor beam channel. Each of the plurality of wall panels are configured to interlockingly engage an immediately adjacent wall panel and an exterior surface of each of the plurality of wall panels form a continuous exterior wall surface.