Any building which contains a permitted use
but which is nonconforming due to insufficient yard distances or lot
area shall not be considered a nonconforming use. Any alterations
or structural changes may be accomplished within the existing frame
of said building, or such building may be restored if damaged by fire
or by natural causes, but any additions shall conform to the specific
setback and yard distance requirements of this chapter. The use of
any such nonconforming building may be changed to any other permitted
use so long as the yard or lot area requirements are not greater.