Any approved nonconforming use which existed at the time of the passage of this chapter may be continued on the lot or in the building so occupied, and any existing building designed, arranged, intended or devoted to a nonconforming use may be reconstructed or structurally altered, subject to the following regulations:
A. 
The structural alteration made in such building shall in no case be made without approval of the Board of Adjustment.
B. 
No nonconforming use if once changed into a conforming use, shall be changed back into a nonconforming use.
C. 
No nonconforming use shall be extended so as to diminish the extent of a conforming use.
D. 
Nothing herein contained shall require any change in plans, construction or designated use of a building for which a building permit has been heretofore issued and construction diligently prosecuted within 1/2 year of the date of such permit.
E. 
The restoration of a nonconforming building or use destroyed in any part will require the approval of the Board of Adjustment. This restoration will not be permitted except during the first year after the destruction.
F. 
Nothing in this chapter shall prevent the restoration of a wall or structure declared unsafe by the Building Inspector.