The buttons on the original playstation controller had a very distinct purpose in their design, according to their designer Teiyu Goto.
The triangle represented a person's viewpoint, meant to look like a head. The square was meant to symbolize a map. The circle and X were meant to represent "yes" and "no" respectively, and it was thought that games released on the Playstation would use controls based on the meaning of these symbols.
Contributed by CosmykTheDolfyn