The buttons on the original playstation controller had a very distinct purpose in their design, according to their designer Teiyu Goto.
The triangle represented a person's viewpoint, meant to look like a head. The square was meant to symbolize a map. The circle and X were meant to represent "yes" and "no" respectively, and it was thought that games released on the Playstation would use controls based on the meaning of these symbols.