Apparently this feature was removed from Messages, which is strange because I know that for a lot of people, 3-way video on iChat used to be a huge selling point for Mac - that and Photo Booth were definitely two reasons high school and college students were buying Macs over PCs.