SIP Express router or Kamailio would be good choices. Asterisk is could also do it, but since its more of a PBX model, it has a lot more unneeded overhead for this kind of function.
Trixbox is a version of Asterisk with a ease-of-use layer wrapped around it.
Its an easier way to get started with Asterisk. If you can setup a Trixbox/Asterisk server with a public IP address, then the remote users can just connect to it with either with VOIP hardware ( an ATA or IP phone). Alternately they could just use free VOIP software on their PCs (e.g. x-lite). Once they are connected with the Asterisk box, then they can talk to each other, use voicemail, etc.