In this paper we contribute a simple yet effective ap-proach for estimating 3D poses of multiple people frommulti-view images. Our proposed coarse-to-fine pipelinefirst aggregates noisy 2D observations from multiple cam-era views into 3D space and then associates them into indi-vidual instances based on a confidence-aware majority vot-ing technique. The final pose estimates are attained froma novel optimization scheme which links high-confidencemulti-view 2D observations and 3D joint candidates. More-over, a statistical parametric body model such as SMPL isleveraged as a regularizing prior for these 3D joint candi-dates. Specifically, both 3D poses and SMPL parametersare optimized jointly in an alternating fashion. Here theparametric models help in correcting implausible 3D poseestimates and filling in missing joint detections while up-dated 3D poses in turn guide obtaining better SMPL esti-mations. By linking 2D and 3D observations, our methodis both accurate and generalizes to different data sourcesbecause it better decouples the final 3D pose from the inter-person constellation and is more robust to noisy 2D de-tections. We systematically evaluate our method on publicdatasets and achieve state-of-the-art performance.