As Colbert explained it, “I’ve always wanted to do the whistling solo to ‘Me and Julio Down By the Schoolyard’, and I called Paul Simon and I said, ‘can I do that?’, and he said, ‘I will not do that with you.’
So instead, Colbert invited the Paul Simon tribute band Trouble Waters to perform the song. After both he and the band’s singer publicly flogged the real Paul Simon (“He seems like a jerk. He seems like a huge jerk”), they commenced what proved to be a pretty faithful rendition. The whistling was particularly spot on, as you can see for yourself below.