I’ve heard about that as well. Apparently, he wanted very badly to be taken as a serious composer, and the less than enthusiastic reception of his concerto really crushed him. I seem to recall an account of Leonard Bernstein hearing it, who apparently was not terribly impressed.
I saw an interview of him talking about his legacy, and he was unhappy that he was most recognized for Lucky Man, which he…disliked. He asked something like, “When people ask me what I’ve written that they’d recognize, what am I going to do - whistle Tarkus?”
dB