Everything you mention is Pop (popular) music, no?

Popular music becomes popular when it provides two elements - A Hook (ear worm if you prefer) and a Beat (groove...).

Even Herbie Hancock had a big hit outside of his Jazz career, I saw him on the Headhunters tour. He never would have filled an arena like that with his beautiful and incredible jazz. "Hard Funk Jazz"?

"Genres" are both marketing and comfort zones.

I rememver when New Wave came out. In Central California, the Pretenders, Talking Heads, The Cars, Devo, Elvis Costello and The Divinyls were all considered New Wave.
All VERY different music, songs and subject matter. Each of these bands was extraordinary in their own way but there are many significant differences in style, sound, etc. For all that, they were lumped onto one genre. The same thing happened with Sympho-Rock - Yes, Emerson Lake & Palmer, UK, Genesis, all lumped into the same category and you had to be British or at least European to be part of that genre.

All their most popular songs, the ones you may still hear today, have great Hooks and Beats.

Genres need to provide a certain "comfort". So Alternative has Alt-Rock, Alt-Country etc. (but not Alt-Alt?). Progressive becomes Prog-Rock and so forth.
Then we have Sub-Genres, probably umpty bajillion of them.

Some become Shape Shifters - The Beatles, David Bowie, Joni Mitchell all created very different music if their careers are taken as a whole.

Many attempted Shape Shifting and ended up creating a new sound - when I saw Jason and The Scorchers it was more or less Hard-Country-Punk-Metal or something of that sort.

One of the more current movements I check out and find interesting is "Guitarists who do a whole bunch of things". Michael Hedges was a pioneer in this "genre" but others have taken it farther still.
This is from 2011, Jon Gom is pretty amazing and sounds like Jon Gom. It is a blend of many influences. https://www.youtube.com/watch?v=nY7GnAq6Znw
Yasmin Williams is fun, I am enjoying listening to her evolution.

So, what do we do about the music of Captain Beefheart? :- D

It took a chunk of my life to get here and I am still not sure where "here" is.