This will be my 9th or 10th year at NOLA JF. The festival proper runs two consecutive weekends at the race track. This year it runs Thurs-Sun both weekends until around 7pm. But during the two week stretch all the music venues in town have shows that run all night and into the morning. Midweek when the track is closed there are free daytime in-store concerts at the Louisiana Music Factory, which is just an amazing record store on Frenchmen at Decatur. You can plan your late nights using the Jazzfest Grids.
There are music venues scattered all around the city, with several in the French Quarter, but the highest concentration by far is in the Marigny along Frenchmen St. I like to stay within a stone's throw of about Frenchmen x Chartres, as that puts you within stumbling distance of about 8 or 10 solid music venues. The Blue Nile is my favorite, with shows upstairs and downstairs nightly. So you can just stay out as late as you can handle it and have a short walk back home when you just caint do it any longer.
I forget which venue holds it, but there's usually a show called 'the Booker Thing' which is a tribute to James Booker with a rotating cast of keyboard players. Marco Benevento, Ivan Neville, Jon Cleary, etc.
You should also check out a show at the Maple Leaf Bar at some point. It'll be a cab ride uptown to get there, but it's one of those only-in-New-Orleans things that you have to check out. It's basically a house that was gutted and turned into a bar, with the one remaining load bearing wall separating the bar from the stage area. It is hot, and crowded, and a lot of fun. I always catch Bonerama at the Rock n Bowl, which is also a really fun venue.
Maybe we need to do a KC Jazzfest hang!