Finally, the new build with custom tone mapping curves is ready for a first test:http://madshi.net/madVRhdrMeasure123b.zip
This was a HUGE amount of work. The key problem is that of course we want the curve to be optimal for scenes with high peak nits and low peak nits, with high FALL and low FALL. So how can we possibly handle that with custom curves? Well, I've solved it like this:
A) You can create custom curves for: 110, 509, 2230 and 10000 frame peak nits. These are 4 different curves. And madVR will seamlessly interpolate between them, depending on the measured peak of each video frame.
B) You can create custom curves for 1x, 2x, 4x, 8x and 16x dynamic tone mapping compression. These are 5 different curves. E.g. if your real display nits is 100nits and madVR's dynamic target nits (FALL algo) wants to render the current frame with a 200nits target, then you have a 2x factor, so the 2x curve is used. If the factor is 1.5x, madVR will interpolate the 1x and 2x curves.
Sadly, we want A) and B) to work at the same time, so there are actually 4 * 5 curves you can define. OUCH. I'm sorry about that, but I couldn't think of any other way to make everything work the way we're used to, but still allow fully custom tone mapping curves.
By default, if you just enter the custom curve editor and save the settings without changing anything, the image you get should be roughly the same as without the custom curves. Meaning, the custom curves default to what madVR does right now. But you now have full flexibility over the curves.
Editing curves is possible in various ways:
1) You can change the brightness compression factor.
2) You can change the knee (compression starting point) - but only as far as BT.2390 can handle.
3) You can change the curve steepness - but only as far as BT.2390 can handle.
4) You can apply a contrast modifier (positive or negative) to either the top or bottom of the curve.
5) If all of the above isn't enough yet, you can manually move every of the 21 curve points for each curve up and down to design a fully custom curve.
Please note that the curves you create are made for your real display peak nits. Your curves might not work for another user who has a different display peak nits. Maybe we can find a way to unify this somehow, but I'm not sure it's possible because a user with a 50nits projector will probably want to use different custom curves than a user with an 800nits LED or OLED.
Please also note that this is an experimental build and not necessarily meant for actual movie watching (although you can try). One thing not clearly defined yet is how madVR switches from simply clipping (if the movie peak is below your real display peak nits) to the custom curves. I assume it works well enough as long as you stick with the default curves. But I might need to add special handling in case you do strong changes (e.g. already doing 2:1 compression even if almost no tone mapping is needed).
P.S: Please understand that for every video frame, the tone mapping curve will usually be interpolated from 4 custom curves. E.g. if the frame peak is 300nits and madVR wants to render it with 1.5x brightness compression, then the tone mapping curve will sit right in the middle of these 4 curves: 110nits_1x, 110nits_2x, 509nits_1x, 509nits_2x. Of course this makes testing more complicated because you can't just modify 1 curve to check how it looks like. You may have to edit all 4. Unless you pick a scene which sits near to these thresholds. E.g. if you pick a scene with has near to 509nits and madVR wants to nearly compress by something near to 2.0x, then you can test by just modifying the 509nits_2x curve and the other 3 curves will have only very little impact.