Still doesn't work. I have no idea what the problem is. Probably somehow related to the coefficient coding, since the bitstream seems to work, the prediction is correct and the error is not very severe.
- Change scan order selection to be more verbose and based on the correct mode for 4x4. Didn't affect the problem with 4x4 luma in any way although it should have.
- Re-enable residual coding as everything seems to work now besides 4x4 luma.
New implementation uses precalculated tables to look up number of reference
pixels available in coded CUs. Otherwise it works just like the previous
version.
- NxN mostly works. Prediction appears to be almost correct but there is a
slight error in the exact values, probably related to filtering.
- Set part-size for Inter.
- Change to Intra Only mode for testing.
- Many small changes here and there. Should have been separate commits probably, but too late.
- Disable SAO and deblocking to be able to see problems with reconstruction better.
- Adjust predictor list to take modes from PUs in addition to 2Nx2N CUs.
- Change intra_get_dir_luma_predictor to take PU index instead of CU index.
- Comment prediction encoding now that I've had to look it up.
- Clean up code and comment.
- Change terminology to match H.265 specification where possible.
- Move transform splitting for depth==0 out of the coding part. It's not
possible to do it here anyway because intra reconstruction is different
if the transform is split.
- Add checking for transform hierarchy depth when coding split flag.
- Fixes bug with cu_data.tr_depth being set. The CU struct was being reused
for inter coded CUs, which did not initialize the tr_depth.
- Add room to cu_data.coeff_top_yuv arrays for the 4x4 PUs data. Will probably have to do the same to other coeff flags. The flags could also probably be combined as they are a bit redundant.
- Doesn't work yet so it's disabled.
- Change encode_transform_coeff to accept PU (Prediction Unit) coordinates
instead of CU coordinates because CUs are 8x8.
Should work exactly the same, but with the prediction mode selection done
separately from the binarization it's easier to see that the implementation
is correct.
One I frame and 99 P frames encoded with SAO off and on.
Processed 100 frames, 6693224 bits AVG PSNR: 30.7248 37.8978 37.8287
Processed 100 frames, 6295072 bits AVG PSNR: 32.2511 38.9373 38.9818
