Commit graph

4580 commits

Author SHA1 Message Date
Joose Sainio d69bdf79f4 [mtt] Fix couple of issues with 64x32 CUs and non square tr skip rdoq 2023-08-15 13:10:13 +03:00
Joose Sainio 7d787c6b22 [ISP] Fix ISP cost calculation and DepQuant with mts 2023-08-15 13:10:13 +03:00
Joose Sainio 6e24b9a7f9 [DepQuant] Fix isp+depquant and trskip + isp 2023-08-15 13:10:12 +03:00
Joose Sainio 93c7e9c296 [DepQuant] Fix for mts and lfnst being quantized incorrectly during search 2023-08-15 13:10:11 +03:00
Joose Sainio dc652c75f9 [DepQuant] Isp and chroma 2023-08-15 13:10:10 +03:00
Joose Sainio 505c26eef3 [DepQuant] Fix 2023-08-15 13:10:10 +03:00
Joose Sainio c6087230a8 [DepQuant] Fix 2023-08-15 13:10:09 +03:00
Joose Sainio 5abe9e57c6 [DepQuant] Working but not necessarily improving 2023-08-15 13:10:09 +03:00
Joose Sainio 5236bc93be [DepQuant] WIP: doesn't crash but bitstream is illegal and quality a lot worse 2023-08-15 13:10:08 +03:00
Joose Sainio bfa699fac6 [DepQuant] WPP: API 2023-08-15 13:10:07 +03:00
Joose Sainio f8994a7fae [DepQuant] WIP: dequant 2023-08-15 13:08:54 +03:00
Joose Sainio 3e66a897d4 [DepQuant] WIP: compiles 2023-08-15 13:08:53 +03:00
Joose Sainio 4dbe0cd6c3 [DepQuant] WIP: easy part done 2023-08-15 13:08:24 +03:00
Joose Sainio 2a33af283e [DepQuant] WIP: initialization done 2023-08-15 13:08:24 +03:00
Joose Sainio 1373a7ac1d [mtt] correct indexing for chroma tree 2023-08-15 13:08:23 +03:00
Joose Sainio d3f42949a7 [mtt] Only consider termination if the cu is completely inside the frame 2023-08-15 13:08:22 +03:00
Joose Sainio 0c63743fc0 [mtt] Early terminations for all intra 2023-08-15 13:08:21 +03:00
Joose Sainio bd3ec75173 [mtt] search early terminations 2023-08-15 13:08:20 +03:00
Joose Sainio 2d00cab4b9 [isp] properly reset cabac context during intra search 2023-08-15 13:08:20 +03:00
Joose Sainio b27eca7c37 [deblock] fix width and height to correct order 2023-08-15 13:08:19 +03:00
Joose Sainio eae7d72384 [isp] Keep cabac contexts up to date for the different isp tus 2023-08-15 13:08:18 +03:00
Joose Sainio c744f79117 [mtt] Fix rdoq for non-square blocks 2023-08-15 13:08:17 +03:00
Joose Sainio 3b09c66d25 [deblock] Use the isp block dimensions instead of cu dimensions fro deblock 2023-08-15 13:08:17 +03:00
Joose Sainio 73956a9a46 [isp] Fix isp bitcost calculation 2023-08-15 13:08:16 +03:00
Joose Sainio f3c8a4f5db [lfnst] Also chroma can only use lfnst if dimensions are minimum 4 2023-08-15 13:08:15 +03:00
Joose Sainio a36a1fb5ff [mtt] There is always at least the height or width amount reference pixels available 2023-08-15 13:08:15 +03:00
Joose Sainio af23c81afa [mtt] Fix reading uninitialized data for local chroma tree 2023-08-15 13:08:14 +03:00
Joose Sainio 9acdab3209 [mtt] Fix lfnst bit counting for 64 wide or tall chroma tree cus 2023-08-15 13:08:13 +03:00
Joose Sainio 812377db45 [mtt] Set cus outside of the frame to zero for initializing partial worktree 2023-08-15 13:08:12 +03:00
Joose Sainio 27d114bc08 [mtt] Fix negative indexing 2023-08-15 13:08:11 +03:00
Joose Sainio 4e203108bc [mtt] Fix ref pixel generation for the second half of 32x2 chroma cus 2023-08-15 13:08:11 +03:00
Joose Sainio 446c53fd00 [mtt] Fix cclm for non 64 divisible heights 2023-08-15 13:08:10 +03:00
Joose Sainio ad2bb20f23 [mtt] Fix deblock for isp and properly set the limit for cclm 2023-08-15 13:08:09 +03:00
Joose Sainio c89ebf8bf1 [cclm] Fix heap corruption for non 64 divisible frames 2023-08-15 13:08:08 +03:00
Joose Sainio d296cac7c3 [mtt] fix reference building for 16x1 2023-08-15 13:08:07 +03:00
Joose Sainio 926ed7e145 [rdoq] partly fix rdoq for 16x1 and 1x16 2023-08-15 13:08:06 +03:00
Joose Sainio 8e4b864e6b [deblock] Fix incorrect direction for transform split of tall blocks at the top CTU row also for chroma 2023-08-15 13:08:05 +03:00
Joose Sainio 34aed10ec1 [mtt] fix 2023-08-15 13:08:05 +03:00
Joose Sainio 1333ab55d9 [mtt] Fix ref building for 32x64 cus 2023-08-15 13:08:04 +03:00
Joose Sainio 1493a2616c [mtt] fix getting collocated chroma for edge cus 2023-08-15 13:08:03 +03:00
Joose Sainio ffe17e48d7 [mtt] minor fixes 2023-08-15 13:08:02 +03:00
Joose Sainio 74591cd39b [tests] no-cpuid for test_cabac_state.sh since it now has mtt in it 2023-08-15 13:08:02 +03:00
Joose Sainio 972670a177 [tests] chmod +x 2023-08-15 13:08:01 +03:00
Joose Sainio 5aa13ad62a [tests] add tests for mtt 2023-08-15 13:08:00 +03:00
Joose Sainio 06fa86c340 [isp] Fix coordinates 2023-08-15 13:07:59 +03:00
Joose Sainio 71516b8155 [mtt] Make sure mtt splits cannot reach a situation where search cannot be performed 2023-08-15 13:07:58 +03:00
Joose Sainio 23e6b9f56c [mtt] Check that we are inside the CTU before checking the ctu data 2023-08-15 13:07:58 +03:00
Joose Sainio facbc794bf [mtt] Fix trying to get split data from depth -1 2023-08-15 13:07:57 +03:00
Joose Sainio 567fa7b2bd [deblock] Fix incorrect direction for transform split of tall blocks at the top CTU row 2023-08-15 13:07:56 +03:00
Joose Sainio 9c2574880a [mtt] Fix deblock for --combine-intra 2023-08-15 13:07:55 +03:00