Commit graph

4640 commits

Author SHA1 Message Date
Marko Viitanen 0239572796 [ibc] Fix some instances where CU_INTER was checked instead of !CU_INTRA 2023-08-23 15:21:45 +03:00
Marko Viitanen 312ac6731c [ibc] dual-tree rebase fixes 2023-08-15 13:24:22 +03:00
Joose Sainio 805afb1331 [fix] Minor fixes 2023-08-15 13:11:50 +03:00
Joose Sainio 8c14fa94ba [mtt] Fix small issues with luma and chroma searches 2023-08-15 13:11:49 +03:00
Joose Sainio 7a5245c5a4 [dual-tree] Fix chroma tree split model context derivation during search 2023-08-15 13:11:31 +03:00
Joose Sainio 707e11dbcf [dual-tree] Small fixes 2023-08-15 13:11:30 +03:00
Joose Sainio 91591c7e7c [dual-tree] Remove the limitation of not allowing 2 height chroma blocks in dual tree 2023-08-15 13:11:29 +03:00
Joose Sainio 146e1cb85e [dual-tree] WIP simplification 2023-08-15 13:11:28 +03:00
Joose Sainio 0f50caa2d0 [mtt] Fix various small issues and DepQuant for non-square blocks 2023-08-15 13:11:27 +03:00
Joose Sainio d222718c22 [mtt] Minor fixes 2023-08-15 13:11:26 +03:00
Joose Sainio d69bdf79f4 [mtt] Fix couple of issues with 64x32 CUs and non square tr skip rdoq 2023-08-15 13:10:13 +03:00
Joose Sainio 7d787c6b22 [ISP] Fix ISP cost calculation and DepQuant with mts 2023-08-15 13:10:13 +03:00
Joose Sainio 6e24b9a7f9 [DepQuant] Fix isp+depquant and trskip + isp 2023-08-15 13:10:12 +03:00
Joose Sainio 93c7e9c296 [DepQuant] Fix for mts and lfnst being quantized incorrectly during search 2023-08-15 13:10:11 +03:00
Joose Sainio dc652c75f9 [DepQuant] Isp and chroma 2023-08-15 13:10:10 +03:00
Joose Sainio 505c26eef3 [DepQuant] Fix 2023-08-15 13:10:10 +03:00
Joose Sainio c6087230a8 [DepQuant] Fix 2023-08-15 13:10:09 +03:00
Joose Sainio 5abe9e57c6 [DepQuant] Working but not necessarily improving 2023-08-15 13:10:09 +03:00
Joose Sainio 5236bc93be [DepQuant] WIP: doesn't crash but bitstream is illegal and quality a lot worse 2023-08-15 13:10:08 +03:00
Joose Sainio bfa699fac6 [DepQuant] WPP: API 2023-08-15 13:10:07 +03:00
Joose Sainio f8994a7fae [DepQuant] WIP: dequant 2023-08-15 13:08:54 +03:00
Joose Sainio 3e66a897d4 [DepQuant] WIP: compiles 2023-08-15 13:08:53 +03:00
Joose Sainio 4dbe0cd6c3 [DepQuant] WIP: easy part done 2023-08-15 13:08:24 +03:00
Joose Sainio 2a33af283e [DepQuant] WIP: initialization done 2023-08-15 13:08:24 +03:00
Joose Sainio 1373a7ac1d [mtt] correct indexing for chroma tree 2023-08-15 13:08:23 +03:00
Joose Sainio d3f42949a7 [mtt] Only consider termination if the cu is completely inside the frame 2023-08-15 13:08:22 +03:00
Joose Sainio 0c63743fc0 [mtt] Early terminations for all intra 2023-08-15 13:08:21 +03:00
Joose Sainio bd3ec75173 [mtt] search early terminations 2023-08-15 13:08:20 +03:00
Joose Sainio 2d00cab4b9 [isp] properly reset cabac context during intra search 2023-08-15 13:08:20 +03:00
Joose Sainio b27eca7c37 [deblock] fix width and height to correct order 2023-08-15 13:08:19 +03:00
Joose Sainio eae7d72384 [isp] Keep cabac contexts up to date for the different isp tus 2023-08-15 13:08:18 +03:00
Joose Sainio c744f79117 [mtt] Fix rdoq for non-square blocks 2023-08-15 13:08:17 +03:00
Joose Sainio 3b09c66d25 [deblock] Use the isp block dimensions instead of cu dimensions fro deblock 2023-08-15 13:08:17 +03:00
Joose Sainio 73956a9a46 [isp] Fix isp bitcost calculation 2023-08-15 13:08:16 +03:00
Joose Sainio f3c8a4f5db [lfnst] Also chroma can only use lfnst if dimensions are minimum 4 2023-08-15 13:08:15 +03:00
Joose Sainio a36a1fb5ff [mtt] There is always at least the height or width amount reference pixels available 2023-08-15 13:08:15 +03:00
Joose Sainio af23c81afa [mtt] Fix reading uninitialized data for local chroma tree 2023-08-15 13:08:14 +03:00
Joose Sainio 9acdab3209 [mtt] Fix lfnst bit counting for 64 wide or tall chroma tree cus 2023-08-15 13:08:13 +03:00
Joose Sainio 812377db45 [mtt] Set cus outside of the frame to zero for initializing partial worktree 2023-08-15 13:08:12 +03:00
Joose Sainio 27d114bc08 [mtt] Fix negative indexing 2023-08-15 13:08:11 +03:00
Joose Sainio 4e203108bc [mtt] Fix ref pixel generation for the second half of 32x2 chroma cus 2023-08-15 13:08:11 +03:00
Joose Sainio 446c53fd00 [mtt] Fix cclm for non 64 divisible heights 2023-08-15 13:08:10 +03:00
Joose Sainio ad2bb20f23 [mtt] Fix deblock for isp and properly set the limit for cclm 2023-08-15 13:08:09 +03:00
Joose Sainio c89ebf8bf1 [cclm] Fix heap corruption for non 64 divisible frames 2023-08-15 13:08:08 +03:00
Joose Sainio d296cac7c3 [mtt] fix reference building for 16x1 2023-08-15 13:08:07 +03:00
Joose Sainio 926ed7e145 [rdoq] partly fix rdoq for 16x1 and 1x16 2023-08-15 13:08:06 +03:00
Joose Sainio 8e4b864e6b [deblock] Fix incorrect direction for transform split of tall blocks at the top CTU row also for chroma 2023-08-15 13:08:05 +03:00
Joose Sainio 34aed10ec1 [mtt] fix 2023-08-15 13:08:05 +03:00
Joose Sainio 1333ab55d9 [mtt] Fix ref building for 32x64 cus 2023-08-15 13:08:04 +03:00
Joose Sainio 1493a2616c [mtt] fix getting collocated chroma for edge cus 2023-08-15 13:08:03 +03:00