Commit graph

4016 commits

Author SHA1 Message Date
Joose Sainio dfff9a8030 [avx2] Move dep quant stuff to strategies 2023-09-26 09:20:55 +03:00
Joose Sainio 0591342b3a [avx2] replace or 2023-09-26 09:20:38 +03:00
Joose Sainio 8b1d6fab59 [avx2] Replace loads and stores with non-avx512 stores 2023-09-26 09:20:37 +03:00
Joose Sainio 6d0a3fa5fc [avx2] Replace _mm_and_epi32 with _mm_and_si128 2023-09-26 09:20:37 +03:00
Joose Sainio 7fdc045690 [dep_quant] Clean up 2023-09-26 09:20:37 +03:00
Joose Sainio 8eb0f66734 [depquant] update_state_eos_avx2 working 2023-09-26 09:20:37 +03:00
Joose Sainio 00cc58bc55 [depquant] Only initialize rate_estimator when necessary 2023-09-26 09:20:37 +03:00
Joose Sainio 00f838306f [depquant] Initialize quant_block only when necessary 2023-09-26 09:20:37 +03:00
Joose Sainio 9e27b4056a [avx2] WIP update_state_eos_avx2 2023-09-26 09:20:36 +03:00
Joose Sainio c56350b8d6 [avx2] and last 2023-09-26 09:20:36 +03:00
Joose Sainio 9f69713c24 [depquant] remove an unnecessary memcpy 2023-09-26 09:20:36 +03:00
Joose Sainio aa48943c22 [avx2] Do decision cost comparison with avx2 2023-09-26 09:20:36 +03:00
Joose Sainio cd6110cfac [depquant] Pre calculate things
sig_ctx_offset gtx_ctx_offset cg_pos pos_y  pos_x next_sbb_right next_sbb_below
2023-09-26 09:20:30 +03:00
Joose Sainio 8f4c3cecbf [avx2] update_states_avx2 working 2023-09-26 09:20:29 +03:00
Joose Sainio 58a66c0654 [avx2] WIP update_states_avx2 2023-09-26 09:20:29 +03:00
Joose Sainio 04be92a8ec [avx2] simplify 2023-09-26 09:20:29 +03:00
Joose Sainio 8b19c468cf [avx2] check_rd_costs_avx2 done 2023-09-26 09:20:29 +03:00
Joose Sainio c6e6f5da33 [avx2] WIP check_rd_costs_avx2, almost? 2023-09-26 09:20:29 +03:00
Joose Sainio 8caabcde1a [avx2] WIP check_rd_costs_avx2 2023-09-26 09:20:28 +03:00
Joose Sainio 2912db5fca [dep_quant.c] Small refactor 2023-09-26 09:20:28 +03:00
Joose Sainio 64d34f8559 [depquant] AoS -> SoA pre quant 2023-09-26 09:20:28 +03:00
Joose Sainio 2f1e9c4020 [depquant] AoS -> SoA all states 2023-09-26 09:20:28 +03:00
Joose Sainio 73442f1bba [depquant] AoS -> SoA for Decision 2023-09-26 09:20:28 +03:00
Marko Viitanen 26ef1dda09 [ibc] Fix chroma SAD handling and disable chroma SAD for now 2023-08-30 15:06:08 +03:00
Marko Viitanen 0239572796 [ibc] Fix some instances where CU_INTER was checked instead of !CU_INTRA 2023-08-23 15:21:45 +03:00
Marko Viitanen 312ac6731c [ibc] dual-tree rebase fixes 2023-08-15 13:24:22 +03:00
Joose Sainio 805afb1331 [fix] Minor fixes 2023-08-15 13:11:50 +03:00
Joose Sainio 8c14fa94ba [mtt] Fix small issues with luma and chroma searches 2023-08-15 13:11:49 +03:00
Joose Sainio 7a5245c5a4 [dual-tree] Fix chroma tree split model context derivation during search 2023-08-15 13:11:31 +03:00
Joose Sainio 707e11dbcf [dual-tree] Small fixes 2023-08-15 13:11:30 +03:00
Joose Sainio 91591c7e7c [dual-tree] Remove the limitation of not allowing 2 height chroma blocks in dual tree 2023-08-15 13:11:29 +03:00
Joose Sainio 146e1cb85e [dual-tree] WIP simplification 2023-08-15 13:11:28 +03:00
Joose Sainio 0f50caa2d0 [mtt] Fix various small issues and DepQuant for non-square blocks 2023-08-15 13:11:27 +03:00
Joose Sainio d222718c22 [mtt] Minor fixes 2023-08-15 13:11:26 +03:00
Joose Sainio d69bdf79f4 [mtt] Fix couple of issues with 64x32 CUs and non square tr skip rdoq 2023-08-15 13:10:13 +03:00
Joose Sainio 7d787c6b22 [ISP] Fix ISP cost calculation and DepQuant with mts 2023-08-15 13:10:13 +03:00
Joose Sainio 6e24b9a7f9 [DepQuant] Fix isp+depquant and trskip + isp 2023-08-15 13:10:12 +03:00
Joose Sainio 93c7e9c296 [DepQuant] Fix for mts and lfnst being quantized incorrectly during search 2023-08-15 13:10:11 +03:00
Joose Sainio dc652c75f9 [DepQuant] Isp and chroma 2023-08-15 13:10:10 +03:00
Joose Sainio 505c26eef3 [DepQuant] Fix 2023-08-15 13:10:10 +03:00
Joose Sainio c6087230a8 [DepQuant] Fix 2023-08-15 13:10:09 +03:00
Joose Sainio 5abe9e57c6 [DepQuant] Working but not necessarily improving 2023-08-15 13:10:09 +03:00
Joose Sainio 5236bc93be [DepQuant] WIP: doesn't crash but bitstream is illegal and quality a lot worse 2023-08-15 13:10:08 +03:00
Joose Sainio bfa699fac6 [DepQuant] WPP: API 2023-08-15 13:10:07 +03:00
Joose Sainio f8994a7fae [DepQuant] WIP: dequant 2023-08-15 13:08:54 +03:00
Joose Sainio 3e66a897d4 [DepQuant] WIP: compiles 2023-08-15 13:08:53 +03:00
Joose Sainio 4dbe0cd6c3 [DepQuant] WIP: easy part done 2023-08-15 13:08:24 +03:00
Joose Sainio 2a33af283e [DepQuant] WIP: initialization done 2023-08-15 13:08:24 +03:00
Joose Sainio 1373a7ac1d [mtt] correct indexing for chroma tree 2023-08-15 13:08:23 +03:00
Joose Sainio d3f42949a7 [mtt] Only consider termination if the cu is completely inside the frame 2023-08-15 13:08:22 +03:00
Joose Sainio 0c63743fc0 [mtt] Early terminations for all intra 2023-08-15 13:08:21 +03:00
Joose Sainio bd3ec75173 [mtt] search early terminations 2023-08-15 13:08:20 +03:00
Joose Sainio 2d00cab4b9 [isp] properly reset cabac context during intra search 2023-08-15 13:08:20 +03:00
Joose Sainio b27eca7c37 [deblock] fix width and height to correct order 2023-08-15 13:08:19 +03:00
Joose Sainio eae7d72384 [isp] Keep cabac contexts up to date for the different isp tus 2023-08-15 13:08:18 +03:00
Joose Sainio c744f79117 [mtt] Fix rdoq for non-square blocks 2023-08-15 13:08:17 +03:00
Joose Sainio 3b09c66d25 [deblock] Use the isp block dimensions instead of cu dimensions fro deblock 2023-08-15 13:08:17 +03:00
Joose Sainio 73956a9a46 [isp] Fix isp bitcost calculation 2023-08-15 13:08:16 +03:00
Joose Sainio f3c8a4f5db [lfnst] Also chroma can only use lfnst if dimensions are minimum 4 2023-08-15 13:08:15 +03:00
Joose Sainio a36a1fb5ff [mtt] There is always at least the height or width amount reference pixels available 2023-08-15 13:08:15 +03:00
Joose Sainio af23c81afa [mtt] Fix reading uninitialized data for local chroma tree 2023-08-15 13:08:14 +03:00
Joose Sainio 9acdab3209 [mtt] Fix lfnst bit counting for 64 wide or tall chroma tree cus 2023-08-15 13:08:13 +03:00
Joose Sainio 812377db45 [mtt] Set cus outside of the frame to zero for initializing partial worktree 2023-08-15 13:08:12 +03:00
Joose Sainio 27d114bc08 [mtt] Fix negative indexing 2023-08-15 13:08:11 +03:00
Joose Sainio 4e203108bc [mtt] Fix ref pixel generation for the second half of 32x2 chroma cus 2023-08-15 13:08:11 +03:00
Joose Sainio 446c53fd00 [mtt] Fix cclm for non 64 divisible heights 2023-08-15 13:08:10 +03:00
Joose Sainio ad2bb20f23 [mtt] Fix deblock for isp and properly set the limit for cclm 2023-08-15 13:08:09 +03:00
Joose Sainio c89ebf8bf1 [cclm] Fix heap corruption for non 64 divisible frames 2023-08-15 13:08:08 +03:00
Joose Sainio d296cac7c3 [mtt] fix reference building for 16x1 2023-08-15 13:08:07 +03:00
Joose Sainio 926ed7e145 [rdoq] partly fix rdoq for 16x1 and 1x16 2023-08-15 13:08:06 +03:00
Joose Sainio 8e4b864e6b [deblock] Fix incorrect direction for transform split of tall blocks at the top CTU row also for chroma 2023-08-15 13:08:05 +03:00
Joose Sainio 34aed10ec1 [mtt] fix 2023-08-15 13:08:05 +03:00
Joose Sainio 1333ab55d9 [mtt] Fix ref building for 32x64 cus 2023-08-15 13:08:04 +03:00
Joose Sainio 1493a2616c [mtt] fix getting collocated chroma for edge cus 2023-08-15 13:08:03 +03:00
Joose Sainio ffe17e48d7 [mtt] minor fixes 2023-08-15 13:08:02 +03:00
Joose Sainio 06fa86c340 [isp] Fix coordinates 2023-08-15 13:07:59 +03:00
Joose Sainio 71516b8155 [mtt] Make sure mtt splits cannot reach a situation where search cannot be performed 2023-08-15 13:07:58 +03:00
Joose Sainio 23e6b9f56c [mtt] Check that we are inside the CTU before checking the ctu data 2023-08-15 13:07:58 +03:00
Joose Sainio facbc794bf [mtt] Fix trying to get split data from depth -1 2023-08-15 13:07:57 +03:00
Joose Sainio 567fa7b2bd [deblock] Fix incorrect direction for transform split of tall blocks at the top CTU row 2023-08-15 13:07:56 +03:00
Joose Sainio 9c2574880a [mtt] Fix deblock for --combine-intra 2023-08-15 13:07:55 +03:00
Joose Sainio 90ce1390c0 [mtt] static 2023-08-15 13:07:54 +03:00
Joose Sainio f6ecb15ced [mtt] Fix implicit splits when mtt is not enabled 2023-08-15 13:07:53 +03:00
Joose Sainio 05218bae21 [jccr] jccr=4 hasn't been necessary for a long time 2023-08-15 13:07:52 +03:00
Joose Sainio b69e9b2958 [mtt] Fix final issues? 2023-08-15 13:07:51 +03:00
Joose Sainio 6620ba8d76 [mtt] fix deblock 2023-08-15 13:07:50 +03:00
Joose Sainio 09baddef17 [mtt] Fix lfnst and chroma coeffs and tests 2023-08-15 13:07:49 +03:00
Joose Sainio 992182dafb WIP 2023-08-15 13:07:48 +03:00
Joose Sainio ba0d43d846 [mtt] Fill chroma data for the whole area covered by the local separate tree chroma cu 2023-08-15 13:07:47 +03:00
Joose Sainio 412dd20f09 [mtt] Fix implicit splits for non ctu divisible frames. 2023-08-15 13:07:46 +03:00
Joose Sainio 2da1a34ff3 [mtt] Fix isp for MTT 2023-08-15 13:07:45 +03:00
Joose Sainio b988c60dd1 [mtt] search works completely with everything except RDOQ deblock and ISP 2023-08-15 13:07:44 +03:00
Joose Sainio 6a6bed7f1f [mtt] WIP 2023-08-15 13:07:43 +03:00
Joose Sainio 065eb6fc03 [mtt] fix lfnst 2023-08-15 13:05:38 +03:00
Joose Sainio 9e644fafd0 [mtt] search with mtt depth 2 and dual tree works without lfnst 2023-08-15 13:05:37 +03:00
Joose Sainio fb146cb6ed [mtt] proper split availability checking for split flag 2023-08-15 13:05:35 +03:00
Joose Sainio d5d9afb1e2 [mtt] fix dual tree 2023-08-15 13:05:19 +03:00
Joose Sainio 8fbefc0de3 [mtt] fix cost calculation 2023-08-15 13:04:29 +03:00
Joose Sainio 657254d38a [mtt] search with depth 1 mtt kinda working 2023-08-15 13:04:28 +03:00
Joose Sainio 13aae7d03d [mtt] All individual mtt splits should be working + uvg_get_possible_splits 2023-08-15 13:04:27 +03:00