Commit graph

3908 commits

Author SHA1 Message Date
Joose Sainio c431529c0e [mtt] Fix cclm for non 64 divisible heights 2022-12-16 14:13:45 +02:00
Joose Sainio d5f78d6351 [mtt] Fix deblock for isp and properly set the limit for cclm 2022-12-16 09:37:56 +02:00
Joose Sainio 7038430646 [cclm] Fix heap corruption for non 64 divisible frames 2022-12-15 13:12:42 +02:00
Joose Sainio 62fabc60dd [mtt] fix reference building for 16x1 2022-12-15 12:41:12 +02:00
Joose Sainio 82f05ff280 [rdoq] partly fix rdoq for 16x1 and 1x16 2022-12-15 11:25:12 +02:00
Joose Sainio ef7c93ead9 [deblock] Fix incorrect direction for transform split of tall blocks at the top CTU row also for chroma 2022-12-15 10:07:47 +02:00
Joose Sainio a23e2ae5f6 [mtt] fix 2022-12-15 09:09:38 +02:00
Joose Sainio 6f80f4f1a2 [mtt] Fix ref building for 32x64 cus 2022-12-14 13:12:03 +02:00
Joose Sainio b5b2b72a24 [mtt] fix getting collocated chroma for edge cus 2022-12-14 12:33:24 +02:00
Joose Sainio 68f3191dd4 [mtt] minor fixes 2022-12-14 12:02:00 +02:00
Joose Sainio 2ccbb79b4d [isp] Fix coordinates 2022-12-14 10:54:52 +02:00
Joose Sainio f09d015ffe [mtt] Make sure mtt splits cannot reach a situation where search cannot be performed 2022-12-14 10:41:36 +02:00
Joose Sainio 6d6d69bb81 [mtt] Check that we are inside the CTU before checking the ctu data 2022-12-14 10:28:58 +02:00
Joose Sainio 01023c8a34 [mtt] Fix trying to get split data from depth -1 2022-12-14 10:16:51 +02:00
Joose Sainio 3d236153e3 [deblock] Fix incorrect direction for transform split of tall blocks at the top CTU row 2022-12-14 10:10:08 +02:00
Joose Sainio 1d7405ec28 [mtt] Fix deblock for --combine-intra 2022-12-14 09:25:54 +02:00
Joose Sainio d05a726a12 [mtt] static 2022-12-14 09:11:40 +02:00
Joose Sainio 73987d3882 [mtt] Fix implicit splits when mtt is not enabled 2022-12-14 09:01:52 +02:00
Joose Sainio 62c91e2e92 [jccr] jccr=4 hasn't been necessary for a long time 2022-12-14 08:47:24 +02:00
Joose Sainio b46aef72f1 [mtt] Fix final issues? 2022-12-14 08:37:06 +02:00
Joose Sainio 0a8850f0e5 [mtt] fix deblock 2022-12-13 14:51:38 +02:00
Joose Sainio ef93fe6578 [mtt] Fix lfnst and chroma coeffs and tests 2022-12-12 10:05:17 +02:00
Joose Sainio b7e7616f89 WIP 2022-12-08 14:52:01 +02:00
Joose Sainio 92ace56c41 [mtt] Fill chroma data for the whole area covered by the local separate tree chroma cu 2022-12-08 10:34:51 +02:00
Joose Sainio 01aa1e8348 [mtt] Fix implicit splits for non ctu divisible frames. 2022-12-06 14:23:31 +02:00
Joose Sainio 2e809abace [mtt] Fix isp for MTT 2022-12-06 11:23:30 +02:00
Joose Sainio d004d81603 [mtt] search works completely with everything except RDOQ deblock and ISP 2022-12-02 13:56:38 +02:00
Joose Sainio 03d15759c1 [mtt] WIP 2022-12-01 14:38:35 +02:00
Joose Sainio 76b0b35c05 [mtt] fix lfnst 2022-11-30 13:26:48 +02:00
Joose Sainio 6344812c22 [mtt] search with mtt depth 2 and dual tree works without lfnst 2022-11-30 10:43:12 +02:00
Joose Sainio b770ee656a [mtt] proper split availability checking for split flag 2022-11-29 12:51:34 +02:00
Joose Sainio 665886dc52 [mtt] fix dual tree 2022-11-29 12:14:58 +02:00
Joose Sainio 95f7096a34 [mtt] fix cost calculation 2022-11-29 08:45:53 +02:00
Joose Sainio e1df38bbd8 [mtt] search with depth 1 mtt kinda working 2022-11-28 11:24:55 +02:00
Joose Sainio badb834ef9 [mtt] All individual mtt splits should be working + uvg_get_possible_splits 2022-11-25 13:59:45 +02:00
Joose Sainio 4c15c0e5ab [mtt] WIP 16x16 TT split 2022-11-24 15:04:57 +02:00
Joose Sainio 2c32c48a0f fix rebase 2022-11-24 12:50:19 +02:00
Joose Sainio 472baa894f [mtt] Single mtt split works for everything else, except 16x16 with TT 2022-11-24 09:33:12 +02:00
Joose Sainio 69f66626ed [mtt] 64x32 and 32x64 2022-11-24 09:29:55 +02:00
Joose Sainio baa7a3bb70 [mtt] Fix sqrt adjustment, cclm calculation on edges of CTU and waip for lfnst 2022-11-24 09:29:44 +02:00
Joose Sainio a45ad25dde [mtt] Fix counting the number of reference pixles and implement WAIP adjustment 2022-11-24 09:29:36 +02:00
Joose Sainio ecc1e3ec89 [mtt] WIP 2022-11-24 09:29:00 +02:00
Joose Sainio f1f46d6008 WIP 2022-11-24 09:26:10 +02:00
Joose Sainio 902ea84187 [mtt] square root adjustment for quantization 2022-11-24 09:23:14 +02:00
Joose Sainio 6fe09411c0 WIP 2022-11-24 09:23:14 +02:00
Joose Sainio 540cf1fa41 [mtt] WIP 2022-11-24 09:23:14 +02:00
Joose Sainio 8a86c8fe3c [mtt] remove work_tree 2022-11-24 09:23:14 +02:00
Joose Sainio 3e119d5338 [mtt] Remove work_tree_copy_down and change work_tree_copy_up not to require the whole work tree as input parameter 2022-11-24 09:23:13 +02:00
Joose Sainio 27726dc375 [mtt] fix dual tree 2022-11-24 09:23:13 +02:00
Joose Sainio df01d5488f [quant] fix fast coeff cost 2022-11-24 09:23:13 +02:00
Joose Sainio 80bd15e6c7 [mtt] fix 2022-11-24 09:22:17 +02:00
Joose Sainio 97001eab7f [mtt] fix 2022-11-24 09:22:17 +02:00
Joose Sainio 0b4d215456 [mtt] fix 2022-11-24 09:22:17 +02:00
Joose Sainio a2ddb630e4 [mtt] also copy top right CU 2022-11-24 09:22:16 +02:00
Joose Sainio 1f3d21362f [mtt] temporarily disable zero coeff rdo 2022-11-24 09:22:16 +02:00
Joose Sainio eda0ead6f4 [mtt] fix 2022-11-24 09:22:16 +02:00
Joose Sainio c9c331ed38 [mtt] fix 2022-11-24 09:22:16 +02:00
Joose Sainio d538061142 [mtt] Only initialize higher depth ctus partially 2022-11-24 09:22:16 +02:00
Joose Sainio 11abe38295 [transform] Simplify chroma transform search a bit 2022-11-24 09:22:16 +02:00
Joose Sainio ec72cbf3ea [mtt] Add function for easily getting all split cu_locs 2022-11-24 09:22:15 +02:00
Joose Sainio b6c36f07ac [mtt] Remove depth from cu_info_t 2022-11-24 09:22:15 +02:00
Joose Sainio 72196bbadb [mtt] Completely remove tr_depth 2022-11-24 09:20:50 +02:00
Joose Sainio 97507b6c30 [mtt] remove depth from cbf 2022-11-24 09:19:47 +02:00
Joose Sainio 97296022a3 [mtt] remove dependency to depth from deblock 2022-11-24 09:19:25 +02:00
Joose Sainio 6b15346841 [mtt] remove unnecessary depth dependency from split flag 2022-11-24 09:19:25 +02:00
Joose Sainio ce754ba991 [mtt] remove lfnst dependency to depth 2022-11-24 09:19:25 +02:00
Joose Sainio cdc3b40733 wip 2022-11-24 09:17:22 +02:00
Joose Sainio bfa91a009a [mtt] Actually remove the last width dependency to depth 2022-11-24 09:15:25 +02:00
Joose Sainio 2e81b73e12 [mtt] remove all rest usages of deriving width and height from depth 2022-11-24 09:12:59 +02:00
Joose Sainio 0134f07949 [mtt] change most if not all of search hierarchy to use cu_loc_t 2022-11-24 09:06:14 +02:00
siivonek 36cda011e0 [isp] Fix reference building. When ISP was in use, not enough samples were generated. Uninitialized memory was referenced. Fix some typos. 2022-10-12 16:19:22 +03:00
siivonek a637b44d31 [isp] Remove some obsolete TODOs and old commented out code. 2022-10-11 16:11:14 +03:00
siivonek ce2c9331de [isp] Fix some CI errors. Some const modifiers were discarded. 2022-10-03 13:13:56 +03:00
siivonek aa696dd17a [lfnst] Fix LFNST error when MIP enabled. 2022-09-29 15:41:33 +03:00
siivonek 0b44c429bd [isp] Fix lfnst constraint check when ISP is used. Remove some obsolete comments. 2022-09-28 12:38:02 +03:00
siivonek c4660d7686 [isp] Fix search. Best LFNST and MTS modes were not selected correctly for ISP modes. 2022-09-27 14:10:03 +03:00
siivonek db20e998b0 [isp] Fix transform selection when MTS & ISP is used. Wrong transform was selected. Change mts parameter name to better reflect its purpose. 2022-09-27 13:12:10 +03:00
siivonek bd2e588a22 [isp] Add lfnst asserts. Fix error in MTS search. Fix chroma lfnst index when no coefficients present. 2022-09-26 14:44:51 +03:00
siivonek 6975febccf [isp] Fix lfnst constraint checks when ISP is in use. Add some asserts. 2022-09-23 15:41:50 +03:00
siivonek 85029fc2ca [isp] Remove unnecessary code from forward dct 32. 2022-09-22 16:49:09 +03:00
siivonek 785ae95302 [isp] Use TR_MAX_WIDTH in ISP checks instead of parameter. 2022-09-22 14:28:40 +03:00
siivonek 18fca3798b [isp] Limit ISP search to block size 32. Size 64 is not allowed. 2022-09-21 12:55:07 +03:00
siivonek e9c3351460 [isp] Add ISP checks to search. LFNST can be used with ISP for larger blocks. Transform skip cannot be used with ISP. 2022-09-20 17:44:10 +03:00
siivonek d553bbba8a [isp] Fix CI errors. 2022-09-20 16:15:23 +03:00
siivonek 80945c985a [isp] Fix cabac issues. There are always four transform blocks even if there are only two ISP splits. Fix prediction issues. PDPC filter was applied when it should be disabled. Fix reference building issues. Left reference was built incorrectly for blocks with height 2. 2022-09-20 15:16:14 +03:00
siivonek e8779fc175 [isp] Fix storing cbfs for small ISP splits. Fix pdpc filtering. Cannot be used if width or height is less than 4. Fix dct related CI errors. 2022-09-16 15:46:31 +03:00
siivonek c46abbdaa1 [isp] Fix some asserts to allow log2_dim 1 block sizes. Fix coefficient group scan order for small dimensions. 2022-09-16 10:37:51 +03:00
siivonek 19106bfced [isp] Implement DCT for small blocks. 2022-09-15 13:45:45 +03:00
siivonek c894e9c6b9 [isp] Fix referene building for depth 2 blocks. Flip horizontal mode dimensions during prediction. Fix reference length during prediction when ISP enabled. 2022-09-14 15:48:49 +03:00
siivonek 124cdd4e4a [isp] Fix luma cbf writing for ISP splits. Do not write luma cbf if first three splits had luma cbf 0. 2022-09-13 13:27:16 +03:00
siivonek 75d4de20e3 [isp] Fix CI errors. 2022-09-12 15:29:53 +03:00
siivonek dda36aef4c [isp] Fix errors in reference building. Use cubic filter during prediction if ISP enabled. 2022-09-12 14:41:44 +03:00
Joose Sainio 1bd3a4cb55 [isp] Use correct coordinates for depth 4 chroma tu cost calculation 2022-09-06 09:30:46 +03:00
Joose Sainio 68263f428b [tr-skip] fix transform skip flag writing 2022-09-06 08:49:22 +03:00
siivonek a7b26f329e [isp] WIP 2022-09-05 18:14:59 +03:00
siivonek 790deb2504 [isp] Fix mistake in pu_loc argument passing, was not used after passing. 2022-09-05 16:57:07 +03:00
Joose Sainio 60ccdbf853 [lfnst] Fix lfnst constraint checking for the new coeff order 2022-09-05 16:51:23 +03:00
Joose Sainio 17abc7a050 [lfnst] lfnst is not allowed for transform split 2022-09-05 12:04:21 +03:00
Joose Sainio bbae4e2b07 [jccr] Fix jccr coefficient copying 2022-09-05 11:20:09 +03:00
Joose Sainio 95b6ddbe67 [cabac] fix cbf_y context for tr splits 2022-09-05 11:10:55 +03:00