Commit graph

335 commits

Author SHA1 Message Date
Joose Sainio 0f50caa2d0 [mtt] Fix various small issues and DepQuant for non-square blocks 2023-08-15 13:11:27 +03:00
Joose Sainio d69bdf79f4 [mtt] Fix couple of issues with 64x32 CUs and non square tr skip rdoq 2023-08-15 13:10:13 +03:00
Joose Sainio 7d787c6b22 [ISP] Fix ISP cost calculation and DepQuant with mts 2023-08-15 13:10:13 +03:00
Joose Sainio 6e24b9a7f9 [DepQuant] Fix isp+depquant and trskip + isp 2023-08-15 13:10:12 +03:00
Joose Sainio dc652c75f9 [DepQuant] Isp and chroma 2023-08-15 13:10:10 +03:00
Joose Sainio 4e203108bc [mtt] Fix ref pixel generation for the second half of 32x2 chroma cus 2023-08-15 13:08:11 +03:00
Joose Sainio d296cac7c3 [mtt] fix reference building for 16x1 2023-08-15 13:08:07 +03:00
Joose Sainio b69e9b2958 [mtt] Fix final issues? 2023-08-15 13:07:51 +03:00
Joose Sainio ba0d43d846 [mtt] Fill chroma data for the whole area covered by the local separate tree chroma cu 2023-08-15 13:07:47 +03:00
Joose Sainio 2da1a34ff3 [mtt] Fix isp for MTT 2023-08-15 13:07:45 +03:00
Joose Sainio 6a6bed7f1f [mtt] WIP 2023-08-15 13:07:43 +03:00
Joose Sainio 9e644fafd0 [mtt] search with mtt depth 2 and dual tree works without lfnst 2023-08-15 13:05:37 +03:00
Joose Sainio d5d9afb1e2 [mtt] fix dual tree 2023-08-15 13:05:19 +03:00
Joose Sainio 657254d38a [mtt] search with depth 1 mtt kinda working 2023-08-15 13:04:28 +03:00
Joose Sainio 13aae7d03d [mtt] All individual mtt splits should be working + uvg_get_possible_splits 2023-08-15 13:04:27 +03:00
Joose Sainio 7b117f171f [mtt] WIP 16x16 TT split 2023-08-15 13:04:26 +03:00
Joose Sainio 43a710e104 fix rebase 2023-08-15 13:04:25 +03:00
Joose Sainio d257376ca0 [mtt] Single mtt split works for everything else, except 16x16 with TT 2023-08-15 13:04:24 +03:00
Joose Sainio 26ee443d2f [mtt] 64x32 and 32x64 2023-08-15 13:04:23 +03:00
Joose Sainio ab21c7e1d7 [mtt] Fix sqrt adjustment, cclm calculation on edges of CTU and waip for lfnst 2023-08-15 13:04:22 +03:00
Joose Sainio 5875dc1ef4 [mtt] Fix counting the number of reference pixles and implement WAIP adjustment 2023-08-15 13:04:21 +03:00
Joose Sainio b893a9268c [mtt] WIP 2023-08-15 13:04:20 +03:00
Joose Sainio 5ba8d45981 WIP 2023-08-15 13:04:19 +03:00
Joose Sainio bbbd391b9e [mtt] WIP 2023-08-15 13:03:40 +03:00
Joose Sainio cfc6aebe3c [mtt] Remove depth from cu_info_t 2023-08-15 13:01:45 +03:00
Joose Sainio b14f6f98ec [mtt] Completely remove tr_depth 2023-08-15 13:00:53 +03:00
Joose Sainio 9a29d9ded3 [mtt] remove depth from cbf 2023-08-15 13:00:51 +03:00
Joose Sainio 6a0864839c [mtt] Actually remove the last width dependency to depth 2023-08-15 13:00:06 +03:00
Joose Sainio dcf879e5ed [mtt] remove all rest usages of deriving width and height from depth 2023-08-15 12:59:39 +03:00
siivonek 0ec16967a1 [isp] Fix reference building. When ISP was in use, not enough samples were generated. Uninitialized memory was referenced. Fix some typos. 2023-08-14 12:21:30 +03:00
siivonek b9822398a0 [isp] Fix lfnst constraint checks when ISP is in use. Add some asserts. 2023-08-14 12:21:25 +03:00
siivonek 89db34d4e0 [isp] Use TR_MAX_WIDTH in ISP checks instead of parameter. 2023-08-14 12:21:24 +03:00
siivonek 01c4d1ddb0 [isp] Fix cabac issues. There are always four transform blocks even if there are only two ISP splits. Fix prediction issues. PDPC filter was applied when it should be disabled. Fix reference building issues. Left reference was built incorrectly for blocks with height 2. 2023-08-14 12:21:21 +03:00
siivonek b8e36bbc4a [isp] Fix storing cbfs for small ISP splits. Fix pdpc filtering. Cannot be used if width or height is less than 4. Fix dct related CI errors. 2023-08-14 12:21:20 +03:00
siivonek 99495c331b [isp] Fix some asserts to allow log2_dim 1 block sizes. Fix coefficient group scan order for small dimensions. 2023-08-14 12:21:19 +03:00
siivonek 910501012f [isp] Fix referene building for depth 2 blocks. Flip horizontal mode dimensions during prediction. Fix reference length during prediction when ISP enabled. 2023-08-14 12:21:18 +03:00
siivonek a28e61eff7 [isp] Fix CI errors. 2023-08-14 12:21:17 +03:00
siivonek 4794104ecc [isp] Fix errors in reference building. Use cubic filter during prediction if ISP enabled. 2023-08-14 12:21:16 +03:00
siivonek a261d4c5b3 [isp] WIP 2023-08-14 12:17:34 +03:00
siivonek 6340dfe4ce [isp] Fix mistake in pu_loc argument passing, was not used after passing. 2023-08-14 12:17:33 +03:00
siivonek 59292d8808 [isp] Add extra logic to reference building to accommodate ISP. Remove some asserts which were invalidated by ISP. 2023-08-14 12:17:31 +03:00
siivonek 33ae02aae0 [isp] Fix mistake in isp cbf writing. Loop index was increased twice. 2023-08-14 12:17:28 +03:00
siivonek b8506c757c [isp] Convert functions to handle new coeff array order. Add function for getting coeff array subset. Fix assert. 2023-08-14 12:17:26 +03:00
siivonek 0ae71feae4 [isp] Fix assert. 2023-08-14 12:17:25 +03:00
siivonek 2e8f008de4 [isp] Redo call hierarchy to include x, y coordinates. 2023-08-14 12:17:24 +03:00
siivonek 39f30563c5 [isp] Fix chroma width error when writing cu loc. Remove redundant IPS mode checks. 2023-08-14 12:16:42 +03:00
siivonek 56ebea7358 [isp] Set cbfs for isp splits after search. Add helper function for isp split number. 2023-08-14 12:16:40 +03:00
siivonek bbb8faea98 [isp] Modify encode transform coeff func to handle non-square blocks, use cu_loc_t where possible. Fix mistake in mts idct generic. 2023-08-14 12:16:38 +03:00
siivonek 7062697beb [isp] Resolve TODOs. Make scan order tables const. 2023-08-14 12:16:37 +03:00
siivonek f8641f7436 [isp] Fix assert. Implement coef cost calculation for isp splits. 2023-08-14 12:16:35 +03:00