4458 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
siivonek	b8e36bbc4a	[isp] Fix storing cbfs for small ISP splits. Fix pdpc filtering. Cannot be used if width or height is less than 4. Fix dct related CI errors.	2023-08-14 12:21:20 +03:00
siivonek	99495c331b	[isp] Fix some asserts to allow log2_dim 1 block sizes. Fix coefficient group scan order for small dimensions.	2023-08-14 12:21:19 +03:00
siivonek	d39fddf0d8	[isp] Implement DCT for small blocks.	2023-08-14 12:21:19 +03:00
siivonek	910501012f	[isp] Fix referene building for depth 2 blocks. Flip horizontal mode dimensions during prediction. Fix reference length during prediction when ISP enabled.	2023-08-14 12:21:18 +03:00
siivonek	7ba557af6b	[isp] Fix luma cbf writing for ISP splits. Do not write luma cbf if first three splits had luma cbf 0.	2023-08-14 12:21:17 +03:00
siivonek	a28e61eff7	[isp] Fix CI errors.	2023-08-14 12:21:17 +03:00
siivonek	4794104ecc	[isp] Fix errors in reference building. Use cubic filter during prediction if ISP enabled.	2023-08-14 12:21:16 +03:00
Joose Sainio	662f31d61d	[isp] Use correct coordinates for depth 4 chroma tu cost calculation	2023-08-14 12:17:36 +03:00
Joose Sainio	08942a5394	[tr-skip] fix transform skip flag writing	2023-08-14 12:17:35 +03:00
siivonek	a261d4c5b3	[isp] WIP	2023-08-14 12:17:34 +03:00
siivonek	6340dfe4ce	[isp] Fix mistake in pu_loc argument passing, was not used after passing.	2023-08-14 12:17:33 +03:00
Joose Sainio	88c33c0489	[lfnst] Fix lfnst constraint checking for the new coeff order	2023-08-14 12:17:33 +03:00
Joose Sainio	e0e96068cc	[lfnst] lfnst is not allowed for transform split	2023-08-14 12:17:32 +03:00
Joose Sainio	cb7f9919e3	[jccr] Fix jccr coefficient copying	2023-08-14 12:17:32 +03:00
Joose Sainio	3e23fd0601	[cabac] fix cbf_y context for tr splits	2023-08-14 12:17:31 +03:00
siivonek	59292d8808	[isp] Add extra logic to reference building to accommodate ISP. Remove some asserts which were invalidated by ISP.	2023-08-14 12:17:31 +03:00
siivonek	33cd44f11b	[isp] Fix chroma coeff writing for ISP.	2023-08-14 12:17:30 +03:00
siivonek	d8d206365c	[isp] Fix jccr coeffs.	2023-08-14 12:17:29 +03:00
siivonek	7398e58431	[isp] Fix coeff cost calculation. Coeff arrays were indexed wrongly.	2023-08-14 12:17:29 +03:00
siivonek	d050efcb87	[isp] Fix error in last sig coeff function call. Height was not used. Fix cbf writing. Fix transform skip flag writing.	2023-08-14 12:17:28 +03:00
siivonek	33ae02aae0	[isp] Fix mistake in isp cbf writing. Loop index was increased twice.	2023-08-14 12:17:28 +03:00
siivonek	4a21039e23	[isp] Fix mistake in function declaration.	2023-08-14 12:17:27 +03:00
siivonek	b8506c757c	[isp] Convert functions to handle new coeff array order. Add function for getting coeff array subset. Fix assert.	2023-08-14 12:17:26 +03:00
siivonek	69dcb04c99	[isp] Use temporary coeff array when quantizing coeffs. After deriving coeffs, copy temp coeffs from linear order to correct arrays with cu order.	2023-08-14 12:17:26 +03:00
siivonek	0ae71feae4	[isp] Fix assert.	2023-08-14 12:17:25 +03:00
siivonek	2e8f008de4	[isp] Redo call hierarchy to include x, y coordinates.	2023-08-14 12:17:24 +03:00
siivonek	10f9b2be26	[isp] Keep lfnst constraint up to date during search.	2023-08-14 12:16:42 +03:00
siivonek	39f30563c5	[isp] Fix chroma width error when writing cu loc. Remove redundant IPS mode checks.	2023-08-14 12:16:42 +03:00
siivonek	b53308f258	[isp] Fix mistake in setting cbfs. Skip stting if ISP is not used.	2023-08-14 12:16:41 +03:00
siivonek	56ebea7358	[isp] Set cbfs for isp splits after search. Add helper function for isp split number.	2023-08-14 12:16:40 +03:00
siivonek	510798cb3d	[isp] Fix mistake in isp cabac write. Intra luma mpm flag bit was checking isp when it did not need to.	2023-08-14 12:16:40 +03:00
siivonek	f86dc29ce7	[isp] Fix mistake in cost calculation. Remove some commented out code blocks.	2023-08-14 12:16:39 +03:00
siivonek	bbb8faea98	[isp] Modify encode transform coeff func to handle non-square blocks, use cu_loc_t where possible. Fix mistake in mts idct generic.	2023-08-14 12:16:38 +03:00
siivonek	7062697beb	[isp] Resolve TODOs. Make scan order tables const.	2023-08-14 12:16:37 +03:00
siivonek	93317cafa4	[isp] Write isp config bit to sps.	2023-08-14 12:16:37 +03:00
siivonek	182d0f4e66	[isp] Remove old_scan tables and related asserts. Fix coefficient group indexing.	2023-08-14 12:16:36 +03:00
siivonek	f8641f7436	[isp] Fix assert. Implement coef cost calculation for isp splits.	2023-08-14 12:16:35 +03:00
siivonek	ae0336fdfc	[isp] Add non-square block handling to functions.	2023-08-14 12:16:34 +03:00
siivonek	031a758d6c	[isp] Count isp cbfs.	2023-08-14 12:16:01 +03:00
siivonek	75175ee2e2	[isp] Fix isp search.	2023-08-14 12:16:00 +03:00
siivonek	8d914ce849	[isp] Implement coefficient encoding for isp splits. Make get_split_dim non static, it was needed elsewhere after all.	2023-08-14 12:15:59 +03:00
siivonek	573ecf80e3	[isp] Move can_use_lfnst_with_isp to intra.c. Remove duplicate functions. Move isp related functions from search to intra. Make isp_split_dim static. Move isp related defines from search to intra.	2023-08-14 12:15:58 +03:00
siivonek	bcbd952dfd	[isp] Add height handling to avx2 reconstruction.	2023-08-14 12:15:57 +03:00
siivonek	7c340fd92b	[isp] Add height to inverse transform skip.	2023-08-14 12:15:56 +03:00
siivonek	318d925028	[isp] Add new convert_to_log2 table. Change all instances which used old convert_to_bit table to change dimensions into log2.	2023-08-14 12:15:55 +03:00
siivonek	6922157ed3	[isp] Fix quantization function calls. Some were not getting height as input.	2023-08-14 12:15:54 +03:00
siivonek	50ad91a94e	[isp] Modify quantization functions to work with non-square blocks.	2023-08-14 12:15:53 +03:00
siivonek	31c8f1356f	[isp] Add height to sig coeff group context calculation function.	2023-08-14 12:15:53 +03:00
siivonek	936256e750	[isp] Fix sig coeff flag context calculation function call. Width & height was swapped.	2023-08-14 12:15:52 +03:00
siivonek	9e7f4eac99	[isp] Change variable name 'type' to 'color'.	2023-08-14 12:15:51 +03:00