Commit graph

4411 commits

Author SHA1 Message Date
siivonek 19106bfced [isp] Implement DCT for small blocks. 2022-09-15 13:45:45 +03:00
siivonek c894e9c6b9 [isp] Fix referene building for depth 2 blocks. Flip horizontal mode dimensions during prediction. Fix reference length during prediction when ISP enabled. 2022-09-14 15:48:49 +03:00
siivonek 124cdd4e4a [isp] Fix luma cbf writing for ISP splits. Do not write luma cbf if first three splits had luma cbf 0. 2022-09-13 13:27:16 +03:00
siivonek 75d4de20e3 [isp] Fix CI errors. 2022-09-12 15:29:53 +03:00
siivonek dda36aef4c [isp] Fix errors in reference building. Use cubic filter during prediction if ISP enabled. 2022-09-12 14:41:44 +03:00
Joose Sainio 1bd3a4cb55 [isp] Use correct coordinates for depth 4 chroma tu cost calculation 2022-09-06 09:30:46 +03:00
Joose Sainio 68263f428b [tr-skip] fix transform skip flag writing 2022-09-06 08:49:22 +03:00
siivonek a7b26f329e [isp] WIP 2022-09-05 18:14:59 +03:00
siivonek 790deb2504 [isp] Fix mistake in pu_loc argument passing, was not used after passing. 2022-09-05 16:57:07 +03:00
Joose Sainio 60ccdbf853 [lfnst] Fix lfnst constraint checking for the new coeff order 2022-09-05 16:51:23 +03:00
Joose Sainio 17abc7a050 [lfnst] lfnst is not allowed for transform split 2022-09-05 12:04:21 +03:00
Joose Sainio bbae4e2b07 [jccr] Fix jccr coefficient copying 2022-09-05 11:20:09 +03:00
Joose Sainio 95b6ddbe67 [cabac] fix cbf_y context for tr splits 2022-09-05 11:10:55 +03:00
siivonek 42bea563e4 [isp] Add extra logic to reference building to accommodate ISP. Remove some asserts which were invalidated by ISP. 2022-09-02 16:10:42 +03:00
siivonek b49a4d0a57 [isp] Fix chroma coeff writing for ISP. 2022-08-31 14:54:30 +03:00
siivonek 043e502a4f [isp] Fix jccr coeffs. 2022-08-31 13:57:00 +03:00
siivonek 9557a20c7f [isp] Fix coeff cost calculation. Coeff arrays were indexed wrongly. 2022-08-31 13:53:17 +03:00
siivonek def855a027 [isp] Fix error in last sig coeff function call. Height was not used. Fix cbf writing. Fix transform skip flag writing. 2022-08-31 12:28:57 +03:00
siivonek a7be4496d5 [isp] Fix mistake in isp cbf writing. Loop index was increased twice. 2022-08-30 12:06:43 +03:00
siivonek c9291ec5a2 [isp] Fix mistake in function declaration. 2022-08-29 12:37:23 +03:00
siivonek 3c5479f99b [isp] Convert functions to handle new coeff array order. Add function for getting coeff array subset. Fix assert. 2022-08-26 16:39:57 +03:00
siivonek 125a3106d6 [isp] Use temporary coeff array when quantizing coeffs. After deriving coeffs, copy temp coeffs from linear order to correct arrays with cu order. 2022-08-24 18:45:13 +03:00
siivonek e245c2ac6e [isp] Fix assert. 2022-08-24 17:12:50 +03:00
siivonek efc0f13501 [isp] Redo call hierarchy to include x, y coordinates. 2022-08-24 17:12:36 +03:00
siivonek 7d7e22fbfb [isp] Keep lfnst constraint up to date during search. 2022-08-24 16:00:53 +03:00
siivonek 4ffbbee606 [isp] Fix chroma width error when writing cu loc. Remove redundant IPS mode checks. 2022-08-24 15:40:14 +03:00
siivonek 1c989906e5 [isp] Fix mistake in setting cbfs. Skip stting if ISP is not used. 2022-08-24 15:11:01 +03:00
siivonek bc3bf8a08c [isp] Set cbfs for isp splits after search. Add helper function for isp split number. 2022-08-24 13:05:59 +03:00
siivonek 35886671df [isp] Fix mistake in isp cabac write. Intra luma mpm flag bit was checking isp when it did not need to. 2022-08-23 13:20:57 +03:00
siivonek 1125248046 [isp] Fix mistake in cost calculation. Remove some commented out code blocks. 2022-08-22 14:39:36 +03:00
siivonek 25e0c48633 [isp] Modify encode transform coeff func to handle non-square blocks, use cu_loc_t where possible. Fix mistake in mts idct generic. 2022-08-21 12:46:07 +03:00
siivonek 4c5ca63632 [isp] Resolve TODOs. Make scan order tables const. 2022-08-19 15:05:22 +03:00
siivonek 4ae6b780b3 [isp] Write isp config bit to sps. 2022-08-18 15:55:23 +03:00
siivonek f0ff249035 [isp] Remove old_scan tables and related asserts. Fix coefficient group indexing. 2022-08-18 15:40:26 +03:00
siivonek fae833c4d2 [isp] Fix assert. Implement coef cost calculation for isp splits. 2022-08-18 15:22:17 +03:00
siivonek 943ae415db [isp] Add non-square block handling to functions. 2022-08-18 15:07:22 +03:00
siivonek eb5e24c15d [isp] Count isp cbfs. 2022-08-18 14:03:53 +03:00
siivonek d9b03ab399 [isp] Fix isp search. 2022-08-18 13:33:29 +03:00
siivonek 6a9ca94d25 [isp] Implement coefficient encoding for isp splits. Make get_split_dim non static, it was needed elsewhere after all. 2022-08-17 16:42:22 +03:00
siivonek 27a735ab5a [isp] Move can_use_lfnst_with_isp to intra.c. Remove duplicate functions. Move isp related functions from search to intra. Make isp_split_dim static. Move isp related defines from search to intra. 2022-08-17 15:33:03 +03:00
siivonek 59846c6842 [isp] Add height handling to avx2 reconstruction. 2022-08-16 16:37:12 +03:00
siivonek 3476d65296 [isp] Add height to inverse transform skip. 2022-08-16 15:52:30 +03:00
siivonek 4a57dc8cc7 [isp] Add new convert_to_log2 table. Change all instances which used old convert_to_bit table to change dimensions into log2. 2022-08-16 15:00:15 +03:00
siivonek db0071555d [isp] Fix quantization function calls. Some were not getting height as input. 2022-08-16 14:05:18 +03:00
siivonek 3bc5dcff27 [isp] Modify quantization functions to work with non-square blocks. 2022-08-15 18:42:49 +03:00
siivonek 911ae942d4 [isp] Add height to sig coeff group context calculation function. 2022-08-12 15:28:30 +03:00
siivonek 4b8cd09fa2 [isp] Fix sig coeff flag context calculation function call. Width & height was swapped. 2022-08-12 14:38:33 +03:00
siivonek db8451eff7 [isp] Change variable name 'type' to 'color'. 2022-08-12 13:35:15 +03:00
siivonek 08dc1d7550 [isp] Add height to get_tr_type function. 2022-08-11 14:41:21 +03:00
siivonek 45e76eae63 [isp] Add height to idct getter function. Check block dimensions in transform 2d functions. 2022-08-11 14:30:04 +03:00