Commit graph

4401 commits

Author SHA1 Message Date
Joose Sainio 17abc7a050 [lfnst] lfnst is not allowed for transform split 2022-09-05 12:04:21 +03:00
Joose Sainio bbae4e2b07 [jccr] Fix jccr coefficient copying 2022-09-05 11:20:09 +03:00
Joose Sainio 95b6ddbe67 [cabac] fix cbf_y context for tr splits 2022-09-05 11:10:55 +03:00
siivonek 42bea563e4 [isp] Add extra logic to reference building to accommodate ISP. Remove some asserts which were invalidated by ISP. 2022-09-02 16:10:42 +03:00
siivonek b49a4d0a57 [isp] Fix chroma coeff writing for ISP. 2022-08-31 14:54:30 +03:00
siivonek 043e502a4f [isp] Fix jccr coeffs. 2022-08-31 13:57:00 +03:00
siivonek 9557a20c7f [isp] Fix coeff cost calculation. Coeff arrays were indexed wrongly. 2022-08-31 13:53:17 +03:00
siivonek def855a027 [isp] Fix error in last sig coeff function call. Height was not used. Fix cbf writing. Fix transform skip flag writing. 2022-08-31 12:28:57 +03:00
siivonek a7be4496d5 [isp] Fix mistake in isp cbf writing. Loop index was increased twice. 2022-08-30 12:06:43 +03:00
siivonek c9291ec5a2 [isp] Fix mistake in function declaration. 2022-08-29 12:37:23 +03:00
siivonek 3c5479f99b [isp] Convert functions to handle new coeff array order. Add function for getting coeff array subset. Fix assert. 2022-08-26 16:39:57 +03:00
siivonek 125a3106d6 [isp] Use temporary coeff array when quantizing coeffs. After deriving coeffs, copy temp coeffs from linear order to correct arrays with cu order. 2022-08-24 18:45:13 +03:00
siivonek e245c2ac6e [isp] Fix assert. 2022-08-24 17:12:50 +03:00
siivonek efc0f13501 [isp] Redo call hierarchy to include x, y coordinates. 2022-08-24 17:12:36 +03:00
siivonek 7d7e22fbfb [isp] Keep lfnst constraint up to date during search. 2022-08-24 16:00:53 +03:00
siivonek 4ffbbee606 [isp] Fix chroma width error when writing cu loc. Remove redundant IPS mode checks. 2022-08-24 15:40:14 +03:00
siivonek 1c989906e5 [isp] Fix mistake in setting cbfs. Skip stting if ISP is not used. 2022-08-24 15:11:01 +03:00
siivonek bc3bf8a08c [isp] Set cbfs for isp splits after search. Add helper function for isp split number. 2022-08-24 13:05:59 +03:00
siivonek 35886671df [isp] Fix mistake in isp cabac write. Intra luma mpm flag bit was checking isp when it did not need to. 2022-08-23 13:20:57 +03:00
siivonek 1125248046 [isp] Fix mistake in cost calculation. Remove some commented out code blocks. 2022-08-22 14:39:36 +03:00
siivonek 25e0c48633 [isp] Modify encode transform coeff func to handle non-square blocks, use cu_loc_t where possible. Fix mistake in mts idct generic. 2022-08-21 12:46:07 +03:00
siivonek 4c5ca63632 [isp] Resolve TODOs. Make scan order tables const. 2022-08-19 15:05:22 +03:00
siivonek 4ae6b780b3 [isp] Write isp config bit to sps. 2022-08-18 15:55:23 +03:00
siivonek f0ff249035 [isp] Remove old_scan tables and related asserts. Fix coefficient group indexing. 2022-08-18 15:40:26 +03:00
siivonek fae833c4d2 [isp] Fix assert. Implement coef cost calculation for isp splits. 2022-08-18 15:22:17 +03:00
siivonek 943ae415db [isp] Add non-square block handling to functions. 2022-08-18 15:07:22 +03:00
siivonek eb5e24c15d [isp] Count isp cbfs. 2022-08-18 14:03:53 +03:00
siivonek d9b03ab399 [isp] Fix isp search. 2022-08-18 13:33:29 +03:00
siivonek 6a9ca94d25 [isp] Implement coefficient encoding for isp splits. Make get_split_dim non static, it was needed elsewhere after all. 2022-08-17 16:42:22 +03:00
siivonek 27a735ab5a [isp] Move can_use_lfnst_with_isp to intra.c. Remove duplicate functions. Move isp related functions from search to intra. Make isp_split_dim static. Move isp related defines from search to intra. 2022-08-17 15:33:03 +03:00
siivonek 59846c6842 [isp] Add height handling to avx2 reconstruction. 2022-08-16 16:37:12 +03:00
siivonek 3476d65296 [isp] Add height to inverse transform skip. 2022-08-16 15:52:30 +03:00
siivonek 4a57dc8cc7 [isp] Add new convert_to_log2 table. Change all instances which used old convert_to_bit table to change dimensions into log2. 2022-08-16 15:00:15 +03:00
siivonek db0071555d [isp] Fix quantization function calls. Some were not getting height as input. 2022-08-16 14:05:18 +03:00
siivonek 3bc5dcff27 [isp] Modify quantization functions to work with non-square blocks. 2022-08-15 18:42:49 +03:00
siivonek 911ae942d4 [isp] Add height to sig coeff group context calculation function. 2022-08-12 15:28:30 +03:00
siivonek 4b8cd09fa2 [isp] Fix sig coeff flag context calculation function call. Width & height was swapped. 2022-08-12 14:38:33 +03:00
siivonek db8451eff7 [isp] Change variable name 'type' to 'color'. 2022-08-12 13:35:15 +03:00
siivonek 08dc1d7550 [isp] Add height to get_tr_type function. 2022-08-11 14:41:21 +03:00
siivonek 45e76eae63 [isp] Add height to idct getter function. Check block dimensions in transform 2d functions. 2022-08-11 14:30:04 +03:00
siivonek c982ed837a [isp] Fix error in mts dct and idct. 2022-08-11 13:55:04 +03:00
siivonek c9abe944fd [isp] Comment out dct non square function. It is not needed since mts dct function will handle transform for non square blocks. 2022-08-11 13:22:36 +03:00
siivonek c163ff005a [isp] Fix avx2 function call. Missing height parameter. 2022-08-11 13:11:41 +03:00
siivonek 83a216d5e6 [isp] Uncomment old scan order code to test against new one. Add assert to ensure old and new tables are the same. 2022-08-11 12:36:49 +03:00
siivonek ea04f08f52 [isp] Fix error in scan order getter. Change define names to better reflect what they do. Add more accurate bookmark comments to scan order buffer table. 2022-08-10 19:37:09 +03:00
siivonek 1fe45f719e [isp] Modify existing scan table calls to use new getter. Add safety assert to getter. 2022-08-10 19:35:15 +03:00
siivonek 35f1ecbe9d [isp] Add scan order getter. Add bookmark comments to scan order buffer. 2022-08-09 14:02:30 +03:00
siivonek a8b6b8052a [isp] Add scan order tables for all possible block sizes. 2022-08-08 16:36:56 +03:00
siivonek 27016d51c0 [isp] Fix some errors. Pass height to functions. Some WIP comments. 2022-08-08 14:16:57 +03:00
siivonek f5a189e275 [isp] Fix error in inverse transform shifting. 2022-08-05 14:16:32 +03:00