Commit graph

218 commits

Author SHA1 Message Date
Joose Sainio 88c33c0489 [lfnst] Fix lfnst constraint checking for the new coeff order 2023-08-14 12:17:33 +03:00
Joose Sainio cb7f9919e3 [jccr] Fix jccr coefficient copying 2023-08-14 12:17:32 +03:00
siivonek d8d206365c [isp] Fix jccr coeffs. 2023-08-14 12:17:29 +03:00
siivonek 7398e58431 [isp] Fix coeff cost calculation. Coeff arrays were indexed wrongly. 2023-08-14 12:17:29 +03:00
siivonek b8506c757c [isp] Convert functions to handle new coeff array order. Add function for getting coeff array subset. Fix assert. 2023-08-14 12:17:26 +03:00
siivonek 69dcb04c99 [isp] Use temporary coeff array when quantizing coeffs. After deriving coeffs, copy temp coeffs from linear order to correct arrays with cu order. 2023-08-14 12:17:26 +03:00
siivonek 2e8f008de4 [isp] Redo call hierarchy to include x, y coordinates. 2023-08-14 12:17:24 +03:00
siivonek 7062697beb [isp] Resolve TODOs. Make scan order tables const. 2023-08-14 12:16:37 +03:00
siivonek ae0336fdfc [isp] Add non-square block handling to functions. 2023-08-14 12:16:34 +03:00
siivonek 031a758d6c [isp] Count isp cbfs. 2023-08-14 12:16:01 +03:00
siivonek 7c340fd92b [isp] Add height to inverse transform skip. 2023-08-14 12:15:56 +03:00
siivonek 318d925028 [isp] Add new convert_to_log2 table. Change all instances which used old convert_to_bit table to change dimensions into log2. 2023-08-14 12:15:55 +03:00
siivonek 8b7d573ae7 [isp] Add height to idct getter function. Check block dimensions in transform 2d functions. 2023-08-14 12:15:50 +03:00
siivonek 35271648db [isp] Fix some errors. Pass height to functions. Some WIP comments. 2023-08-14 12:15:44 +03:00
siivonek 6a3ddfd0bc [isp] Modify inverse transform to handle non-square blocks. 2023-08-14 12:15:42 +03:00
siivonek 626c9b02ea [isp] Modify transform and quantization functions to handle non-square blocks. Add strategy headers to CMakelist. 2023-08-14 12:15:41 +03:00
siivonek 06532dce02 [isp] Implement ISP search and partitioning. Add helper function for constructing cu_loc types. WIP stuff for transform. 2023-08-14 12:15:40 +03:00
Joose Sainio ea32ef33ac [lfnst] handle transform skip correctly during search 2022-07-08 10:57:26 +03:00
Joose Sainio b35a75b2eb [lfnst] Fix lfnst with rdoq 2022-06-29 16:25:25 +03:00
Joose Sainio 6ef532775b [intra] Fix various issues with cclm, mip, dual-tree, and lfnst 2022-06-29 15:09:34 +03:00
Joose Sainio 68243e284f [cleanup] fix warnings 2022-06-28 16:02:22 +03:00
Joose Sainio e25ea52f6f [lfnst] Fix mistakes 2022-06-28 15:32:33 +03:00
Joose Sainio b8b603feb7 [lfnst] fix compile 2022-06-28 15:32:33 +03:00
Joose Sainio 75e500da10 [lfnst] LFNST working with dual tree 2022-06-28 15:32:32 +03:00
Joose Sainio d16d6e3dd8 [dual-tree] [lfnst] allow counting lfnst bits for chroma in dual-tree 2022-06-28 15:32:31 +03:00
Joose Sainio 37590add20 [lfnst] [dual-tree] LFNST should work with dual tree 2022-06-28 15:32:31 +03:00
Joose Sainio 74c931a7c7 [lfnst] cost on chroma when necessary and fixes 2022-06-28 15:32:27 +03:00
Joose Sainio 20010cf759 [lfnst] Fix hash mismatches for depth 4 chroma 2022-06-28 15:32:10 +03:00
Joose Sainio d7f7a2d99b [lfnst] working for 32x32 2022-06-28 15:31:42 +03:00
Joose Sainio a6d79407ab [lfnst] various small fixes 2022-06-28 15:30:28 +03:00
Joose Sainio cfc3fa9f09 [lfnst] Include lfnst in chroma search 2022-06-28 15:29:56 +03:00
Joose Sainio 36137e2bee [lfnst] WIP 2022-06-28 15:29:50 +03:00
Joose Sainio 3da4a313ce Fix rebase mistakes 2022-06-28 15:29:25 +03:00
Joose Sainio 4dba21ea54 [jccr] Remove select_jccr_mode function 2022-06-28 15:29:11 +03:00
Joose Sainio 658d639f8b [jccr] Chroma tr_search for inter 2022-06-28 15:29:05 +03:00
Joose Sainio 85bfeea602 [jccr] Move chroma transform search related functions to transform.c 2022-06-28 15:28:53 +03:00
Joose Sainio f056178e80 [jccr] Chroma transform search kinda working 2022-06-28 15:27:21 +03:00
Joose Sainio 03246c031e [cabac] Fix trskip writing 2022-06-28 15:26:19 +03:00
Joose Sainio c11e30fdb4 [mts] get mts constraints for bit counting 2022-06-28 15:26:11 +03:00
Joose Sainio 480a907c07 [lfnst] change kvz_ to uvg_ 2022-06-28 15:20:44 +03:00
siivonek d15fb473a2 [lfnst] Fix error in mts skip check. Skip value was derived from tr_skip instead of correct tr_idx. 2022-06-28 15:20:44 +03:00
siivonek 4fee153318 [lfnst] Fix assert. No need for >= 0 check for unsigned integer. 2022-06-28 15:20:43 +03:00
siivonek cf4343989f [lfnst] Add early termination to mts/lfnst search if MTS_SKIP is selected. Resolve TODOs. 2022-06-28 15:20:42 +03:00
siivonek 7551952eeb [lfnst] Fix separate tree type checks. Fix lfnst search. Lfnst is now done in outer loop. 2022-06-28 15:20:42 +03:00
siivonek 64c5cfb127 [lfnst] Fix error in scan table selection. 2022-06-28 15:20:41 +03:00
siivonek 1c5f7f9dfc [lfnst] Fix errors in quantization code. Lfnst was called in wrong place. Fix error in lfnst algorithm, 16 bit integer overflow issue. 2022-06-28 15:20:41 +03:00
siivonek 2975b1fef0 [lfnst] WIP Implement lfnst search. Resolve some TODOs. 2022-06-28 15:20:40 +03:00
siivonek acfc28d7ce [lfnst] Fix CI errors. Lfnst tables didn't have kvz prefix. 2022-06-28 15:20:37 +03:00
siivonek e11fd3154d [lfnst] Implement cabac write. 2022-06-28 15:20:36 +03:00
siivonek 929828392a [lfnst] Implement inverse lfnst. 2022-06-28 15:20:36 +03:00