Commit graph

692 commits

Author SHA1 Message Date
siivonek e8779fc175 [isp] Fix storing cbfs for small ISP splits. Fix pdpc filtering. Cannot be used if width or height is less than 4. Fix dct related CI errors. 2022-09-16 15:46:31 +03:00
siivonek c46abbdaa1 [isp] Fix some asserts to allow log2_dim 1 block sizes. Fix coefficient group scan order for small dimensions. 2022-09-16 10:37:51 +03:00
siivonek 19106bfced [isp] Implement DCT for small blocks. 2022-09-15 13:45:45 +03:00
siivonek c894e9c6b9 [isp] Fix referene building for depth 2 blocks. Flip horizontal mode dimensions during prediction. Fix reference length during prediction when ISP enabled. 2022-09-14 15:48:49 +03:00
siivonek 75d4de20e3 [isp] Fix CI errors. 2022-09-12 15:29:53 +03:00
siivonek dda36aef4c [isp] Fix errors in reference building. Use cubic filter during prediction if ISP enabled. 2022-09-12 14:41:44 +03:00
siivonek 42bea563e4 [isp] Add extra logic to reference building to accommodate ISP. Remove some asserts which were invalidated by ISP. 2022-09-02 16:10:42 +03:00
siivonek def855a027 [isp] Fix error in last sig coeff function call. Height was not used. Fix cbf writing. Fix transform skip flag writing. 2022-08-31 12:28:57 +03:00
siivonek 3c5479f99b [isp] Convert functions to handle new coeff array order. Add function for getting coeff array subset. Fix assert. 2022-08-26 16:39:57 +03:00
siivonek efc0f13501 [isp] Redo call hierarchy to include x, y coordinates. 2022-08-24 17:12:36 +03:00
siivonek 25e0c48633 [isp] Modify encode transform coeff func to handle non-square blocks, use cu_loc_t where possible. Fix mistake in mts idct generic. 2022-08-21 12:46:07 +03:00
siivonek 4c5ca63632 [isp] Resolve TODOs. Make scan order tables const. 2022-08-19 15:05:22 +03:00
siivonek f0ff249035 [isp] Remove old_scan tables and related asserts. Fix coefficient group indexing. 2022-08-18 15:40:26 +03:00
siivonek 943ae415db [isp] Add non-square block handling to functions. 2022-08-18 15:07:22 +03:00
siivonek 59846c6842 [isp] Add height handling to avx2 reconstruction. 2022-08-16 16:37:12 +03:00
siivonek 3476d65296 [isp] Add height to inverse transform skip. 2022-08-16 15:52:30 +03:00
siivonek 4a57dc8cc7 [isp] Add new convert_to_log2 table. Change all instances which used old convert_to_bit table to change dimensions into log2. 2022-08-16 15:00:15 +03:00
siivonek db0071555d [isp] Fix quantization function calls. Some were not getting height as input. 2022-08-16 14:05:18 +03:00
siivonek 3bc5dcff27 [isp] Modify quantization functions to work with non-square blocks. 2022-08-15 18:42:49 +03:00
siivonek 911ae942d4 [isp] Add height to sig coeff group context calculation function. 2022-08-12 15:28:30 +03:00
siivonek 08dc1d7550 [isp] Add height to get_tr_type function. 2022-08-11 14:41:21 +03:00
siivonek 45e76eae63 [isp] Add height to idct getter function. Check block dimensions in transform 2d functions. 2022-08-11 14:30:04 +03:00
siivonek c982ed837a [isp] Fix error in mts dct and idct. 2022-08-11 13:55:04 +03:00
siivonek c9abe944fd [isp] Comment out dct non square function. It is not needed since mts dct function will handle transform for non square blocks. 2022-08-11 13:22:36 +03:00
siivonek c163ff005a [isp] Fix avx2 function call. Missing height parameter. 2022-08-11 13:11:41 +03:00
siivonek 83a216d5e6 [isp] Uncomment old scan order code to test against new one. Add assert to ensure old and new tables are the same. 2022-08-11 12:36:49 +03:00
siivonek 1fe45f719e [isp] Modify existing scan table calls to use new getter. Add safety assert to getter. 2022-08-10 19:35:15 +03:00
siivonek 27016d51c0 [isp] Fix some errors. Pass height to functions. Some WIP comments. 2022-08-08 14:16:57 +03:00
siivonek f5a189e275 [isp] Fix error in inverse transform shifting. 2022-08-05 14:16:32 +03:00
siivonek eed1c6d965 [isp] Fix mistake in transform if clause. 2022-08-05 13:48:35 +03:00
siivonek 2b67d23b84 [isp] Modify inverse transform to handle non-square blocks. 2022-08-05 13:40:04 +03:00
siivonek 3346eea6ce [isp] Modify transform and quantization functions to handle non-square blocks. Add strategy headers to CMakelist. 2022-08-04 21:47:46 +03:00
siivonek 468c108a44 [isp] Fix avx2 function call. 2022-07-22 13:10:21 +03:00
siivonek ae4092a5ca [isp] Change function calls to cu_loc_t. 2022-07-21 14:35:12 +03:00
siivonek 7f844c643a [isp] Modify generic intra pred functions to handle non-square blocks. 2022-07-20 16:02:27 +03:00
Joose Sainio 02aa36f1a2 [tests] Fix final issue with avx2 satd and update test results 2022-07-05 10:28:59 +03:00
Joose Sainio 42adfb52a7 [satd] Satd scaling on avx2 implementations and re-enable satd tests 2022-07-05 09:34:59 +03:00
Joose Sainio b35a75b2eb [lfnst] Fix lfnst with rdoq 2022-06-29 16:25:25 +03:00
Joose Sainio 68243e284f [cleanup] fix warnings 2022-06-28 16:02:22 +03:00
Joose Sainio 75e500da10 [lfnst] LFNST working with dual tree 2022-06-28 15:32:32 +03:00
Joose Sainio 74c931a7c7 [lfnst] cost on chroma when necessary and fixes 2022-06-28 15:32:27 +03:00
Joose Sainio 20010cf759 [lfnst] Fix hash mismatches for depth 4 chroma 2022-06-28 15:32:10 +03:00
Joose Sainio d7f7a2d99b [lfnst] working for 32x32 2022-06-28 15:31:42 +03:00
Joose Sainio 36137e2bee [lfnst] WIP 2022-06-28 15:29:50 +03:00
Joose Sainio 3da4a313ce Fix rebase mistakes 2022-06-28 15:29:25 +03:00
Joose Sainio f056178e80 [jccr] Chroma transform search kinda working 2022-06-28 15:27:21 +03:00
Joose Sainio 27b730c2e9 [jccr] WIP chroma transform search 2022-06-28 15:26:47 +03:00
Joose Sainio 09832c3c85 [intra] fix various things 2022-06-28 15:24:02 +03:00
Joose Sainio 23fda23322 [intra] Count fractional bits with get_coeff_cabac_cost 2022-06-28 15:22:33 +03:00
Joose Sainio c16f42e9df [satd] DC scaling 2022-06-28 15:20:56 +03:00