hashirama/uvg266
1
0
Fork 0
mirror of https://github.com/ultravideo/uvg266.git synced 2024-11-27 19:24:06 +00:00
Code Issues Actions Projects Releases Wiki Activity
4584 commits 3 branches 6 tags 9.2 MiB
Commit graph

3936 commits

Author SHA1 Message Date
siivonek 510798cb3d [isp] Fix mistake in isp cabac write. Intra luma mpm flag bit was checking isp when it did not need to. 2023-08-14 12:16:40 +03:00
siivonek f86dc29ce7 [isp] Fix mistake in cost calculation. Remove some commented out code blocks. 2023-08-14 12:16:39 +03:00
siivonek bbb8faea98 [isp] Modify encode transform coeff func to handle non-square blocks, use cu_loc_t where possible. Fix mistake in mts idct generic. 2023-08-14 12:16:38 +03:00
siivonek 7062697beb [isp] Resolve TODOs. Make scan order tables const. 2023-08-14 12:16:37 +03:00
siivonek 93317cafa4 [isp] Write isp config bit to sps. 2023-08-14 12:16:37 +03:00
siivonek 182d0f4e66 [isp] Remove old_scan tables and related asserts. Fix coefficient group indexing. 2023-08-14 12:16:36 +03:00
siivonek f8641f7436 [isp] Fix assert. Implement coef cost calculation for isp splits. 2023-08-14 12:16:35 +03:00
siivonek ae0336fdfc [isp] Add non-square block handling to functions. 2023-08-14 12:16:34 +03:00
siivonek 031a758d6c [isp] Count isp cbfs. 2023-08-14 12:16:01 +03:00
siivonek 75175ee2e2 [isp] Fix isp search. 2023-08-14 12:16:00 +03:00
siivonek 8d914ce849 [isp] Implement coefficient encoding for isp splits. Make get_split_dim non static, it was needed elsewhere after all. 2023-08-14 12:15:59 +03:00
siivonek 573ecf80e3 [isp] Move can_use_lfnst_with_isp to intra.c. Remove duplicate functions. Move isp related functions from search to intra. Make isp_split_dim static. Move isp related defines from search to intra. 2023-08-14 12:15:58 +03:00
siivonek bcbd952dfd [isp] Add height handling to avx2 reconstruction. 2023-08-14 12:15:57 +03:00
siivonek 7c340fd92b [isp] Add height to inverse transform skip. 2023-08-14 12:15:56 +03:00
siivonek 318d925028 [isp] Add new convert_to_log2 table. Change all instances which used old convert_to_bit table to change dimensions into log2. 2023-08-14 12:15:55 +03:00
siivonek 6922157ed3 [isp] Fix quantization function calls. Some were not getting height as input. 2023-08-14 12:15:54 +03:00
siivonek 50ad91a94e [isp] Modify quantization functions to work with non-square blocks. 2023-08-14 12:15:53 +03:00
siivonek 31c8f1356f [isp] Add height to sig coeff group context calculation function. 2023-08-14 12:15:53 +03:00
siivonek 936256e750 [isp] Fix sig coeff flag context calculation function call. Width & height was swapped. 2023-08-14 12:15:52 +03:00
siivonek 9e7f4eac99 [isp] Change variable name 'type' to 'color'. 2023-08-14 12:15:51 +03:00
siivonek 09b905c6c4 [isp] Add height to get_tr_type function. 2023-08-14 12:15:51 +03:00
siivonek 8b7d573ae7 [isp] Add height to idct getter function. Check block dimensions in transform 2d functions. 2023-08-14 12:15:50 +03:00
siivonek 370bd07c55 [isp] Fix error in mts dct and idct. 2023-08-14 12:15:49 +03:00
siivonek 3a874ab5dd [isp] Comment out dct non square function. It is not needed since mts dct function will handle transform for non square blocks. 2023-08-14 12:15:49 +03:00
siivonek f9116441da [isp] Fix avx2 function call. Missing height parameter. 2023-08-14 12:15:48 +03:00
siivonek 6f756e831d [isp] Uncomment old scan order code to test against new one. Add assert to ensure old and new tables are the same. 2023-08-14 12:15:47 +03:00
siivonek c4d1f80f8f [isp] Fix error in scan order getter. Change define names to better reflect what they do. Add more accurate bookmark comments to scan order buffer table. 2023-08-14 12:15:46 +03:00
siivonek 8131e970e5 [isp] Modify existing scan table calls to use new getter. Add safety assert to getter. 2023-08-14 12:15:46 +03:00
siivonek 6ff9ae074e [isp] Add scan order getter. Add bookmark comments to scan order buffer. 2023-08-14 12:15:45 +03:00
siivonek 55d77c6b50 [isp] Add scan order tables for all possible block sizes. 2023-08-14 12:15:45 +03:00
siivonek 35271648db [isp] Fix some errors. Pass height to functions. Some WIP comments. 2023-08-14 12:15:44 +03:00
siivonek a9090c99b5 [isp] Fix error in inverse transform shifting. 2023-08-14 12:15:43 +03:00
siivonek cd7e091992 [isp] Fix mistake in transform if clause. 2023-08-14 12:15:43 +03:00
siivonek 6a3ddfd0bc [isp] Modify inverse transform to handle non-square blocks. 2023-08-14 12:15:42 +03:00
siivonek 626c9b02ea [isp] Modify transform and quantization functions to handle non-square blocks. Add strategy headers to CMakelist. 2023-08-14 12:15:41 +03:00
siivonek 06532dce02 [isp] Implement ISP search and partitioning. Add helper function for constructing cu_loc types. WIP stuff for transform. 2023-08-14 12:15:40 +03:00
siivonek 6236cc29be [isp] Fix avx2 function call. 2023-08-14 12:15:39 +03:00
siivonek ec4909095c [isp] Do not filter references if ISP is used. 2023-08-14 12:15:38 +03:00
siivonek 96df3ffd64 [isp] Change function calls to cu_loc_t. 2023-08-14 12:15:38 +03:00
siivonek 9406c5c31d [isp] Modify generic intra pred functions to handle non-square blocks. 2023-08-14 12:15:37 +03:00
siivonek 03b22e561e [isp] Add ISP command line option. 2023-08-14 12:15:36 +03:00
Marko Viitanen 1a1fea1a19 Merge branch 'implement_ibc' 2023-08-09 09:34:29 +03:00
Marko Viitanen 18b4a8be79 [ibc] Include the chroma in crc 2023-07-27 10:58:20 +03:00
Marko Viitanen 20875a9819 [ibc] Calculate hashes every 4 pixels and change the IBC costs a bit 2023-07-27 10:29:55 +03:00
Marko Viitanen 0fefd3f621 [ibc] Add hash based starting point finder for reqular inter search *experimental* 2023-07-24 22:55:52 +03:00
Marko Viitanen 6f4d538f4f [ibc] Clean up the ibc search, utilize hash based starting points if ibc=2 2023-07-24 22:07:22 +03:00
Marko Viitanen 6fe629e666 [ibc] A bit of cleanup and skip IBC search if cost is already less than 500 2023-07-21 20:40:00 +03:00
Marko Viitanen 3cef3c0119 Change the hardcoded general_level_idc from 5.2 to 6.3 2023-07-21 20:19:43 +03:00
Marko Viitanen 95dc4aa0cb [ibc] Fix the IBC buffer limitation, 256x64 pixels allowed 2023-07-21 20:15:24 +03:00
Marko Viitanen 8ff184a6b3 [ibc] Fill the IBC hashmap at the start of LCU search and use reverse map for "pos to hash" 2023-07-21 20:14:23 +03:00
Marko Viitanen 457d650f49 [ibc] Fix for CRC calculations 
- Input for the 64bit crc intrinsic was 32bit
 2023-07-19 09:57:24 +03:00
Marko Viitanen 15fb6f8183 [ibc] Add first version of the IBC hash search 2023-06-29 21:57:06 +03:00
Marko Viitanen 8cec02280f [ibc] Use IBC hashmap in LCU row basis 2023-06-28 23:06:04 +03:00
Marko Viitanen 76d66591c5 [ibc] Implement CRC for 8x8 block and generate a full hashmap at the frame load 2023-06-26 21:24:10 +03:00
Marko Viitanen 4b1f5ca7e2 [ibc] Add the hashmap to the frame and fix some small issues with hashmap and crc32c 
- crc32c_4x4 strategy was not working, made some changes to the initialization
 2023-06-22 21:44:49 +03:00
Marko Viitanen 30321e6dd4 [ibc] Fix uvg_hashmap_hash definition 2023-06-22 14:45:05 +03:00
Marko Viitanen a32a318d18 [ibc] Add CRC32C functions, with SSE 4.2 optimized CRC calculations 2023-06-22 14:39:56 +03:00
Marko Viitanen 7252befc17 [ibc] Add a hashmap implementation for IBC hash search 2023-06-22 14:39:56 +03:00
Marko Viitanen 68382f9e25 [ibc] Handle 4x4 block cases 2023-06-22 14:39:55 +03:00
Marko Viitanen 31fbf453c1 [ibc] Fix IBCFlag writing with I-frames and clean up some code 2023-06-22 14:39:55 +03:00
Marko Viitanen 8aded6406b [ibc] Fix issue in search 2023-05-08 12:26:06 +03:00
Marko Viitanen 34c7c432f9 [ibc] Fix deblocking for the IBC blocks 2023-05-08 11:58:40 +03:00
Joose Sainio e636db489f [cfg] Parameterize intra rough search granularity 2022-09-28 08:49:38 +03:00
Joose Sainio ed6a0528fe Further make things faster 2022-08-30 15:17:05 +03:00
Joose Sainio b0b2b0e536 Try making ultrafast all intra a bit faster 2022-08-29 14:11:08 +03:00
Marko Viitanen 6de2e2d581 [ibc] Fix some git merge issues and IBC merge candidate selection 2022-08-03 10:46:02 +03:00
Marko Viitanen 6a0e2a062d [ibc] Implement a proper search for IBC based on Inter search 2022-07-29 11:53:11 +03:00
Marko Viitanen 65c017c2f2 [ibc] Add check for above block in IBC search 2022-07-29 11:53:11 +03:00
Marko Viitanen 09e62a68fe [ibc] Fix merge candidate selection bug and IBC HMVP size reset at the start of the lcu row 2022-07-29 11:53:10 +03:00
Marko Viitanen d288cc46e9 [ibc] Fix coding of IBC in P and B slices, enable in search 2022-07-29 11:53:10 +03:00
Marko Viitanen 48584eead9 [ibc] Reset the jccr flags to fix a bug with IBC 2022-07-29 11:53:10 +03:00
Marko Viitanen 7ce01b4826 [ibc] Tune search costs a bit and revert debug vector scaling 2022-07-29 11:53:09 +03:00
Marko Viitanen 0fdf96fab2 [ibc] Change internal MV storage to INTERNAL_MV_PREC and code it as full-pel 2022-07-29 11:53:09 +03:00
Marko Viitanen cc4c757695 [ibc] Fix bugs on IBC reconstruction and add a simple search for I-frames 2022-07-29 11:53:08 +03:00
Marko Viitanen d9164f3cfe [ibc] Simplify the IBC merge candidate and mv cand selection 2022-07-29 11:53:08 +03:00
Marko Viitanen a46a4531a3 [ibc] Add HMVP for IBC and correct AMVP selection 2022-07-29 11:52:17 +03:00
Marko Viitanen dbc2006ba9 [ibc] Implement IBC reconstruction function when blocks are completely in the ibc buffer 2022-07-29 11:52:16 +03:00
Marko Viitanen b49d32af21 [ibc] Add IBC buffers 2022-07-29 11:52:15 +03:00
Marko Viitanen 6ec4c37b47 [ibc] Add IBC Flag context and code the bits, disable by default for now 2022-07-29 11:49:49 +03:00
Marko Viitanen 20d0a9b65e [ibc] Add --ibc parameter and config values for Intra Block Copy 2022-07-29 11:49:49 +03:00
Marko Viitanen cd2d4066d5 Fix scaled MV clipping and remove some unused variables 2022-07-28 13:59:11 +03:00
Marko Viitanen 3dd738ebb5 Fix mv_t rounding problems in some functions 2022-07-27 13:02:37 +03:00
Marko Viitanen 5ce1035291 [debug] Fix Motion Vector debug code not to overflow on videos not divisible by LCU_WIDTH 2022-07-27 12:48:39 +03:00
Marko Viitanen b7b7c22e44 Change mv_t to int32_t because of possible overflow in large videos 2022-07-27 12:48:39 +03:00
Joose Sainio ea32ef33ac [lfnst] handle transform skip correctly during search 2022-07-08 10:57:26 +03:00
Joose Sainio 03fb6ce92e [lfnst] Fix lfnst+tr_skip for dual tree 2022-07-08 10:57:26 +03:00
Joose Sainio 450cd00290 [mts] Fix cost calculation 2022-07-08 10:56:35 +03:00
Joose Sainio f9212b4e44 [mts] Don't do tr-skip when tr-skip is disabled 2022-07-06 15:15:28 +03:00
Joose Sainio dc7c8eeb41 [tr-skip] fix uvg_encode_ts_residual 2022-07-06 10:51:01 +03:00
Joose Sainio 427d611a00 [intra] Perform chroma search for rd2 2022-07-05 12:29:27 +03:00
Joose Sainio e2c34e7c25 [lfnst] Fix lfnst for --rd 2 2022-07-05 12:19:21 +03:00
Joose Sainio 02aa36f1a2 [tests] Fix final issue with avx2 satd and update test results 2022-07-05 10:28:59 +03:00
Joose Sainio 42adfb52a7 [satd] Satd scaling on avx2 implementations and re-enable satd tests 2022-07-05 09:34:59 +03:00
Joose Sainio 1f6a62e70e [fix-up] Force lfnst off when trying the mode from below depth block 2022-07-04 13:45:16 +03:00
Joose Sainio 3de4b99aec [jccr] Fix cost calculation 2022-07-04 13:41:14 +03:00
Joose Sainio 3a6414c31d [dual-tree] Fix deblock 2022-06-30 14:21:03 +03:00
Joose Sainio 5fefea025f [lfnst] get constarints for jccr mode 2022-06-29 16:35:55 +03:00
Joose Sainio b35a75b2eb [lfnst] Fix lfnst with rdoq 2022-06-29 16:25:25 +03:00
Joose Sainio 6ef532775b [intra] Fix various issues with cclm, mip, dual-tree, and lfnst 2022-06-29 15:09:34 +03:00
Joose Sainio 06d277bc78 [doc] update manpage and readme 2022-06-28 16:25:25 +03:00
Joose Sainio 68243e284f [cleanup] fix warnings 2022-06-28 16:02:22 +03:00
Joose Sainio b4ab9debf1 [lfnst] fix lfnst with cclm 2022-06-28 15:32:34 +03:00
Joose Sainio e25ea52f6f [lfnst] Fix mistakes 2022-06-28 15:32:33 +03:00
Joose Sainio 2fbbae834b [cclm] fix cclm for 4x4 2022-06-28 15:32:33 +03:00
Joose Sainio b8b603feb7 [lfnst] fix compile 2022-06-28 15:32:33 +03:00
Joose Sainio a0dd412811 [cclm] fix cclm bound calculation 2022-06-28 15:32:32 +03:00
Joose Sainio 75e500da10 [lfnst] LFNST working with dual tree 2022-06-28 15:32:32 +03:00
Joose Sainio faba18fe17 [dual-tree] only perform lfnst search when lfnst is enabled 2022-06-28 15:32:32 +03:00
Joose Sainio d16d6e3dd8 [dual-tree] [lfnst] allow counting lfnst bits for chroma in dual-tree 2022-06-28 15:32:31 +03:00
Joose Sainio 37590add20 [lfnst] [dual-tree] LFNST should work with dual tree 2022-06-28 15:32:31 +03:00
Joose Sainio 6c7dc9004c [dual-tree] Fix split context state updating 2022-06-28 15:32:31 +03:00
Joose Sainio b0d616b03c [dual-tree][tests] Fix some issues and enable cabac state test to test for dual tree 2022-06-28 15:32:30 +03:00
Joose Sainio 345c50ecee [dual-tree] rename kvz_ to uvg_ 2022-06-28 15:32:30 +03:00
Joose Sainio 3f12ee58b0 [dual-tree] fix --pu-depth-intra 4-4 for dual tree 2022-06-28 15:32:30 +03:00
Joose Sainio b8215baa30 [dual-tree] Fix CCLM+dual tree 2022-06-28 15:32:30 +03:00
Joose Sainio cf144e2724 [dual-tree] Works for all depths with basic tools 2022-06-28 15:32:29 +03:00
Joose Sainio 1c313e9c19 [dual-tree] works for depths 1 and 2 2022-06-28 15:32:29 +03:00
Joose Sainio 15cb06ded1 [dual-tree] Fix at least for implicit splits 2022-06-28 15:32:29 +03:00
Joose Sainio ed8496e57e [dual-tree] Matches except for cutoff bottom CTUs for forced depth=1 2022-06-28 15:32:28 +03:00
Joose Sainio 2017cb122a [dual-tree] Actually does whole frame 2022-06-28 15:32:28 +03:00
Joose Sainio abd00d04a1 [dual-tree] Still not working but bitstream valid 2022-06-28 15:32:28 +03:00
Joose Sainio 0adb0846d2 [dual-tree] Bitstream valid, hash missmatches 2022-06-28 15:32:28 +03:00
Joose Sainio be2ef18fea [dual-tree] Not working 2022-06-28 15:32:27 +03:00
Joose Sainio 8fba042e02 [dual-tree]preliminary preparation for dual tree 2022-06-28 15:32:27 +03:00
Joose Sainio 74c931a7c7 [lfnst] cost on chroma when necessary and fixes 2022-06-28 15:32:27 +03:00
Joose Sainio 20010cf759 [lfnst] Fix hash mismatches for depth 4 chroma 2022-06-28 15:32:10 +03:00
Joose Sainio ed602d1c07 [lfnst] Cabac state matches for all sizes but hash mismatches 2022-06-28 15:31:55 +03:00
Joose Sainio 7a7bf045e6 [lfnst] 16x16 2022-06-28 15:31:42 +03:00
Joose Sainio d7f7a2d99b [lfnst] working for 32x32 2022-06-28 15:31:42 +03:00
Joose Sainio b75ce57fce [intra] Fix chroma search for rd=2 2022-06-28 15:30:56 +03:00
Joose Sainio 6413854f3d [intra] fix intra recon 2022-06-28 15:30:47 +03:00
Joose Sainio a6d79407ab [lfnst] various small fixes 2022-06-28 15:30:28 +03:00
Joose Sainio cfc3fa9f09 [lfnst] Include lfnst in chroma search 2022-06-28 15:29:56 +03:00
Joose Sainio 36137e2bee [lfnst] WIP 2022-06-28 15:29:50 +03:00
Joose Sainio 55a0e35892 [lfnst] Count lfsnt bits 2022-06-28 15:29:41 +03:00
Joose Sainio 66fef4934e Fix another rebase mistake 2022-06-28 15:29:34 +03:00
Joose Sainio 3da4a313ce Fix rebase mistakes 2022-06-28 15:29:25 +03:00
Joose Sainio 882b00068b [intra] Perform intra chroma search before the mode is selected 2022-06-28 15:29:20 +03:00
Joose Sainio 4dba21ea54 [jccr] Remove select_jccr_mode function 2022-06-28 15:29:11 +03:00
Joose Sainio 658d639f8b [jccr] Chroma tr_search for inter 2022-06-28 15:29:05 +03:00
Joose Sainio 85bfeea602 [jccr] Move chroma transform search related functions to transform.c 2022-06-28 15:28:53 +03:00
Joose Sainio 489876b9c7 [tr-skip] Add cmdline option for chroma transform skip 2022-06-28 15:28:47 +03:00
Joose Sainio 3afff7940c [jccr] update tr-skip context for jccr blocks 2022-06-28 15:28:40 +03:00
Joose Sainio c7a7a4719a [debug] remove debug prints 2022-06-28 15:28:29 +03:00
Joose Sainio 835b7fa736 [test] Add capability for testing the consistency of the cabac state 2022-06-28 15:28:06 +03:00
Joose Sainio 02931e8c24 [intra] Only use the old search if tr_depth != depth 2022-06-28 15:27:56 +03:00
Joose Sainio fbd5911aa9 [tr-skip] Update tr_skip context for chroma & count bits 2022-06-28 15:27:30 +03:00
Joose Sainio f056178e80 [jccr] Chroma transform search kinda working 2022-06-28 15:27:21 +03:00
Joose Sainio 27b730c2e9 [jccr] WIP chroma transform search 2022-06-28 15:26:47 +03:00
Joose Sainio c309b81532 [intra] Fix chroma bit counting 2022-06-28 15:26:41 +03:00
Joose Sainio aa2f412c20 [rd] Don't calculate cost for non-basic transforms that do not have coeffs 2022-06-28 15:26:36 +03:00
Joose Sainio 079d1e9aaf [debug] Disable this for now 2022-06-28 15:26:25 +03:00
Joose Sainio 03246c031e [cabac] Fix trskip writing 2022-06-28 15:26:19 +03:00
Joose Sainio c11e30fdb4 [mts] get mts constraints for bit counting 2022-06-28 15:26:11 +03:00
Joose Sainio 81df3ef2e0 [mts] Reset mts constraints 2022-06-28 15:25:21 +03:00
Joose Sainio 675eb5b508 [debug] remove debug prints 2022-06-28 15:25:15 +03:00
Joose Sainio 2f5fd05f88 [intra] fix warnings 2022-06-28 15:25:09 +03:00
Joose Sainio 418c65cbf1 [intra] count mts and tr_skip bits 2022-06-28 15:25:01 +03:00
Joose Sainio 804bf3afcb [intra] Limit rough search 2022-06-28 15:24:39 +03:00
Joose Sainio 4c66561ebe [debug] Temporarily move split flag counting 2022-06-28 15:24:32 +03:00
Joose Sainio 8c939c1405 [intra] Fix chroma cbf counting for 4x4 2022-06-28 15:24:25 +03:00
Joose Sainio 2feb3daaaf [intra] Count chroma mode bits 2022-06-28 15:24:18 +03:00
Joose Sainio dfbe9dad87 [intra] intra dist = MIN(satd_dist, sad_dist*2) 2022-06-28 15:24:12 +03:00
Joose Sainio 09832c3c85 [intra] fix various things 2022-06-28 15:24:02 +03:00
Joose Sainio 6fcec1fdd3 [mrl] [mip] initial rd improvement 2022-06-28 15:23:56 +03:00
Joose Sainio 1b3885d5be [intra] Chroma working 2022-06-28 15:23:36 +03:00
Joose Sainio 0575c9c592 [intra] Fix warnings 2022-06-28 15:23:25 +03:00
Joose Sainio 80ab347d93 [intra] Luma working 2022-06-28 15:22:47 +03:00
Joose Sainio 23fda23322 [intra] Count fractional bits with get_coeff_cabac_cost 2022-06-28 15:22:33 +03:00
Joose Sainio 6d080b215c [intra] WIP: improve search 2022-06-28 15:22:20 +03:00
Joose Sainio c16f42e9df [satd] DC scaling 2022-06-28 15:20:56 +03:00
Joose Sainio 480a907c07 [lfnst] change kvz_ to uvg_ 2022-06-28 15:20:44 +03:00
siivonek 7abbf14d50 [lfnst] Fix lfnst constraint calculation check. Constraints will now be calculated regardless of tr_idx value. 2022-06-28 15:20:44 +03:00
siivonek d15fb473a2 [lfnst] Fix error in mts skip check. Skip value was derived from tr_skip instead of correct tr_idx. 2022-06-28 15:20:44 +03:00
siivonek 4d3688cf17 [lfnst] Fix error in luma tr_skip check. All cbf was checked when only luma cbf needed to be checked. 2022-06-28 15:20:43 +03:00
siivonek 9171e03a53 [lfnst] Add cu intra check to lfnst calls. 2022-06-28 15:20:43 +03:00
siivonek 4fee153318 [lfnst] Fix assert. No need for >= 0 check for unsigned integer. 2022-06-28 15:20:43 +03:00
siivonek b836c6f529 [lfnst] Fix error in is_lfnst_allowed. Error caused cabac to fail on depth 4 chroma blocks. 2022-06-28 15:20:43 +03:00
siivonek cf4343989f [lfnst] Add early termination to mts/lfnst search if MTS_SKIP is selected. Resolve TODOs. 2022-06-28 15:20:42 +03:00
siivonek e056e87ca0 [lfnst] Fix help text missing line break. Fix luma transform skip check. 2022-06-28 15:20:42 +03:00
siivonek 7551952eeb [lfnst] Fix separate tree type checks. Fix lfnst search. Lfnst is now done in outer loop. 2022-06-28 15:20:42 +03:00
siivonek 896bc093d3 [lfnst] Remove unused stuff from lfnst constraint check. Use temp values for storing lfnst constraints during search instead of updating values inside pred cu. 2022-06-28 15:20:41 +03:00
siivonek 7b6b6586c6 [lfnst] Implement lfnst constraint check during search. 2022-06-28 15:20:41 +03:00
siivonek 64c5cfb127 [lfnst] Fix error in scan table selection. 2022-06-28 15:20:41 +03:00
siivonek 1c5f7f9dfc [lfnst] Fix errors in quantization code. Lfnst was called in wrong place. Fix error in lfnst algorithm, 16 bit integer overflow issue. 2022-06-28 15:20:41 +03:00
siivonek 2975b1fef0 [lfnst] WIP Implement lfnst search. Resolve some TODOs. 2022-06-28 15:20:40 +03:00
siivonek de18eef956 [lfnst] Chroma transform data is no longer stored in top left block. Remove -4x-4 coordinate offset. 2022-06-28 15:20:40 +03:00
siivonek 76947c4c41 [lfnst] Fix rebase errors. 2022-06-28 15:20:40 +03:00
siivonek b9c71b7f01 [lfnst] Fix cabac write issues. Lfnst constraints needed to be reset after encode_transform_coeff call. 2022-06-28 15:20:39 +03:00
siivonek ca2890d7e9 [lfnst] Fix error in inv lfnst call in quant generic. Add lfnst calls to quant avx2. Improve lfnst flag sps write. 2022-06-28 15:20:39 +03:00
siivonek 7928149d05 [lfnst] Fix CI errors. Had to remove const modifier from function call. 2022-06-28 15:20:39 +03:00
siivonek c1190e47d6 [lfnst] Fix cabac write issues. Remove is_luma from encode coeff function call, that information is already passed with type. Change parameter name 'type' -> 'color' to better reflect its purpose. 2022-06-28 15:20:38 +03:00
siivonek e3d92a5296 [lfnst] Add separate function to check if lfnst can be used with ISP. 2022-06-28 15:20:38 +03:00
siivonek acfc28d7ce [lfnst] Fix CI errors. Lfnst tables didn't have kvz prefix. 2022-06-28 15:20:37 +03:00
siivonek ee9258b2d9 [lfnst] WIP Fix errors in lfnst cabac write. 2022-06-28 15:20:37 +03:00
siivonek 560be8c5b4 [lfnst] Fix writing sps lfnst flag. 2022-06-28 15:20:37 +03:00
siivonek 835f576eef [lfnst] Fix error in writing lfnst constraints. Added check to test if CU data exists before referencing it. 2022-06-28 15:20:37 +03:00
siivonek e11fd3154d [lfnst] Implement cabac write. 2022-06-28 15:20:36 +03:00
siivonek 929828392a [lfnst] Implement inverse lfnst. 2022-06-28 15:20:36 +03:00
siivonek fb289ce875 [lfnst] Change lfnst context name. It now follows the naming convention of other similar contexts. 2022-06-28 15:20:36 +03:00