Commit graph

297 commits

Author SHA1 Message Date
Joose Sainio 95b6ddbe67 [cabac] fix cbf_y context for tr splits 2022-09-05 11:10:55 +03:00
siivonek b49a4d0a57 [isp] Fix chroma coeff writing for ISP. 2022-08-31 14:54:30 +03:00
siivonek def855a027 [isp] Fix error in last sig coeff function call. Height was not used. Fix cbf writing. Fix transform skip flag writing. 2022-08-31 12:28:57 +03:00
siivonek 3c5479f99b [isp] Convert functions to handle new coeff array order. Add function for getting coeff array subset. Fix assert. 2022-08-26 16:39:57 +03:00
siivonek efc0f13501 [isp] Redo call hierarchy to include x, y coordinates. 2022-08-24 17:12:36 +03:00
siivonek 4ffbbee606 [isp] Fix chroma width error when writing cu loc. Remove redundant IPS mode checks. 2022-08-24 15:40:14 +03:00
siivonek bc3bf8a08c [isp] Set cbfs for isp splits after search. Add helper function for isp split number. 2022-08-24 13:05:59 +03:00
siivonek 35886671df [isp] Fix mistake in isp cabac write. Intra luma mpm flag bit was checking isp when it did not need to. 2022-08-23 13:20:57 +03:00
siivonek 1125248046 [isp] Fix mistake in cost calculation. Remove some commented out code blocks. 2022-08-22 14:39:36 +03:00
siivonek 25e0c48633 [isp] Modify encode transform coeff func to handle non-square blocks, use cu_loc_t where possible. Fix mistake in mts idct generic. 2022-08-21 12:46:07 +03:00
siivonek 4c5ca63632 [isp] Resolve TODOs. Make scan order tables const. 2022-08-19 15:05:22 +03:00
siivonek f0ff249035 [isp] Remove old_scan tables and related asserts. Fix coefficient group indexing. 2022-08-18 15:40:26 +03:00
siivonek 943ae415db [isp] Add non-square block handling to functions. 2022-08-18 15:07:22 +03:00
siivonek 6a9ca94d25 [isp] Implement coefficient encoding for isp splits. Make get_split_dim non static, it was needed elsewhere after all. 2022-08-17 16:42:22 +03:00
siivonek 27a735ab5a [isp] Move can_use_lfnst_with_isp to intra.c. Remove duplicate functions. Move isp related functions from search to intra. Make isp_split_dim static. Move isp related defines from search to intra. 2022-08-17 15:33:03 +03:00
siivonek 4a57dc8cc7 [isp] Add new convert_to_log2 table. Change all instances which used old convert_to_bit table to change dimensions into log2. 2022-08-16 15:00:15 +03:00
siivonek 83a216d5e6 [isp] Uncomment old scan order code to test against new one. Add assert to ensure old and new tables are the same. 2022-08-11 12:36:49 +03:00
siivonek 1fe45f719e [isp] Modify existing scan table calls to use new getter. Add safety assert to getter. 2022-08-10 19:35:15 +03:00
siivonek 35f1ecbe9d [isp] Add scan order getter. Add bookmark comments to scan order buffer. 2022-08-09 14:02:30 +03:00
Joose Sainio ea32ef33ac [lfnst] handle transform skip correctly during search 2022-07-08 10:57:26 +03:00
Joose Sainio 03fb6ce92e [lfnst] Fix lfnst+tr_skip for dual tree 2022-07-08 10:57:26 +03:00
Joose Sainio f9212b4e44 [mts] Don't do tr-skip when tr-skip is disabled 2022-07-06 15:15:28 +03:00
Joose Sainio dc7c8eeb41 [tr-skip] fix uvg_encode_ts_residual 2022-07-06 10:51:01 +03:00
Joose Sainio b35a75b2eb [lfnst] Fix lfnst with rdoq 2022-06-29 16:25:25 +03:00
Joose Sainio 6ef532775b [intra] Fix various issues with cclm, mip, dual-tree, and lfnst 2022-06-29 15:09:34 +03:00
Joose Sainio 68243e284f [cleanup] fix warnings 2022-06-28 16:02:22 +03:00
Joose Sainio e25ea52f6f [lfnst] Fix mistakes 2022-06-28 15:32:33 +03:00
Joose Sainio 37590add20 [lfnst] [dual-tree] LFNST should work with dual tree 2022-06-28 15:32:31 +03:00
Joose Sainio 6c7dc9004c [dual-tree] Fix split context state updating 2022-06-28 15:32:31 +03:00
Joose Sainio b0d616b03c [dual-tree][tests] Fix some issues and enable cabac state test to test for dual tree 2022-06-28 15:32:30 +03:00
Joose Sainio 345c50ecee [dual-tree] rename kvz_ to uvg_ 2022-06-28 15:32:30 +03:00
Joose Sainio cf144e2724 [dual-tree] Works for all depths with basic tools 2022-06-28 15:32:29 +03:00
Joose Sainio 1c313e9c19 [dual-tree] works for depths 1 and 2 2022-06-28 15:32:29 +03:00
Joose Sainio abd00d04a1 [dual-tree] Still not working but bitstream valid 2022-06-28 15:32:28 +03:00
Joose Sainio 0adb0846d2 [dual-tree] Bitstream valid, hash missmatches 2022-06-28 15:32:28 +03:00
Joose Sainio be2ef18fea [dual-tree] Not working 2022-06-28 15:32:27 +03:00
Joose Sainio 8fba042e02 [dual-tree]preliminary preparation for dual tree 2022-06-28 15:32:27 +03:00
Joose Sainio 20010cf759 [lfnst] Fix hash mismatches for depth 4 chroma 2022-06-28 15:32:10 +03:00
Joose Sainio a6d79407ab [lfnst] various small fixes 2022-06-28 15:30:28 +03:00
Joose Sainio 36137e2bee [lfnst] WIP 2022-06-28 15:29:50 +03:00
Joose Sainio 55a0e35892 [lfnst] Count lfsnt bits 2022-06-28 15:29:41 +03:00
Joose Sainio 66fef4934e Fix another rebase mistake 2022-06-28 15:29:34 +03:00
Joose Sainio 882b00068b [intra] Perform intra chroma search before the mode is selected 2022-06-28 15:29:20 +03:00
Joose Sainio 835b7fa736 [test] Add capability for testing the consistency of the cabac state 2022-06-28 15:28:06 +03:00
Joose Sainio f056178e80 [jccr] Chroma transform search kinda working 2022-06-28 15:27:21 +03:00
Joose Sainio c309b81532 [intra] Fix chroma bit counting 2022-06-28 15:26:41 +03:00
Joose Sainio 03246c031e [cabac] Fix trskip writing 2022-06-28 15:26:19 +03:00
Joose Sainio 675eb5b508 [debug] remove debug prints 2022-06-28 15:25:15 +03:00
Joose Sainio 2f5fd05f88 [intra] fix warnings 2022-06-28 15:25:09 +03:00
Joose Sainio 418c65cbf1 [intra] count mts and tr_skip bits 2022-06-28 15:25:01 +03:00
Joose Sainio 8c939c1405 [intra] Fix chroma cbf counting for 4x4 2022-06-28 15:24:25 +03:00
Joose Sainio 2feb3daaaf [intra] Count chroma mode bits 2022-06-28 15:24:18 +03:00
Joose Sainio 80ab347d93 [intra] Luma working 2022-06-28 15:22:47 +03:00
Joose Sainio 23fda23322 [intra] Count fractional bits with get_coeff_cabac_cost 2022-06-28 15:22:33 +03:00
Joose Sainio 6d080b215c [intra] WIP: improve search 2022-06-28 15:22:20 +03:00
Joose Sainio 480a907c07 [lfnst] change kvz_ to uvg_ 2022-06-28 15:20:44 +03:00
siivonek 4d3688cf17 [lfnst] Fix error in luma tr_skip check. All cbf was checked when only luma cbf needed to be checked. 2022-06-28 15:20:43 +03:00
siivonek b836c6f529 [lfnst] Fix error in is_lfnst_allowed. Error caused cabac to fail on depth 4 chroma blocks. 2022-06-28 15:20:43 +03:00
siivonek cf4343989f [lfnst] Add early termination to mts/lfnst search if MTS_SKIP is selected. Resolve TODOs. 2022-06-28 15:20:42 +03:00
siivonek e056e87ca0 [lfnst] Fix help text missing line break. Fix luma transform skip check. 2022-06-28 15:20:42 +03:00
siivonek 7551952eeb [lfnst] Fix separate tree type checks. Fix lfnst search. Lfnst is now done in outer loop. 2022-06-28 15:20:42 +03:00
siivonek 2975b1fef0 [lfnst] WIP Implement lfnst search. Resolve some TODOs. 2022-06-28 15:20:40 +03:00
siivonek de18eef956 [lfnst] Chroma transform data is no longer stored in top left block. Remove -4x-4 coordinate offset. 2022-06-28 15:20:40 +03:00
siivonek 76947c4c41 [lfnst] Fix rebase errors. 2022-06-28 15:20:40 +03:00
siivonek b9c71b7f01 [lfnst] Fix cabac write issues. Lfnst constraints needed to be reset after encode_transform_coeff call. 2022-06-28 15:20:39 +03:00
siivonek 7928149d05 [lfnst] Fix CI errors. Had to remove const modifier from function call. 2022-06-28 15:20:39 +03:00
siivonek c1190e47d6 [lfnst] Fix cabac write issues. Remove is_luma from encode coeff function call, that information is already passed with type. Change parameter name 'type' -> 'color' to better reflect its purpose. 2022-06-28 15:20:38 +03:00
siivonek e3d92a5296 [lfnst] Add separate function to check if lfnst can be used with ISP. 2022-06-28 15:20:38 +03:00
siivonek ee9258b2d9 [lfnst] WIP Fix errors in lfnst cabac write. 2022-06-28 15:20:37 +03:00
siivonek e11fd3154d [lfnst] Implement cabac write. 2022-06-28 15:20:36 +03:00
Marko Viitanen fb17bbc1ca [cleanup] Fix MSVC warnings about type conversion and signed/unsigned comparison 2022-06-18 13:43:16 +03:00
Joose Sainio 153afc6739 Merge branch 'improve-intra-search' 2022-05-30 13:35:46 +03:00
Joose Sainio d5b6d2771c [deltaqp] Fix delta qp writing for 4x4 for CUs 2022-05-05 14:31:22 +03:00
Marko Viitanen 3a0c5b78a3 [deltaqp] Fix delta qp headers and cabac bits, add vaq and bitrate tests 2022-05-04 11:18:34 +03:00
Marko Viitanen b5dce4c94e Rename all files containing "kvazaar" or "kvz" to "uvg266" and "uvg" 2022-04-28 14:26:05 +03:00
Marko Viitanen f14ea35ec9 Rename kvz_ to uvg_ 2022-04-28 14:18:09 +03:00
Joose Sainio 0381d7d779 Move 4x4 chroma tu info to bottom right cu when intra is split into 4x4 2022-04-21 09:46:54 +03:00
Joose Sainio b413aa5c43 Improve jccr search 2022-04-20 09:31:34 +03:00
Joose Sainio b093248ca7 Maybe working for regular intra search 2022-04-15 11:39:16 +03:00
Joose Sainio debe5924a9 Fix 2022-04-15 11:38:31 +03:00
Joose Sainio 03fd8f1545 Don't write jccr flag for inter CUs when it cannot be enabled 2022-03-31 11:45:49 +03:00
Joose Sainio f5753394a0 Update intra chroma mode contexts 2022-03-31 11:30:27 +03:00
Joose Sainio 0d7800c7f1 Use kvz_write_split_flag for all split flags during search 2022-03-25 13:56:03 +02:00
Joose Sainio e1fcd8b2e4 Add missing bits to kvz_mock_encode_coding_unit 2022-03-25 13:27:34 +02:00
Joose Sainio eb8663fbd1 Move split flag coding to its own function 2022-03-25 13:09:37 +02:00
Joose Sainio b321015426 Use kvz_luma_mode_bits to count luma_mode_bits 2022-03-24 14:57:18 +02:00
Joose Sainio 7aa361696d Facilitate bit counting in encode_intra_luma_coding_unit 2022-03-24 13:34:26 +02:00
Joose Sainio b2a94d4276 Move transform coding and chroma cu coding out of encode_intra_coding_unit 2022-03-24 12:09:14 +02:00
Joose Sainio 0be443d309 WIP: Merge remote-tracking branch 'remotes/kvz_github/master' into update-cabac-during-search 2022-03-21 11:03:46 +02:00
Joose Sainio 352d6750f5 Remove instrumentation code 2022-03-18 08:28:32 +02:00
Joose Sainio d1ba62aea9 Better inter bit_costs for rdo < 2 2022-03-18 08:28:31 +02:00
Joose Sainio 5ed1ffb5d4 WIP: pre Christmas 2022-03-18 08:28:28 +02:00
Joose Sainio 64b2806cc8 Add couple of missing bits to the calculation and get intra neighbours from lcu rather than cu_array 2022-03-18 08:28:27 +02:00
Joose Sainio aea1133e6a Function for mock coding a CU and counting the bits 2022-03-18 08:27:22 +02:00
Joose Sainio d8648fe1de Preparation for accurately counting inter bits 2022-03-18 08:19:57 +02:00
Joose Sainio d2299adb1c Disable bit debug code when VERBOSE is not defined and count bits when combining the intra cus 2022-03-18 08:18:03 +02:00
Joose Sainio d5e4e831f4 Preliminary code for outputting bit costs during the search 2022-03-18 08:17:41 +02:00
siivonek c9cbadad8a Merge branch intra_mip. 2022-03-17 12:04:59 +02:00
Joose Sainio f1a3aa60dd Don't write prediction mode for 4x4 intra cus 2022-03-14 10:34:06 +02:00
siivonek df5cbbe82f [mip] Fix issue with invalid MIP modes written into cabac. Fix Mip mode cost estimation. Implement function to derive mip flag context id. Fix some asserts. Replace floor log 2 implementation with existing kvz math function. 2022-02-09 12:55:17 +02:00