Commit graph

4377 commits

Author SHA1 Message Date
Joose Sainio 85bfeea602 [jccr] Move chroma transform search related functions to transform.c 2022-06-28 15:28:53 +03:00
Joose Sainio 489876b9c7 [tr-skip] Add cmdline option for chroma transform skip 2022-06-28 15:28:47 +03:00
Joose Sainio 3afff7940c [jccr] update tr-skip context for jccr blocks 2022-06-28 15:28:40 +03:00
Joose Sainio c7a7a4719a [debug] remove debug prints 2022-06-28 15:28:29 +03:00
Joose Sainio 0630a98f6c [tests] temporaroly disable satd test 2022-06-28 15:28:12 +03:00
Joose Sainio 835b7fa736 [test] Add capability for testing the consistency of the cabac state 2022-06-28 15:28:06 +03:00
Joose Sainio 02931e8c24 [intra] Only use the old search if tr_depth != depth 2022-06-28 15:27:56 +03:00
Joose Sainio fbd5911aa9 [tr-skip] Update tr_skip context for chroma & count bits 2022-06-28 15:27:30 +03:00
Joose Sainio f056178e80 [jccr] Chroma transform search kinda working 2022-06-28 15:27:21 +03:00
Joose Sainio 27b730c2e9 [jccr] WIP chroma transform search 2022-06-28 15:26:47 +03:00
Joose Sainio c309b81532 [intra] Fix chroma bit counting 2022-06-28 15:26:41 +03:00
Joose Sainio aa2f412c20 [rd] Don't calculate cost for non-basic transforms that do not have coeffs 2022-06-28 15:26:36 +03:00
Joose Sainio 079d1e9aaf [debug] Disable this for now 2022-06-28 15:26:25 +03:00
Joose Sainio 03246c031e [cabac] Fix trskip writing 2022-06-28 15:26:19 +03:00
Joose Sainio c11e30fdb4 [mts] get mts constraints for bit counting 2022-06-28 15:26:11 +03:00
Joose Sainio 81df3ef2e0 [mts] Reset mts constraints 2022-06-28 15:25:21 +03:00
Joose Sainio 675eb5b508 [debug] remove debug prints 2022-06-28 15:25:15 +03:00
Joose Sainio 2f5fd05f88 [intra] fix warnings 2022-06-28 15:25:09 +03:00
Joose Sainio 418c65cbf1 [intra] count mts and tr_skip bits 2022-06-28 15:25:01 +03:00
Joose Sainio 804bf3afcb [intra] Limit rough search 2022-06-28 15:24:39 +03:00
Joose Sainio 4c66561ebe [debug] Temporarily move split flag counting 2022-06-28 15:24:32 +03:00
Joose Sainio 8c939c1405 [intra] Fix chroma cbf counting for 4x4 2022-06-28 15:24:25 +03:00
Joose Sainio 2feb3daaaf [intra] Count chroma mode bits 2022-06-28 15:24:18 +03:00
Joose Sainio dfbe9dad87 [intra] intra dist = MIN(satd_dist, sad_dist*2) 2022-06-28 15:24:12 +03:00
Joose Sainio 09832c3c85 [intra] fix various things 2022-06-28 15:24:02 +03:00
Joose Sainio 6fcec1fdd3 [mrl] [mip] initial rd improvement 2022-06-28 15:23:56 +03:00
Joose Sainio 1b3885d5be [intra] Chroma working 2022-06-28 15:23:36 +03:00
Joose Sainio 0575c9c592 [intra] Fix warnings 2022-06-28 15:23:25 +03:00
Joose Sainio 80ab347d93 [intra] Luma working 2022-06-28 15:22:47 +03:00
Joose Sainio 23fda23322 [intra] Count fractional bits with get_coeff_cabac_cost 2022-06-28 15:22:33 +03:00
Joose Sainio 6d080b215c [intra] WIP: improve search 2022-06-28 15:22:20 +03:00
Joose Sainio c16f42e9df [satd] DC scaling 2022-06-28 15:20:56 +03:00
Joose Sainio 480a907c07 [lfnst] change kvz_ to uvg_ 2022-06-28 15:20:44 +03:00
siivonek 7abbf14d50 [lfnst] Fix lfnst constraint calculation check. Constraints will now be calculated regardless of tr_idx value. 2022-06-28 15:20:44 +03:00
siivonek d15fb473a2 [lfnst] Fix error in mts skip check. Skip value was derived from tr_skip instead of correct tr_idx. 2022-06-28 15:20:44 +03:00
siivonek 4d3688cf17 [lfnst] Fix error in luma tr_skip check. All cbf was checked when only luma cbf needed to be checked. 2022-06-28 15:20:43 +03:00
siivonek 9171e03a53 [lfnst] Add cu intra check to lfnst calls. 2022-06-28 15:20:43 +03:00
siivonek 4fee153318 [lfnst] Fix assert. No need for >= 0 check for unsigned integer. 2022-06-28 15:20:43 +03:00
siivonek b836c6f529 [lfnst] Fix error in is_lfnst_allowed. Error caused cabac to fail on depth 4 chroma blocks. 2022-06-28 15:20:43 +03:00
siivonek cf4343989f [lfnst] Add early termination to mts/lfnst search if MTS_SKIP is selected. Resolve TODOs. 2022-06-28 15:20:42 +03:00
siivonek e056e87ca0 [lfnst] Fix help text missing line break. Fix luma transform skip check. 2022-06-28 15:20:42 +03:00
siivonek 7551952eeb [lfnst] Fix separate tree type checks. Fix lfnst search. Lfnst is now done in outer loop. 2022-06-28 15:20:42 +03:00
siivonek 896bc093d3 [lfnst] Remove unused stuff from lfnst constraint check. Use temp values for storing lfnst constraints during search instead of updating values inside pred cu. 2022-06-28 15:20:41 +03:00
siivonek 7b6b6586c6 [lfnst] Implement lfnst constraint check during search. 2022-06-28 15:20:41 +03:00
siivonek 64c5cfb127 [lfnst] Fix error in scan table selection. 2022-06-28 15:20:41 +03:00
siivonek 1c5f7f9dfc [lfnst] Fix errors in quantization code. Lfnst was called in wrong place. Fix error in lfnst algorithm, 16 bit integer overflow issue. 2022-06-28 15:20:41 +03:00
siivonek 2975b1fef0 [lfnst] WIP Implement lfnst search. Resolve some TODOs. 2022-06-28 15:20:40 +03:00
siivonek de18eef956 [lfnst] Chroma transform data is no longer stored in top left block. Remove -4x-4 coordinate offset. 2022-06-28 15:20:40 +03:00
siivonek 76947c4c41 [lfnst] Fix rebase errors. 2022-06-28 15:20:40 +03:00
siivonek b9c71b7f01 [lfnst] Fix cabac write issues. Lfnst constraints needed to be reset after encode_transform_coeff call. 2022-06-28 15:20:39 +03:00