hashirama/uvg266
1
0
Fork 0
mirror of https://github.com/ultravideo/uvg266.git synced 2024-12-12 08:54:06 +00:00
Code Issues Actions Projects Releases Wiki Activity
4647 commits 3 branches 6 tags 11 MiB
Commit graph

3996 commits

Author SHA1 Message Date
Joose Sainio 8e4b864e6b [deblock] Fix incorrect direction for transform split of tall blocks at the top CTU row also for chroma 2023-08-15 13:08:05 +03:00
Joose Sainio 34aed10ec1 [mtt] fix 2023-08-15 13:08:05 +03:00
Joose Sainio 1333ab55d9 [mtt] Fix ref building for 32x64 cus 2023-08-15 13:08:04 +03:00
Joose Sainio 1493a2616c [mtt] fix getting collocated chroma for edge cus 2023-08-15 13:08:03 +03:00
Joose Sainio ffe17e48d7 [mtt] minor fixes 2023-08-15 13:08:02 +03:00
Joose Sainio 06fa86c340 [isp] Fix coordinates 2023-08-15 13:07:59 +03:00
Joose Sainio 71516b8155 [mtt] Make sure mtt splits cannot reach a situation where search cannot be performed 2023-08-15 13:07:58 +03:00
Joose Sainio 23e6b9f56c [mtt] Check that we are inside the CTU before checking the ctu data 2023-08-15 13:07:58 +03:00
Joose Sainio facbc794bf [mtt] Fix trying to get split data from depth -1 2023-08-15 13:07:57 +03:00
Joose Sainio 567fa7b2bd [deblock] Fix incorrect direction for transform split of tall blocks at the top CTU row 2023-08-15 13:07:56 +03:00
Joose Sainio 9c2574880a [mtt] Fix deblock for --combine-intra 2023-08-15 13:07:55 +03:00
Joose Sainio 90ce1390c0 [mtt] static 2023-08-15 13:07:54 +03:00
Joose Sainio f6ecb15ced [mtt] Fix implicit splits when mtt is not enabled 2023-08-15 13:07:53 +03:00
Joose Sainio 05218bae21 [jccr] jccr=4 hasn't been necessary for a long time 2023-08-15 13:07:52 +03:00
Joose Sainio b69e9b2958 [mtt] Fix final issues? 2023-08-15 13:07:51 +03:00
Joose Sainio 6620ba8d76 [mtt] fix deblock 2023-08-15 13:07:50 +03:00
Joose Sainio 09baddef17 [mtt] Fix lfnst and chroma coeffs and tests 2023-08-15 13:07:49 +03:00
Joose Sainio 992182dafb WIP 2023-08-15 13:07:48 +03:00
Joose Sainio ba0d43d846 [mtt] Fill chroma data for the whole area covered by the local separate tree chroma cu 2023-08-15 13:07:47 +03:00
Joose Sainio 412dd20f09 [mtt] Fix implicit splits for non ctu divisible frames. 2023-08-15 13:07:46 +03:00
Joose Sainio 2da1a34ff3 [mtt] Fix isp for MTT 2023-08-15 13:07:45 +03:00
Joose Sainio b988c60dd1 [mtt] search works completely with everything except RDOQ deblock and ISP 2023-08-15 13:07:44 +03:00
Joose Sainio 6a6bed7f1f [mtt] WIP 2023-08-15 13:07:43 +03:00
Joose Sainio 065eb6fc03 [mtt] fix lfnst 2023-08-15 13:05:38 +03:00
Joose Sainio 9e644fafd0 [mtt] search with mtt depth 2 and dual tree works without lfnst 2023-08-15 13:05:37 +03:00
Joose Sainio fb146cb6ed [mtt] proper split availability checking for split flag 2023-08-15 13:05:35 +03:00
Joose Sainio d5d9afb1e2 [mtt] fix dual tree 2023-08-15 13:05:19 +03:00
Joose Sainio 8fbefc0de3 [mtt] fix cost calculation 2023-08-15 13:04:29 +03:00
Joose Sainio 657254d38a [mtt] search with depth 1 mtt kinda working 2023-08-15 13:04:28 +03:00
Joose Sainio 13aae7d03d [mtt] All individual mtt splits should be working + uvg_get_possible_splits 2023-08-15 13:04:27 +03:00
Joose Sainio 7b117f171f [mtt] WIP 16x16 TT split 2023-08-15 13:04:26 +03:00
Joose Sainio 43a710e104 fix rebase 2023-08-15 13:04:25 +03:00
Joose Sainio d257376ca0 [mtt] Single mtt split works for everything else, except 16x16 with TT 2023-08-15 13:04:24 +03:00
Joose Sainio 26ee443d2f [mtt] 64x32 and 32x64 2023-08-15 13:04:23 +03:00
Joose Sainio ab21c7e1d7 [mtt] Fix sqrt adjustment, cclm calculation on edges of CTU and waip for lfnst 2023-08-15 13:04:22 +03:00
Joose Sainio 5875dc1ef4 [mtt] Fix counting the number of reference pixles and implement WAIP adjustment 2023-08-15 13:04:21 +03:00
Joose Sainio b893a9268c [mtt] WIP 2023-08-15 13:04:20 +03:00
Joose Sainio 5ba8d45981 WIP 2023-08-15 13:04:19 +03:00
Joose Sainio 70cbaae619 [mtt] square root adjustment for quantization 2023-08-15 13:04:18 +03:00
Joose Sainio f19084569d WIP 2023-08-15 13:04:17 +03:00
Joose Sainio bbbd391b9e [mtt] WIP 2023-08-15 13:03:40 +03:00
Joose Sainio 02a5adf768 [mtt] remove work_tree 2023-08-15 13:01:57 +03:00
Joose Sainio f2abdd6424 [mtt] Remove work_tree_copy_down and change work_tree_copy_up not to require the whole work tree as input parameter 2023-08-15 13:01:56 +03:00
Joose Sainio 03b91992a3 [mtt] fix dual tree 2023-08-15 13:01:55 +03:00
Joose Sainio 536c0ff2ef [quant] fix fast coeff cost 2023-08-15 13:01:54 +03:00
Joose Sainio cf5f7658a0 [mtt] fix 2023-08-15 13:01:53 +03:00
Joose Sainio 1668b65f3f [mtt] fix 2023-08-15 13:01:53 +03:00
Joose Sainio e931c096db [mtt] fix 2023-08-15 13:01:52 +03:00
Joose Sainio c590e5ec73 [mtt] also copy top right CU 2023-08-15 13:01:51 +03:00
Joose Sainio a1e7664db3 [mtt] temporarily disable zero coeff rdo 2023-08-15 13:01:50 +03:00
Joose Sainio 239ee88306 [mtt] fix 2023-08-15 13:01:49 +03:00
Joose Sainio 1cf1501542 [mtt] fix 2023-08-15 13:01:48 +03:00
Joose Sainio 924a93b60e [mtt] Only initialize higher depth ctus partially 2023-08-15 13:01:48 +03:00
Joose Sainio 274e71dff6 [transform] Simplify chroma transform search a bit 2023-08-15 13:01:47 +03:00
Joose Sainio 58c6af8c87 [mtt] Add function for easily getting all split cu_locs 2023-08-15 13:01:46 +03:00
Joose Sainio cfc6aebe3c [mtt] Remove depth from cu_info_t 2023-08-15 13:01:45 +03:00
Joose Sainio b14f6f98ec [mtt] Completely remove tr_depth 2023-08-15 13:00:53 +03:00
Joose Sainio 9a29d9ded3 [mtt] remove depth from cbf 2023-08-15 13:00:51 +03:00
Joose Sainio e3dbeda7f7 [mtt] remove dependency to depth from deblock 2023-08-15 13:00:50 +03:00
Joose Sainio 89af7bda8e [mtt] remove unnecessary depth dependency from split flag 2023-08-15 13:00:49 +03:00
Joose Sainio 0b6f666a1b [mtt] remove lfnst dependency to depth 2023-08-15 13:00:08 +03:00
Joose Sainio 790b1fad48 wip 2023-08-15 13:00:07 +03:00
Joose Sainio 6a0864839c [mtt] Actually remove the last width dependency to depth 2023-08-15 13:00:06 +03:00
Joose Sainio dcf879e5ed [mtt] remove all rest usages of deriving width and height from depth 2023-08-15 12:59:39 +03:00
Joose Sainio 26dcadc149 [mtt] change most if not all of search hierarchy to use cu_loc_t 2023-08-15 12:47:11 +03:00
siivonek 0ec16967a1 [isp] Fix reference building. When ISP was in use, not enough samples were generated. Uninitialized memory was referenced. Fix some typos. 2023-08-14 12:21:30 +03:00
siivonek b16c404362 [isp] Remove some obsolete TODOs and old commented out code. 2023-08-14 12:21:30 +03:00
siivonek 95d73116f9 [isp] Fix some CI errors. Some const modifiers were discarded. 2023-08-14 12:21:29 +03:00
siivonek 90e2a17759 [lfnst] Fix LFNST error when MIP enabled. 2023-08-14 12:21:28 +03:00
siivonek 7005d222d5 [isp] Fix lfnst constraint check when ISP is used. Remove some obsolete comments. 2023-08-14 12:21:28 +03:00
siivonek 3c861e4c02 [isp] Fix search. Best LFNST and MTS modes were not selected correctly for ISP modes. 2023-08-14 12:21:27 +03:00
siivonek b4cc321349 [isp] Fix transform selection when MTS & ISP is used. Wrong transform was selected. Change mts parameter name to better reflect its purpose. 2023-08-14 12:21:26 +03:00
siivonek 85f6b00394 [isp] Add lfnst asserts. Fix error in MTS search. Fix chroma lfnst index when no coefficients present. 2023-08-14 12:21:26 +03:00
siivonek b9822398a0 [isp] Fix lfnst constraint checks when ISP is in use. Add some asserts. 2023-08-14 12:21:25 +03:00
siivonek 701257cdd2 [isp] Remove unnecessary code from forward dct 32. 2023-08-14 12:21:25 +03:00
siivonek 89db34d4e0 [isp] Use TR_MAX_WIDTH in ISP checks instead of parameter. 2023-08-14 12:21:24 +03:00
siivonek c4bc2d6b10 [isp] Limit ISP search to block size 32. Size 64 is not allowed. 2023-08-14 12:21:23 +03:00
siivonek 5713fbff1a [isp] Add ISP checks to search. LFNST can be used with ISP for larger blocks. Transform skip cannot be used with ISP. 2023-08-14 12:21:22 +03:00
siivonek 7282534879 [isp] Fix CI errors. 2023-08-14 12:21:22 +03:00
siivonek 01c4d1ddb0 [isp] Fix cabac issues. There are always four transform blocks even if there are only two ISP splits. Fix prediction issues. PDPC filter was applied when it should be disabled. Fix reference building issues. Left reference was built incorrectly for blocks with height 2. 2023-08-14 12:21:21 +03:00
siivonek b8e36bbc4a [isp] Fix storing cbfs for small ISP splits. Fix pdpc filtering. Cannot be used if width or height is less than 4. Fix dct related CI errors. 2023-08-14 12:21:20 +03:00
siivonek 99495c331b [isp] Fix some asserts to allow log2_dim 1 block sizes. Fix coefficient group scan order for small dimensions. 2023-08-14 12:21:19 +03:00
siivonek d39fddf0d8 [isp] Implement DCT for small blocks. 2023-08-14 12:21:19 +03:00
siivonek 910501012f [isp] Fix referene building for depth 2 blocks. Flip horizontal mode dimensions during prediction. Fix reference length during prediction when ISP enabled. 2023-08-14 12:21:18 +03:00
siivonek 7ba557af6b [isp] Fix luma cbf writing for ISP splits. Do not write luma cbf if first three splits had luma cbf 0. 2023-08-14 12:21:17 +03:00
siivonek a28e61eff7 [isp] Fix CI errors. 2023-08-14 12:21:17 +03:00
siivonek 4794104ecc [isp] Fix errors in reference building. Use cubic filter during prediction if ISP enabled. 2023-08-14 12:21:16 +03:00
Joose Sainio 662f31d61d [isp] Use correct coordinates for depth 4 chroma tu cost calculation 2023-08-14 12:17:36 +03:00
Joose Sainio 08942a5394 [tr-skip] fix transform skip flag writing 2023-08-14 12:17:35 +03:00
siivonek a261d4c5b3 [isp] WIP 2023-08-14 12:17:34 +03:00
siivonek 6340dfe4ce [isp] Fix mistake in pu_loc argument passing, was not used after passing. 2023-08-14 12:17:33 +03:00
Joose Sainio 88c33c0489 [lfnst] Fix lfnst constraint checking for the new coeff order 2023-08-14 12:17:33 +03:00
Joose Sainio e0e96068cc [lfnst] lfnst is not allowed for transform split 2023-08-14 12:17:32 +03:00
Joose Sainio cb7f9919e3 [jccr] Fix jccr coefficient copying 2023-08-14 12:17:32 +03:00
Joose Sainio 3e23fd0601 [cabac] fix cbf_y context for tr splits 2023-08-14 12:17:31 +03:00
siivonek 59292d8808 [isp] Add extra logic to reference building to accommodate ISP. Remove some asserts which were invalidated by ISP. 2023-08-14 12:17:31 +03:00
siivonek 33cd44f11b [isp] Fix chroma coeff writing for ISP. 2023-08-14 12:17:30 +03:00
siivonek d8d206365c [isp] Fix jccr coeffs. 2023-08-14 12:17:29 +03:00
siivonek 7398e58431 [isp] Fix coeff cost calculation. Coeff arrays were indexed wrongly. 2023-08-14 12:17:29 +03:00
siivonek d050efcb87 [isp] Fix error in last sig coeff function call. Height was not used. Fix cbf writing. Fix transform skip flag writing. 2023-08-14 12:17:28 +03:00
siivonek 33ae02aae0 [isp] Fix mistake in isp cbf writing. Loop index was increased twice. 2023-08-14 12:17:28 +03:00
siivonek 4a21039e23 [isp] Fix mistake in function declaration. 2023-08-14 12:17:27 +03:00
siivonek b8506c757c [isp] Convert functions to handle new coeff array order. Add function for getting coeff array subset. Fix assert. 2023-08-14 12:17:26 +03:00
siivonek 69dcb04c99 [isp] Use temporary coeff array when quantizing coeffs. After deriving coeffs, copy temp coeffs from linear order to correct arrays with cu order. 2023-08-14 12:17:26 +03:00
siivonek 0ae71feae4 [isp] Fix assert. 2023-08-14 12:17:25 +03:00
siivonek 2e8f008de4 [isp] Redo call hierarchy to include x, y coordinates. 2023-08-14 12:17:24 +03:00
siivonek 10f9b2be26 [isp] Keep lfnst constraint up to date during search. 2023-08-14 12:16:42 +03:00
siivonek 39f30563c5 [isp] Fix chroma width error when writing cu loc. Remove redundant IPS mode checks. 2023-08-14 12:16:42 +03:00
siivonek b53308f258 [isp] Fix mistake in setting cbfs. Skip stting if ISP is not used. 2023-08-14 12:16:41 +03:00
siivonek 56ebea7358 [isp] Set cbfs for isp splits after search. Add helper function for isp split number. 2023-08-14 12:16:40 +03:00
siivonek 510798cb3d [isp] Fix mistake in isp cabac write. Intra luma mpm flag bit was checking isp when it did not need to. 2023-08-14 12:16:40 +03:00
siivonek f86dc29ce7 [isp] Fix mistake in cost calculation. Remove some commented out code blocks. 2023-08-14 12:16:39 +03:00
siivonek bbb8faea98 [isp] Modify encode transform coeff func to handle non-square blocks, use cu_loc_t where possible. Fix mistake in mts idct generic. 2023-08-14 12:16:38 +03:00
siivonek 7062697beb [isp] Resolve TODOs. Make scan order tables const. 2023-08-14 12:16:37 +03:00
siivonek 93317cafa4 [isp] Write isp config bit to sps. 2023-08-14 12:16:37 +03:00
siivonek 182d0f4e66 [isp] Remove old_scan tables and related asserts. Fix coefficient group indexing. 2023-08-14 12:16:36 +03:00
siivonek f8641f7436 [isp] Fix assert. Implement coef cost calculation for isp splits. 2023-08-14 12:16:35 +03:00
siivonek ae0336fdfc [isp] Add non-square block handling to functions. 2023-08-14 12:16:34 +03:00
siivonek 031a758d6c [isp] Count isp cbfs. 2023-08-14 12:16:01 +03:00
siivonek 75175ee2e2 [isp] Fix isp search. 2023-08-14 12:16:00 +03:00
siivonek 8d914ce849 [isp] Implement coefficient encoding for isp splits. Make get_split_dim non static, it was needed elsewhere after all. 2023-08-14 12:15:59 +03:00
siivonek 573ecf80e3 [isp] Move can_use_lfnst_with_isp to intra.c. Remove duplicate functions. Move isp related functions from search to intra. Make isp_split_dim static. Move isp related defines from search to intra. 2023-08-14 12:15:58 +03:00
siivonek bcbd952dfd [isp] Add height handling to avx2 reconstruction. 2023-08-14 12:15:57 +03:00
siivonek 7c340fd92b [isp] Add height to inverse transform skip. 2023-08-14 12:15:56 +03:00
siivonek 318d925028 [isp] Add new convert_to_log2 table. Change all instances which used old convert_to_bit table to change dimensions into log2. 2023-08-14 12:15:55 +03:00
siivonek 6922157ed3 [isp] Fix quantization function calls. Some were not getting height as input. 2023-08-14 12:15:54 +03:00
siivonek 50ad91a94e [isp] Modify quantization functions to work with non-square blocks. 2023-08-14 12:15:53 +03:00
siivonek 31c8f1356f [isp] Add height to sig coeff group context calculation function. 2023-08-14 12:15:53 +03:00
siivonek 936256e750 [isp] Fix sig coeff flag context calculation function call. Width & height was swapped. 2023-08-14 12:15:52 +03:00
siivonek 9e7f4eac99 [isp] Change variable name 'type' to 'color'. 2023-08-14 12:15:51 +03:00
siivonek 09b905c6c4 [isp] Add height to get_tr_type function. 2023-08-14 12:15:51 +03:00
siivonek 8b7d573ae7 [isp] Add height to idct getter function. Check block dimensions in transform 2d functions. 2023-08-14 12:15:50 +03:00
siivonek 370bd07c55 [isp] Fix error in mts dct and idct. 2023-08-14 12:15:49 +03:00
siivonek 3a874ab5dd [isp] Comment out dct non square function. It is not needed since mts dct function will handle transform for non square blocks. 2023-08-14 12:15:49 +03:00
siivonek f9116441da [isp] Fix avx2 function call. Missing height parameter. 2023-08-14 12:15:48 +03:00
siivonek 6f756e831d [isp] Uncomment old scan order code to test against new one. Add assert to ensure old and new tables are the same. 2023-08-14 12:15:47 +03:00
siivonek c4d1f80f8f [isp] Fix error in scan order getter. Change define names to better reflect what they do. Add more accurate bookmark comments to scan order buffer table. 2023-08-14 12:15:46 +03:00
siivonek 8131e970e5 [isp] Modify existing scan table calls to use new getter. Add safety assert to getter. 2023-08-14 12:15:46 +03:00
siivonek 6ff9ae074e [isp] Add scan order getter. Add bookmark comments to scan order buffer. 2023-08-14 12:15:45 +03:00
siivonek 55d77c6b50 [isp] Add scan order tables for all possible block sizes. 2023-08-14 12:15:45 +03:00
siivonek 35271648db [isp] Fix some errors. Pass height to functions. Some WIP comments. 2023-08-14 12:15:44 +03:00
siivonek a9090c99b5 [isp] Fix error in inverse transform shifting. 2023-08-14 12:15:43 +03:00
siivonek cd7e091992 [isp] Fix mistake in transform if clause. 2023-08-14 12:15:43 +03:00
siivonek 6a3ddfd0bc [isp] Modify inverse transform to handle non-square blocks. 2023-08-14 12:15:42 +03:00
siivonek 626c9b02ea [isp] Modify transform and quantization functions to handle non-square blocks. Add strategy headers to CMakelist. 2023-08-14 12:15:41 +03:00
siivonek 06532dce02 [isp] Implement ISP search and partitioning. Add helper function for constructing cu_loc types. WIP stuff for transform. 2023-08-14 12:15:40 +03:00
siivonek 6236cc29be [isp] Fix avx2 function call. 2023-08-14 12:15:39 +03:00
siivonek ec4909095c [isp] Do not filter references if ISP is used. 2023-08-14 12:15:38 +03:00
siivonek 96df3ffd64 [isp] Change function calls to cu_loc_t. 2023-08-14 12:15:38 +03:00
siivonek 9406c5c31d [isp] Modify generic intra pred functions to handle non-square blocks. 2023-08-14 12:15:37 +03:00