Commit graph

3744 commits

Author SHA1 Message Date
Pauli Oikkonen 6799019db0 Move fast coeff table to transform.h
Guess this is a more logical place for it
2020-10-29 15:20:27 +02:00
Pauli Oikkonen 4712ce5f59 Round the fast coeff result instead of flooring 2020-10-29 15:20:27 +02:00
Pauli Oikkonen 0fb09c9920 New filtered coeff weight by QP values 2020-10-29 15:20:27 +02:00
Pauli Oikkonen 9bf0cb27b1 Constrain fast cost estimation to QPs we have weights for 2020-10-29 15:20:27 +02:00
Pauli Oikkonen 24d487f553 New weights for 12 <= QP <= 42
Trained using MSU ultrafast settings now
2020-10-29 15:20:27 +02:00
Pauli Oikkonen 3e1c6d84b8 Fix issues in fast coeff estimation
Allow weight table to start from nonzero QP, and round weights to Q8.8
instead of flooring them
2020-10-29 15:20:27 +02:00
Pauli Oikkonen 5f91bda762 Use newer data for fast coeff cost estimation
Same training dataset, but this time only buckets 0...3 were used to
approximate the function, no sign/cg width bucket.
2020-10-29 15:20:27 +02:00
Pauli Oikkonen 2abd733199 Use unsigned min() to correctly clip -32768
If a coeff happens to be -32768 (0x8000), its 16-bit abs() is also
0x8000. It should ultimately be clipped to 3, so interpret absolute
values as unsigned instead to make that happen.
2020-10-29 15:20:27 +02:00
Pauli Oikkonen b93b90c0d7 Implement new fast coeff cost estimator in AVX2 2020-10-29 15:20:27 +02:00
Pauli Oikkonen 2f74a112b3 Try first lookup table based fast coeff estimation 2020-10-29 15:20:27 +02:00
Marko Viitanen 5338a9f0b6 [CI] Disable tsan and distcheck 2020-10-13 14:45:15 +03:00
Marko Viitanen 2db3a07b14 Prevent cu_sig_model_chroma array from being indexed over the limit 2020-10-13 14:14:57 +03:00
Marko Viitanen 2bd9bcdf28 [CI] Use QP 37 on one intra test 2020-10-13 14:06:59 +03:00
Marko Viitanen f4948dda6f Fix array size for bdpcm_mode[] 2020-10-13 12:51:20 +03:00
Marko Viitanen eb664fdcec [CI] Execute travis requirements install script in gitlab-ci 2020-10-13 12:39:31 +03:00
Marko Viitanen daf5050ff8 [CI] Disable werror and 10bit tests in gitlab-ci 2020-10-13 12:21:37 +03:00
Marko Viitanen 9e3e8f51f6 Change kvz_g_tc_table_8x8 from uint8_t to uint16_t to fit all the values 2020-10-13 12:05:27 +03:00
Marko Viitanen 26f4f45c6d Use correct pred_mode cabac models -> fixes inter cabac bits 2020-10-13 12:04:31 +03:00
Marko Viitanen 5798eea6f8 Merge branch 'CI-fix' 2020-10-13 12:02:51 +03:00
Marko Viitanen 1986130ab6 [CI] Only enable intra testing for now 2020-10-13 12:02:18 +03:00
Marko Viitanen 9b0a45a8b2 [CI] Use correct test set 2020-10-09 09:43:13 +03:00
Marko Viitanen 1c5d944ef4 [CI] Disable werror for now 2020-10-09 09:40:57 +03:00
Marko Viitanen 5a6806cbf7 [CI] Limit testing parameters to those that work 2020-10-09 09:37:15 +03:00
Marko Viitanen 25b5ad72d7 [CI] Replace HM with VTM and remove tests that are not needed 2020-10-09 09:28:58 +03:00
Marko Viitanen 3c7eb55292 Disable output of cabac debug when in "count only" mode
- Some code cleanup
2020-10-09 08:45:43 +03:00
Marko Viitanen fa25621c77 Force certain intra modes off 2020-10-09 08:44:40 +03:00
Marko Viitanen 54b8fd054d Fix Chroma QP scaling issue 2020-10-02 15:40:23 +03:00
Marko Viitanen 11229997b6 Fix NAL header layer_id 2020-10-01 11:10:40 +03:00
siivonek bc1206a4d3 Define qp_delta_min & max in global.h instead of calculating them locally. 2020-09-29 13:46:27 +02:00
siivonek eeb2840505 Add examples folder and readme.md for examples. Add example roi file. 2020-09-28 17:15:12 +02:00
Marko Viitanen ac2032eb65 Fixing P/B frame headers and debug output formatting 2020-09-28 14:58:07 +03:00
Marko Viitanen bddfb47a55 Merge remote-tracking branch 'remotes/kvazaar_github/master' 2020-09-25 11:49:11 +03:00
Marko Viitanen 551a3991cf Cleanup headers 2020-09-24 09:31:44 +03:00
siivonek d5fe6c3737 Remove xfail for VAQ from 10bit-test since it should be working now. 2020-09-22 20:15:23 +02:00
siivonek 0f3ef786b9 Modify delta QP range assert so it will work with any valid bit depth. Modify VAQ code so it will clip the QP to a proper range which is dependent on bit depth 2020-09-22 20:15:23 +02:00
siivonek fe6f93a951 Fix delta QP range check assert. Add separate asserts based on bit depth. 2020-09-22 20:15:22 +02:00
Marko Viitanen 449975b0fb Fixed cubic filter usage in intra angular modes 2020-09-21 14:58:34 +03:00
Joose Sainio 8143ab971c Merge branch 'stats-files'
# Conflicts:
#	src/cfg.c
#	src/cli.c
#	src/kvazaar.h
2020-09-16 09:25:00 +03:00
Joose Sainio 1c06bd7f3d Fix POC to be correct for all GOPs and Intra periods, fix issue with vaq 2020-09-14 14:25:48 +03:00
Sami Ahovainio 4d87fb2397 fixed potential out of bounds iteration 2020-09-10 12:59:39 +03:00
Sami Ahovainio 7e2bbcfb64 Merge remote-tracking branch 'remotes/origin/y4m' to master 2020-09-09 16:39:12 +03:00
Sami Ahovainio 245d0cd2cf updated readme 2020-09-09 16:14:04 +03:00
Sami Ahovainio 5d521a2444 Added option to force yuv as file format and made the options and file endings case insensitive 2020-09-09 16:05:59 +03:00
Joose Sainio 984e5f72f1 update documents 2020-09-09 13:49:13 +03:00
Joose Sainio 3fb8b7ebc6 Add --stats-file-prefix option
When the option is defined with an option four files prefixlambda.txt,
prefixqp.txt, prefixdist.txt, and prefixbits.txt that have the corresponding
data for each ctu. This is a debug feature.
2020-09-09 12:35:47 +03:00
Sami Ahovainio 5b0d510647 updated readme 2020-09-07 17:07:58 +03:00
Sami Ahovainio 84cabd9c20 Fixed sign match 2020-09-07 15:39:31 +03:00
Sami Ahovainio d691849594 Added frame header reading for both read and seek functions 2020-09-07 15:31:08 +03:00
Sami Ahovainio cbcee67821 y4m start header parsing ready 2020-09-07 15:31:07 +03:00
Joose Sainio c10b841e7c Merge remote-tracking branch 'remotes/origin/fix-sao-parameter' into master 2020-09-07 13:10:36 +03:00