Commit graph

157 commits

Author SHA1 Message Date
Ari Koivula f0e990905e Remove chroma mode "36".
- It's an unnecessary chore to handle this special case everywhere (it means
  chroma_mode == intra_mode). Better just to use the actual mode.
2014-05-14 19:56:35 +03:00
Ari Koivula 9147b7acbf Split residual quantization to separate luma and chroma function. 2014-05-14 11:19:48 +03:00
Laurent Fasnacht 6c6adf18c7 Refactor encoder_state 2014-05-07 11:47:31 +02:00
Ari Koivula 4490e8afd6 Remove depth dimension from picture->cu_array.
- It isn't used for anything anymore.

- It was used in the past to hold information during search, but now that
  information is held in lcu_t structs.
2014-04-28 10:18:22 +03:00
Laurent Fasnacht 19b1642aa2 Removed all cabac parameters (cabac is part of encoder_state) 2014-04-22 11:46:53 +02:00
Laurent Fasnacht 5fea5875a5 Huge refactoring
Split some parts of encoder_control into encoder_state
(idea: encoder_control is immutable)

Goal is to allow multiple substreams in the future.
2014-04-22 10:39:12 +02:00
Ari Koivula 0704c43836 Address clang-analyzer warning about undefined behavior in intra.
- Related to issue #35.
2014-04-17 18:43:08 +03:00
Laurent Fasnacht 3396264f3c Moved g_cur_lambda_cost into encoder_control.cur_lambda_cost 2014-04-17 12:00:21 +02:00
Laurent Fasnacht 4a9c239027 Remove g_bitdepth 2014-04-17 11:13:13 +02:00
Ari Koivula b35f33b3da Address warnings about unused values.
- Related to issue #35.
2014-04-16 18:05:03 +03:00
Laurent Fasnacht 78c579053a encoder_control should be const in nearly all the code 2014-04-14 10:56:06 +02:00
Ari Koivula 29787efbbc Fix whitespace.
Fix some whitespace issues from a merge.
2014-04-11 17:06:21 +03:00
Marko Viitanen de1c0b7e8d Fixed intra RDO to include mode bitcost 2014-04-10 16:28:41 +03:00
Marko Viitanen a657cf84d9 Insert most probable (predicted) intra modes to RDO search 2014-04-10 15:59:36 +03:00
Marko Viitanen 05169d9476 Added more modes to RDO mode selection in intra search
Now 8 best modes for sizes 4x4 and 8x8 are added to RDO checking and 3 for other block sizes as before, only applies when --rd 2
2014-04-10 15:20:49 +03:00
Ari Koivula 5fa5e01e05 Merge branch 'intra-cleanup'
Conflicts:
	src/intra.c
	src/intra.h
	src/search.c
2014-04-10 13:51:14 +03:00
Ari Koivula 40c2fa4d46 Change intra reconstruction to use the same prediction function as search.
- This fixes a bug with intra search. It sometimes used filtered reference
  pixels for 4x4 blocks leading to inaccurate cost estimate.
2014-04-10 12:09:19 +03:00
Ari Koivula d5c3ad7a2b Move intra prediction generation to its own function. 2014-04-10 11:27:15 +03:00
Ari Koivula 088dd9ab96 Clean up intra mode search.
- This changes the bitstream a little bit, because it changes the order in
  which the modes are tried and when two modes have the same cost the first
  one is chosen.

- Dst buffer was removes as it was no longer used.
2014-04-10 10:25:57 +03:00
Marko Viitanen 43ae0a3b9a Implemented RDO cost calculation to Intra modes 2014-04-10 10:25:20 +03:00
Laurent Fasnacht 8a14bd3b7b Remove cabac global variable 2014-04-04 14:26:40 +02:00
Ari Koivula 313466fdff Remove unused variables.
- Working towards issue #11.

- Either removed or redefined variables to not cause a warning.
2014-04-03 15:37:59 +03:00
Ari Koivula 462b75f0f3 Merge branch 'intra-NxN'
Conflicts:
	src/search.c
2014-03-12 15:08:39 +02:00
Ari Koivula 409b094acf Try and get NxN to work again.
- Moved NxN search to be done on the same level as other searches, as it's
  really not any different from 2Nx2N.
- Produces working bitstream but reconstruction is different.
2014-03-11 19:19:20 +02:00
Marko Viitanen aa0fb6ccad Store intra coding bit costs to CU structure 2014-03-11 12:15:50 +02:00
Ari Koivula 9dde96f25e Refactor search_cu_intra.
- Formatted.
- Renamed and consolidated variables.
- Removed unused x and y pos from intra_prediction function.
2014-03-07 16:25:25 +02:00
Ari Koivula 47af5207c5 Remove dead code and fix white space. 2014-03-06 18:35:17 +02:00
Marko Viitanen b5fca8b206 Added missing chroma ref pixel init and fixed pic width/height in intra ref border
intra_build_reference_border() was given wrong picture width and height on chroma.
Chroma top-left reference pixels were not copied on init_lcu_t()
2014-03-03 12:21:55 +02:00
Marko Viitanen f1882a0d82 Fixed CU type initialization and PU index calculations 2014-02-28 15:56:27 +02:00
Ari Koivula c05e91472d Fix intra reference border construction on LCU boundary. 2014-02-28 15:43:59 +02:00
Marko Viitanen 83e2079587 Fixed chroma stride in intra_build_reference_border() 2014-02-28 15:24:47 +02:00
Ari Koivula 19f67d4466 Fix LCU reference pixel indexing in reference border construction. 2014-02-28 14:22:13 +02:00
Marko Viitanen 9968fd46b0 Fixed chroma array index calculations in transform_tree() and intra_recon_lcu() 2014-02-27 16:42:32 +02:00
Marko Viitanen 7a1bf6b17d Intra fix: encode_transform_tree() call was missing and depth was not set 2014-02-27 10:53:01 +02:00
Marko Viitanen f788661a54 Fixed coeff flag derivation from NxN blocks 2014-02-26 16:54:49 +02:00
Marko Viitanen b4181dd398 Fixed problems with the new structure and commented out old search 2014-02-26 14:57:57 +02:00
Marko Viitanen 40138f082e Modified intra_get_dir_luma_predictor() to use the new structure
Also fixed intra_pred_ratecost() parameters and usage of intra_build_reference_border()
2014-02-26 12:38:30 +02:00
Marko Viitanen 1197e24b3f Modified intra_prediction() to use new structure and renamed intra_merge_cost() 2014-02-26 12:34:22 +02:00
Marko Viitanen 68b4e9e3db Modified intra_build_reference_border() to work on the new structure 2014-02-26 12:30:12 +02:00
Marko Viitanen d0a9c5e83b Implemented intra_recon_lcu() to reconstruct intra in the new structure 2014-02-26 12:28:24 +02:00
Marko Viitanen fb9e3ad1d2 Implemented proper intra mode cost calculation
This does not currently affect the search primary search defining the used block sizes, only the refining second intra search. Gain 1.9% BD-rate on All Intra 600f of BQMall QP 22,27,32,37.
2014-02-26 12:26:54 +02:00
Luca Barbato 0e67f6ffa5 intra: Do not use ambiguous - position
Clang warns about it.
2014-02-21 15:07:15 +01:00
Luca Barbato 934a4e3b88 whitespace: Drop trailing spaces 2014-02-21 15:03:35 +01:00
Ari Koivula 8e63dd29bb Fix compiler warnings for VS2010 /W4 in intra.c.
- Working towards issue #11.
- Removed intra_get_block_mode as unused.
- Removed unused parameters from functions. Many of them were remnants from
  earlier data structures and earlier features of HEVC that have been removed.
- Lots of implicit conversions from larger types to smaller ones. I tried to
  avoid turning all of them to explicit ones this time and opted for changing
  the original data type instead. Had to do it in few cases though to stop the
  changes from propagating too widely.
2014-02-14 17:15:54 +02:00
Ari Koivula 09b742796f Simplify intra reference border construction.
Doesn't change anything but makes the control flow more clear.
2014-02-10 16:19:17 +02:00
Ari Koivula b440c48d07 Fix intra reconstruction on the bottom of the frame.
When the CU was both at the bottom of frame and at the bottom of LCU,
uninitialized pixels were being used as reference.
2014-02-10 14:31:47 +02:00
Ari Koivula 7bb9c25447 Remove duplicate implementation of intra_build_reference_border.
The search_buildReferenceBorder was an ugly hack and a place for bugs to hide
that should never have existed. Now it doesn't.

The change reduces PSNR a little, but also reduces the bitrate, when the
expected result was to have no change in either. I'm guessing there was still
some bug in the search_buildReferenceBorder, but the bug could also be in
intra_build_reference_border. Will have to do more testing to be sure, but
having one place to look at will be better than having two.
2014-02-03 18:35:11 +02:00
Marko Viitanen ac8141b328 Fixed a bug in intra search and some memory leaks, changed version to 0.2.1
Intra search was using faulty border data and selected modes were a bit random. Around -0.5% (LP) and -2% (AI) change in BD-rate was seen in limited testing conditions.
2014-01-31 16:54:43 +02:00
Ari Koivula 5d12706510 Merge branch 'intra_NxN'
Conflicts:
	src/encoder.c

- Chroma RDOQ changes conflicted because I had moved the chroma
  quantization/dequantization to it's own function.
- Merged to master because I want my code to show up in github. =)
  All the old stuff still works, even though NxN doesn't work, so there
  is no reason not to merge anyway.
2014-01-29 17:04:41 +02:00
Ari Koivula e33655fde5 Remove debug stuff and disable intra NxN in preparation for a merge.
- Re-enable intra search based on reconstructed image.
  - This didn't have as much of an effect as I thought it would.
- Re-enable SAO and deblocking.
- Disable NxN searching. (4x4 luma coding is still broken)
2014-01-29 16:45:37 +02:00
Ari Koivula 9f855c16db Fix building of reference border in the bottom of the frame.
- Don't write to the top-left corner during construction of left border,
  because that is handled later that is handled later.
2014-01-29 14:22:31 +02:00
Ari Koivula ca454b67df Fix intra filtering for 4x4 luma blocks. 2014-01-29 14:22:31 +02:00
Ari Koivula 81b51f17d8 Reimplement intra_build_reference_border to support NxN.
New implementation uses precalculated tables to look up number of reference
pixels available in coded CUs. Otherwise it works just like the previous
version.

- NxN mostly works. Prediction appears to be almost correct but there is a
  slight error in the exact values, probably related to filtering.
2014-01-29 14:22:30 +02:00
Ari Koivula 71a996f50f Work on reconstruction and searching.
- Set part-size for Inter.
- Change to Intra Only mode for testing.
- Many small changes here and there. Should have been separate commits probably, but too late.
- Disable SAO and deblocking to be able to see problems with reconstruction better.
2014-01-29 14:18:01 +02:00
Ari Koivula 5e2f22f447 Try to fix intra prediction mode coding for NxN.
- Adjust predictor list to take modes from PUs in addition to 2Nx2N CUs.
- Change intra_get_dir_luma_predictor to take PU index instead of CU index.
- Comment prediction encoding now that I've had to look it up.
2014-01-29 13:52:17 +02:00
Ari Koivula 77339efa57 Add COPYING file and add boilerplate for copyright and GPLv2 to every file. 2014-01-24 12:48:48 +02:00
Ari Koivula a38a5fd647 Improve transform tree coding.
- Clean up code and comment.
- Change terminology to match H.265 specification where possible.
- Move transform splitting for depth==0 out of the coding part. It's not
  possible to do it here anyway because intra reconstruction is different
  if the transform is split.
- Add checking for transform hierarchy depth when coding split flag.
- Fixes bug with cu_data.tr_depth being set. The CU struct was being reused
  for inter coded CUs, which did not initialize the tr_depth.
2014-01-14 18:39:24 +02:00
Ari Koivula c3f4a628af Try to fix stuff related to NxN intra transform splitting. 2014-01-03 16:47:27 +02:00
Ari Koivula 4c84a35391 Add part size to intra_set_block_mode. 2014-01-03 13:01:13 +02:00
Ari Koivula 5083e6db10 Change cu_info.intra into an array to support NxN split. 2014-01-02 15:13:20 +02:00
Marko Viitanen 601d64f04f Merge branch 'merge-mode' 2013-10-25 15:40:58 +03:00
Ari Koivula c0f06fdb9e Change intra prediction to use SAD instead of SATD.
Inter uses only SAD and we don't have mechanisms for comparing these two.
2013-10-21 17:42:55 +03:00
Marko Viitanen c44f0ff540 Refactoring: all int16_t pixel info in intra to pixel typedef 2013-10-18 16:19:17 +03:00
Ari Koivula d74d4e4ef1 Refactor intra sad calculation functions.
- Move SATD (hadamart) code to it's own functions.
- Generate functions for fixed block sizes with macros, as if using templates.
- Define new interface with function pointer cost_16bit_nxn_func that different
  cost functions can use.
- Hide fixed size block cost functions with getters.
2013-10-16 17:09:03 +03:00
Ari Koivula b8dd664026 Move all 8-bit pixels into new pixel typedef. 2013-10-15 14:36:30 +03:00
Ari Koivula 04f1dde8a1 Fix overwriting of motion vectors during search.
- Makes everything besides search to only use the bottom most layer of the
  picture.cu_array structure.
2013-09-25 16:21:51 +03:00
Ari Koivula 8d5dd67f76 Refactor: Change name of CU_info.CU to cu_array. 2013-09-20 12:47:53 +03:00
Marko Viitanen 3a9449e5e0 Refactor: intra.c full cleanup 2013-09-19 16:21:53 +03:00
Ari Koivula adf619f9dd Refactor: Update file comments. 2013-09-18 18:01:20 +03:00
Ari Koivula de447ee18b Refactoring: intra.h renaming. 2013-09-18 15:13:45 +03:00
Marko Viitanen a018567854 Refactor: picture.h renaming. 2013-09-18 14:58:46 +03:00
Marko Viitanen 0bf4f65a55 Refactor: encoder.h renaming 2013-09-18 14:49:01 +03:00
Ari Koivula 44a5498e30 Reorder includes to avoid hidden dependencies.
- Includes of global.h have been moved to headers because most headers
  require stdint.h.
- Includes required by the header have been moved from the .c to the header.
- Spaces have been added between includes to distinguish classes of includes.
2013-09-18 12:29:23 +03:00
Ari Koivula 5acb280407 Change "ToDo" to "TODO".
Visual Studio recognized only TODO.
2013-09-09 14:22:53 +03:00
Marko Viitanen b3a724afb0 Added codes for inter coding 2013-08-02 16:35:30 +03:00
Marko Viitanen 06fd1d6fa9 Fix for sign bit hiding, not working yet 2013-06-12 15:41:57 +03:00
Marko Viitanen 4c1e015c5f Work on Deblocking filter 2013-05-22 17:27:15 +03:00
Marko Viitanen fd2e25727c Fixed transform split bug by moving intra prediction to the transform-quant loop 2013-05-21 15:57:22 +03:00
Marko Viitanen 2a70231c06 Fixed Chroma 4x4 2013-05-20 17:26:57 +03:00
Marko Viitanen 1b007efe43 Fixed bug with 8x8 split_flag coding. 4x4 chroma still produces wrong data. 2013-05-16 15:27:54 +03:00
Marko Viitanen 3942bbda51 Added short-term ref list parameters to allow HM10.0 to decode it properly 2013-05-07 10:36:04 +03:00
Marko Viitanen 003093b1ef Added inter.c/.h 2013-04-24 10:35:27 +03:00
Marko Viitanen e8f04b32f1 Bug fixes, this version produces bit-perfect output 2013-04-18 14:37:56 +03:00
Marko Viitanen 58ba4b306c Fixes for search-tree, almost working, some visible errors 2013-04-18 14:04:15 +03:00
Marko Viitanen ae506ceaf5 Moved all the intra-search functionalities to search-tree, does not work completely. 2013-04-17 17:08:52 +03:00
Marko Viitanen a737e8660c New search.c/.h files and changed department name on all files 2013-04-16 11:23:03 +03:00
Marko Viitanen 01c7f267d7 Added CPUID fetch assembly functions (x86 and x64) 2013-04-10 16:55:31 +03:00
Marko Viitanen 560917a532 Fixed 8x8 luma encoding and some of 4x4 chroma by disabling modes using different scanning order and changing 4x4 DST to DCT in chroma 2013-04-09 15:59:12 +03:00
Marko Viitanen 9888eaf9a4 Added and fixed x64 build 2013-04-05 14:55:58 +03:00
Marko Viitanen d5ab9f0437 Fixed partsize context model initialization 2013-04-04 15:08:28 +03:00
Marko Viitanen a059965855 Fixed DC-prediction (filtering) 2013-04-03 13:56:07 +03:00
Marko Viitanen d0be802ced Added filter.c/.h and deblocking functions 2013-04-03 11:05:07 +03:00
Marko Viitanen 55cc82925d Code refactoring to allow transform split (and 64x64 prediction with 32x32 transform) 2013-03-25 17:17:24 +02:00
Marko Viitanen 78894b05d1 Fixed dequantization and made preparations for smaller blocks. This version produces bit-perfect output. 2013-03-21 17:12:55 +02:00
Marko Viitanen fa4c4acbe5 Fixed transform subdiv context derivation and changed block-allocations to use LCU_WIDTH 2013-03-21 11:56:19 +02:00
Marko Viitanen 05bbd4daee Added support for 16x16 luma coeff coding and disabled PCM 2013-03-20 17:27:47 +02:00
Marko Viitanen ff5652609e Fixed intra filtering process for future use-cases 2013-03-19 17:12:43 +02:00
Marko Viitanen 73d6ed2ff3 Fixed all warnings and removed debugging codes 2013-03-19 16:23:33 +02:00
Marko Viitanen 08cc0e97ab Intra angular fixed, intra filtering. This version produces bit-perfect output. 2013-03-19 15:45:50 +02:00
Marko Viitanen 83b904e34d Clearing prediction data after picture is coded and fixed planar prediction 2013-03-15 10:30:42 +02:00